Value

Enhancing insights and coherence through collaboration

This is a case study for Principle V3: Clarity and Insight

While the Department for Work and Pensions (DWP) has responsibility for administering most benefits available in Great Britain, some benefits such as Tax Credits and Child Benefit are administered by HM Revenue and Customs (HMRC).

In March 2020, DWP and HMRC produced a new joint-release for the first time on children in low-income families (CILIF) at a local area level. This release has replaced DWP’s Children in out-of-work benefit households and HMRC’s Personal tax credits: Children in low-income families local measure releases. The new statistics provide a more coherent picture of children in low-income families by drawing together administrative data from both producers to provide insights on benefits, tax credit and employment incomes within families from which local area estimates of children in low-income families are published. From 2021, this release is now solely produced by DWP.

DWP also produces Benefit Combination statistics, which offer a picture of the number of individuals claiming at least one benefit as well as the number of claimants for each combination of benefits. Currently users need to look at statistics produced by both the DWP and HMRC to gain a complete picture of how many families and households are claiming benefits.

DWP and HMRC have been working together to develop a joint publication which would bring together the Tax Credits and Child Benefit statistics with those produced by DWP. There have been challenges with delivering this work due to the difference in timing of the data feeds that DWP and HMRC work with. However, they are working towards producing annual statistics on the numbers of individuals claiming common combinations of benefits which DWP and HMRC are responsible for, on a common snapshot date.

The Heads of Profession for Statistics in DWP and HMRC have discussed this joint publication and are supportive of the work to develop the experimental Benefit Combinations statistics. In the interim, DWP has provided details in the DWP benefit statistics background note of benefits administered by HMRC and signposted to relevant HMRC statistics. While work is still ongoing to develop the joint publication, it is great to see the joint focus on aiding public understanding of benefit provision by developing these statistics.

This example shows how producers can work together across departmental boundaries to make greater use of existing data sources through linking, and the benefits this brings in terms improved insights and coherence across a statistical topic.

Using Reproducible Analytical Pipelines (RAP) to improve statistics

This is case study for Principle V4: Innovation and improvement

In 2021, OSR published its review on Reproducible Analytical Pipelines: Overcoming barriers to adoption. The Reproducible Analytical Pipeline, also referred to as RAP, is a set of principles and good practices for data analysis and presentation.

RAP was developed by statistics producers in the Department for Culture, Media and Sport and the Government Digital Service in 2017 as a solution to overcome several problems: in particular, time-consuming and error-prone manual processes, and an overreliance on spreadsheets and proprietary software for data storage, analysis and presentation. RAP combines modern statistical tools with software development good practice to carry out all the steps of statistical production, from input data to the final output, in a high quality, sustainable and transparent way.

A minimum standard of RAP was developed by the Best Practice and Impact Team (now the Analysis Standards and Pipelines (ASAP) team) which are:

Peer review to ensure the process is reproducible and identify improvements
No or minimal manual interference, for example copy-paste, point-click or drag-drop steps – instead the process should be carried out using computer code which can be inspected by others
Open-source programming languages, such as R or Python, for coding so that processes do not rely on proprietary software licenses and can be reproduced by statistics producers and users
Version control software, such as Git, to guarantee an audit trail of changes made to code
Publication of code, whenever possible, on code hosting platforms such as GitHub to improve transparency
Well-commented code and embedded documentation to ensure the process can be understood and used by others
Embedding of existing quality assurance practices in code, following guidance set by organisations and the GSS

These fundamental principles that form the basis for the minimum standard can be further enhanced – for example by writing code in modular functions that allow for reuse, or introducing unit tests to ensure that code works as expected. It is also important to note that adopting RAP principles is not necessarily about incorporating all of the above – implementing just some of these principles will generate valuable improvements.

RAP benefits – enabling innovation and improvement in official statistics – the ONS Centre for Crime and Justice (CCJ)

The Nature of Crime data tables produced by the Centre for Crime and Justice (CCJ) at ONS previously relied heavily on Excel and SPSS. To reduce manual effort, save time and improve reproducibility, the CCJ replaced the existing process with R and python code and introduced Git for version control.

Implementing RAP principles resulted in a significant reduction in the time taken to produce the statistics: what was originally three weeks’ worth of work for thirteen analysts was reduced to under an hours’ work for one. The CCJ were also able to create new analysis more quickly (as an example, it took an hour to add nine new tables to the python pipeline).

With the time saved, the CCJ focused on providing more value for users – publishing historic time series, adding more measures and granularity to the tables, and developing its survey processes to provide new crime estimates about COVID-19. The team adapted the code for this project in order to automate the production of other statistics, such as those on violent crime. Overall, implementing RAP allowed the CCJ to continue to meet its existing output commitments whilst freeing up resources to focus on meeting user needs.

The code for the crime tables is available on GitHub and the team has blogged about its RAP transformation.

Planning how to implement RAP principles – Statistics producers should be empowered to develop RAPs themselves

The process to achieve the above results involved demonstrating the efficiency and quality improvements to senior leaders at the CCJ who then established a team to deliver further RAP developments. With agreement from their line managers, the members of staff who were interested dedicated two days a week to this team. Support from the Deputy Director and other senior leaders was essential in protecting this time commitment and prioritising development work among competing priorities. This level of senior support also meant that analysts felt more able to get involved in the project in the first place.

To support the development work, the Good Practice Team (GPT), now ASAP provided mentoring and training. This helped to embed RAP knowledge and skills within CCJ. Despite some initial apprehension about implementing RAP, the team members became confident in the new skills they developed and felt proud of their work and have now gone on to create and share their own crime_analysis package. The CCJ applied this approach to offering mentoring internally without the support of GPT and continued to focus on skills development across the division. To illustrate this, CCJ are now using a pair-programming technique to quality assure code and have created a bespoke RAP learning pathway specific to the data and table production processes for the team.

This example shows how producers can enable innovation and improvement in official statistics when they are empowered to develop RAP in their areas. With commitment and support from senior managers to implement RAP principles, the team have been able to continue to meet its existing output commitments, while using its newly freed up resources to focus on meeting new user needs.

Providing new statistical insights during the pandemic

This is a case study for Principle V3: Clarity and Insight.

The Government Statistical Service (GSS) has been proactively responding to address user needs for both new data and enhanced insight during the pandemic. For example, HMRC statisticians have actively sought to answer society’s key questions about economic changes in a timely way through the preparation, production, and publication of new statistics on its Coronavirus Job Retention Scheme (CJRS) and its Self-Employment Income Support Scheme (SEISS). And the ONS has adapted its Opinion and Lifestyles Survey (OPN) to provide new insights on the social aspects of the pandemic, through its Coronavirus and the Social Impacts on Great Britain.

Economic insights from HMRC’s worker support schemes

The CJRS is a scheme for employers who can claim support with wage costs for employees put on furlough during the pandemic, while the SEISS supports self-employed individuals. Before the statistics were developed, HMRC started Tweeting daily information on both schemes in response to public requests for information. Simultaneously, HMRC quickly developed Experimental Official Statistics, to transparently provide useful information on the status of the schemes.

The new CJRS statistics include a headline time series and detailed statistics on jobs furloughed at the end of each month, with analysis by employer size and sector. Also covered is analysis of jobs furloughed by geography, employees’ age and gender and breakdowns by type of furlough (full and flexible). To produce the statistics, HMRC combined data from CJRS claims with data from their Pay-As-You-Earn (PAYE) Real Time Information (RTI) system. HMRC has further developed the statistics in response to user feedback and most recently added detailed industry breakdowns and figures for the total number of jobs put on furlough at any time since the start of the scheme.

HMRC’s February 2021 SEISS publication covers the third SEISS grant which was awarded by HMRC up to December 31, 2020. The information presented includes age of claimant; gender; sector of self-employment activity; and geography, with further breakdowns providing additional insights.

HMRC are open with users about aspects of uncertainty in estimates labelling the statistics and analysis by labelling them with a frank summary to ensure appropriate interpretations by users. The statistics are released in a timely manner at an interval that meets user needs, for both demographic and geographical breakdowns.

Social insights from the Opinions and Lifestyles Survey (OPN)

The OPN previously operated as a monthly telephone and online survey of British households, providing timely and relevant insights to its users. During the pandemic, it became an important source of information for understanding the social impact of the pandemic.

From 20 March 2020, the OPN became weekly and each week, some of the survey’s questions were changed to reflect changing circumstances and priorities. Since then, estimates measuring the impact of the pandemic on people, households and communities in Great Britain have been published on a weekly basis.

With a user group of a wide range of government departments, academics and charities providing input into the questions asked, the survey has been used to rapidly and flexibly provide information on areas of user interest such as: people’s compliance with government measures to stop the spread of COVID-19; people’s experiences of home-schooling and working from home; as well as people’s well-being and attitudes towards vaccination as the pandemic has progressed.

OPN summary results are presented with breakdowns by age, sex, region and country, and the published data sets include confidence interval s to enable their appropriate interpretation.

Summary

These examples demonstrate how the GSS is responding to the need for new statistics to enhance public insight and understanding around the economic and social aspects of the pandemic. HMRC have shown an understanding of the needs of different types of users, and have brought clarity and insight to the extent of the UK government’s economic response in support of workers during the pandemic. And by adapting the Opinion and Lifestyles Survey to focus on the social impacts of coronavirus, ONS has also proactively responded to the changing data needs of users and helped to provide robust and timely insights into public’s attitudes around coronavirus and government responses to it.