UK Statistical System UK Statistics Authority Office for Statistics Regulation
Case studies Quality Assurance

Using independent reviews to inform assurances around quality

This is a case study for Principle Q3: Assured quality.

HMRC took an in-depth look at its approach to quality after it found a significant error in its Corporation Tax statistics. HMRC’s action in inviting an independent review of their data quality management approach is an example of best practice for other statistics producers.  

Corporation Tax is levied on the taxable profits of companies and makes up about 9% of HMRC’s total tax receipts. HMRC’s Corporation Tax statistics provide annual data on receipts and liabilities, obtained from an administrative data source.  

HMRC found an error in its published Corporation Tax receipts statistics in 2019, which led to substantial revisions across these statistics, affecting the period from April 2011 to July 2019. HMRC carried out a range of activities to assess this issue and identify mitigating actions to take to improve the quality of its statistics, including inviting the OSR to carry out a review of the principles and processes underpinning the quality of HMRC’s official statistics. OSR carried out a review of HMRC’s quality management approach and published its findings in April 2020.   

The review primarily focused on source data quality and the importance of statistics producers understanding the nature and quality of the data they work with. The review report contained nine recommendations, which included producing process maps of end-to-end processes, developing Reproducible Analytical Pipelines, and reducing HMRC’s suite of publications.  

The expectation is that these changes will improve the quality of HMRC’s statistics going forwards, for example by making sure analysts fully understand the data they are working with. All nine recommendations were welcomed by HMRC’s senior leaders, and a programme of work has been designed to implement the recommendations in 2021 to 2022 and beyond. This work has already increased HMRC’s level of assurance around the quality of its data and led to further improvements. 

In asking OSR to carry out an independent review, HMRC showed a proactive and open approach to strengthening data quality. The review and HMRC’s response to it, emphasise the importance of having analytical leaders that transparently campaign for and support changes and innovations that can enhance the quality of statistics. This provides an example for other producers who might wish to inform their own assurances around the quality of the data or the statistics that they produce, through periodic or systematic, independent reviews. 

Improving quality assurance and its communication to aid user interpretation

This is a case study for Principle Q3: Assured quality.

The Department for Work and Pensions (DWP) publishes statistics on new National Insurance number (NINo) registrations to adult overseas nationals, on a quarterly basis. In 2017, following an assessment by OSR, the statistics had its National Statistics designation suspended. Since then, DWP have implemented a range of improvements leading to re-designation of the statistics in November 2020. One of the areas of greatest improvement is their quality assurance processes and how they communicate these procedures with users.

The team have worked hard to review their quality assurance processes and apply guidance from OSR’s Quality Assurance of Administrative Data (QAAD) toolkit, as discussed in their Quality report. Applying guidance from QAAD supports good practice in monitoring the quality of data over time and in identifying data quality issues, whilst the Quality report itself demonstrates transparency about the quality assurance approach taken.

More recently, the production team have focused on harnessing aspects of Reproducible Analytical Pipelines (RAP). DWP have a separate dedicated team focused on introducing RAP across the department. Working closely with them, the production team – recognizing the benefits associated with quality assurance, streamlining and automating the production of the publication – have started to adopt many of the RAP principles. This has also involved team members upskilling and developing their programming skills. These are DWP’s first statistics to be produced using RAP and the team continues to develop and further integrate RAP principals into its production processes.

To further explain the quality assurance processes, the administrative data and the strengths and limitations of the statistics, an informative background and methodology document is available for users. This includes illustrative figures of the data journey – showing how different data sources fit together – and flow diagrams documenting the steps in the quality assurance process.  The inclusion of these helpful diagrams, alongside descriptions and further detail within the text, offers users a clear explanation of the strengths and limitations of the data, and enables an understanding the underlying data and processes.

This case study shows how DWP has made improvements to how it assures the quality of its NINO statistics and how it communicates this assurance transparently to aid users in their appropriate interpretation. The enhanced quality assurance processes and their clear communication through new documentation, reflect a positive and engaged approach to enhancing the quality of the statistics and improvements to their overall public value.