Voluntary Application: UK Health Security Agency (UKHSA)

Winner of the Award for Statistical Excellence in Trustworthiness, Quality and Value 2022 for the COVID-19 dashboard.

“The COVID-19 dashboard team are truly delighted to receive this endorsement from the RSS and the OSR for our work. We put honesty and transparency at the heart of this project, to provide COVID-19 data to anyone who needs it. We know that our users value this data and it is needed to enable people to go about their daily lives. We are therefore proud to demonstrate how we uphold the trustworthiness, quality and value principles so that the dashboard can be used with confidence. We are very grateful for all the user feedback and to our team for their dedication to producing such high-quality dashboard during difficult circumstances.”

Clare Griffiths, Head of the COVID-19 Dashboard at the UK Health Security Agency

Who are the UK Health and Security Agency?

The UK Health Security Agency (UKHSA) is responsible for protecting every member of every community from the impact of infectious diseases, chemical, biological, radiological and nuclear incidents and other health threats. UKHSA provide intellectual, scientific and operational leadership at national and local level, as well as on the global stage, to make the nation’s health secure.

What is the COVID-19 dashboard?

The UKHSA created the COVID-19 Dashboard as one of the main tools to establish rates of COVID and vaccination across the UK, that quickly became one of the main and trusted sources of data on the pandemic. It has been used extensively by decision makers and wider, experiencing over 1 million users per day during peak times.  These users include government ministers and other decision makers who rely on this information to help shape the UK’s COVID response, as well as the public making decisions about their daily lives.

These statistics remain high-profile, are complicated to produce, receive daily scrutiny and much wider discussion, so must fully demonstrate trustworthiness, quality and value.  The UKHSA team continue to modify the dashboard, to undertake user engagement and make improvements. The dashboard data continues to remain incredibly important to people as we move to the living with COVID phase of the government’s strategy.

Applying the Code

Providing COVID dashboard users with trustworthiness, Quality and Value (TQV) was essential from that start of pulling this data together. UKHSA are proud of the commitment made to the Code and have prominently published a statement of compliance on their website so it’s visible and easy to find.

Trustworthiness, Quality and Value

The dashboard statistics are developed by a team of analysts and data scientists at UKHSA, in collaboration with devolved administrations, NHS England, NHS Test and Trace and the DHSC, under the guidance of the Head of Profession for Statistics. Data are compiled from reputable sources, including government and NHS departments in England, Northern Ireland, Scotland and Wales.

This presents challenges, for instance, making sure that users can easily compare data and understand differences in how nations produce data. The dashboard team maintain close contact with these organisations, to ensure our work is aligned. Data must be current, so the dashboard is updated every weekday: a big task for a small team.

Engaging with users has been key to the dashboards success and getting the data correct, with over 40,000 people involved in the last survey. UKHSA know the public like the open data and the API data download options the dashboard provides, with over 1 million downloads each week by individuals for their own data purposes.

Collecting user feedback through the survey will continue, along with interviews, use of analytics and email contact, with nearly 30,000 interactions and communications so far. This has enabled UKHSA to maintain very high levels of trust in the data, continually make improvements and add new features.

Voluntary Application: Demonstrating the value of the COVID-19 Dashboard through the Code of Practice for Statistics

Background

The Coronavirus (COVID-19) in the UK Dashboard is the official UK government website for data on COVID-19. The pandemic is constantly changing, so to help people make quick decisions we are producing the data at pace. The dashboard isn’t formally badged as official statistics (due to, for example, the need for a speedy turnover and ministerial involvement in decision making), but it does meet many of the official statistics criteria, so we decided to voluntarily apply the Code of Practice for Statistics.

This signals that our statistics meet the 3 ‘pillars’ of the Code:

  1. they have public value
  2. they are high quality
  3. they are trustworthy

In this case study we focus on the value pillar, which states that statistics and data should:

  • be useful
  • be easy to access
  • remain relevant
  • support understanding of important issues

The five principles of the ‘Value’ pillar are:

  1. Relevance to users
  2. Accessibility
  3. Clarity and insight
  4. Innovation and improvement
  5. Efficiency and proportionality

Benefits of applying the code

Applying the code voluntarily shows our commitment to transparency and our desire to increase user trust in the dashboard. Our published statement of compliance helps users to understand the processes involved. Its application not only benefits users, but also helps us as producers to demonstrate what we are doing well and identify areas for improvement.

Available and well used by all

The dashboard is extremely well-used, with:

  • 1 million unique users per day, on average
  • up to 70 million daily hits

Peak time for use is 4pm when the dashboard is updated. There can be around:

  • 30,000 concurrent hits per minute right before 4pm
  • 250,000 to 300,000 per minute as soon as the data are released

We make as much data available on the dashboard as possible, without compromising quality or confidentiality. Our API and data download options allow people to use the data for their own purposes. Again this is highly popular, with downloads averaging around 1.3 million per day.

We want the dashboard to be accessible to as many users as possible. It is simple to use, helping anyone to navigate the data and visualise trends over time and across geographic regions. We commissioned an accessibility audit in July 2020 and provide a full accessibility statement on the dashboard. For example, users can:

  • read simple summaries of the data
  • access data in a variety of formats. The dashboard has many visual elements, but all data is also provided in text form.

We continually seek to improve functionality and make sure we comply with the Web Content Accessibility Guidelines version 2.1 AA standard. We list non-compliances for transparency, and, where possible, explain how we will fix these or why we don’t comply.

Innovative and continually improving

We want to provide the most reliable statistics and ensure the best user experience possible. We always seek user feedback to help us develop and improve the dashboard. We collect this via:

  • individual research sessions, with over 100 conducted to date
  • emails to our feedback mailbox, which is monitored daily and receives many hundreds of email each week
  • a programme of regular user surveys. The latest had over 38,000 responses. Our next survey will be in early 2022
  • analytics, for example, to find out which parts of the site are most popular

Feedback helps us to improve data presentation to make sure it is easy to understand and meets the needs of different users. In response to user feedback, in 2022 we plan to research and improve how people experience the site on mobile devices.

Changing the dashboard to meet the needs of the pandemic – home page comparison

As the pandemic continues to evolve, user interests and needs also change. We constantly reassess our data and design to meet these shifting requirements. For example, in mid-2020 there was high interest in testing capacity, so test data was prominent on the dashboard. However, interest has now shifted and is more focused on the vaccination programme (in particular boosters since the emergence of the Omicron variant). To meet this changing need, we have made vaccinations more prominent, and expanded our data to include trends, maps and local data (via postcode search functionality). We are also in constant collaboration with devolved administrations to improve the extent of UK-wide data.

August 2020: Tests prominent, no postcode search for local data, figures for whole pandemic, no trends.

August 2020: Tests prominent, no postcode search for local data, figures for whole pandemic, no trends.

February 2022: Vaccinations prominent​, postcode search for local data added​, tests moved down the page​, trends added: shown with figures, arrows and colour.

February 2022: Vaccinations prominent​, postcode search for local data added​, tests moved down the page​, trends added: shown with figures, arrows and colour.

Meeting different user needs

We present data in a variety of ways to suit different users and help aid understanding. This means the data are of value to people looking at it both for personal interest (around 89% of our audience) and for professional reasons.​

We provide:​

  • visualisations, including graphs, maps (choropleths), and arrows to indicate trends​
  • simple headline figures in web and PDF formats​
  • full data tables​
  • data downloads in 4 different formats (CSV, JSON, JSONL, and XML)​
  • multiple Application Programming Interfaces (APIs)​
  • supplementary downloads, such as population denominators

We provide data and metadata at the greatest level of detail possible for those who need it, and simpler overviews for users who need less detail, with explanations provided in plain English.​

Open, efficient and technologically advanced

We are open and transparent about dashboard developments and any issues that occur. You can find details on the latest updates, changes and data issues on the ‘What’s new’ pages. Some of the more recent additions to the dashboard are: ​

  • virus tests (all virus tests and lab-based virus tests) by specimen date added for England plus regions and local authorities
  • age breakdowns and local data added for booster or 3rd dose vaccinations
  • new metrics documentation page listing all current and historical metrics (searchable by name, category, type or availability by area type)

We strive to make our work ‘low burden, high benefit’. For example, we use existing datasets and add value to collections that have been stood up for COVID (for example, hospital data) by reusing the data and presenting it to the public. We use leading edge technology and data pipelining from numerous data sources.

Value means more than just the number of people viewing the dashboard. We know this data is of huge public interest, but we also want to make sure we are presenting it as clearly and accurately as possible. We use rigorous statistical and design processes to make sure we are giving people the information they need in a way they can easily interpret. We engage with the people who use the dashboard as much as possible, to make sure we are not making assumptions and that we’re addressing their changing concerns in this fast-moving landscape.

We want our Dashboard to be valued by all users. Applying the Code pillars allows us to feel confident that our work is easy to access, remains relevant, and supports understanding of important issues in relation to COVID-19. The trust we have built is evident in the volume of use and overall popularity. As the pandemic evolves our Dashboard will also change, but what will remain constant is our willingness to ensure our statistics meet the Code.