How the Code of Practice for Statistics can help support evaluation
The Code of Practice for Statistics is framed around three pillars: Trustworthiness, Quality and Value which are universal to all uses of data, statistics, and analysis, whatever the setting. Here’s an explainer about how these pillars can help guide evaluation more broadly – think TQV.
We’ve produced this explainer because there is a lot of interest in and ambition for the role that evaluation can play in policymaking. This is extremely welcome. To make a success of it, we’d encourage thinking of evaluation not just from a technical perspective, but also how to be ambitious for how evaluation is published and communicated. In doing so, we draw on the Code of Practice for Statistics, which underpins the publication of statistics by Government. We break these ambitions down into three areas:
- Ambitions for the use of evaluation
- Ambitions for the design of evaluation
- Ambitions for the impact of evaluation
The Magenta Book gives detailed guidance on how to conduct evaluation.
1. Ambitions for the use of evaluation
A culture of evaluation supports effective policy and relies on openness: being open to learning what works and what doesn’t, as well as open about the use of evaluation and releasing information in an orderly way so others can use it:
- publish information about the evaluation ahead of doing it
- publish the results of all evaluations in a managed way
- publish the data and methods so others can scrutinise and replicate the analysis
The Trustworthiness pillar can help you and your organisation in being open.
Trustworthiness: confidence in the people and organisations that produce statistics and data
Trustworthiness results when the people, systems and processes within organisations enable and support the production of data and analysis.
It will come from the organisations being well led, well managed and open, and the people who work there being impartial and skilled in what they do.
Showing you are trustworthy means being truthful, impartial, and independent – free from vested interests.
Information is shared responsibly and with integrity – being open and transparent about your decisions, plans and progress.
It means you look after people’s information ethically and securely, manage data in ways that are consistent with relevant legislation, and serve the public good.
2. Ambitions for the design of evaluation
Design choices lie at the heart of excellent evaluation. High-quality evaluation happens when:
- the evaluation question is well defined
- the design is proportionate for its purpose and the scale of the policy
- the right data and robust methods are selected, tested, and explained
The Quality pillar can help you in making these design choices.
Quality: data and methods that produce assured statistics
Quality means that statistics meet their intended uses, are based on suitable data and methods, and are not materially misleading.
Quality requires skilled professional judgement about collecting, preparing, analysing, and publishing statistics and data in ways that meet the needs of people who want to use the analysis.
The organisation should communicate the quality of the data and methods chosen to its audiences.
A commitment to quality is shown by considering and describing how the evaluation sources and selects data and chooses and tests methods to ensure their suitability. It is reflected in seeking expert review at key stages and by informing users about the quality and limitations of the evaluation.
3. Ambitions for the impact of evaluation
Impactful evaluation addresses the right questions and ensures the correct understanding of the evidence. It means that:
- there is a clear basis for the evaluation informed by stakeholder insight
- the evaluation clearly finds at an early enough point whether the policy is beneficial
- the findings are clearly and accessibly communicated with the audiences in mind
The Value pillar can support you in ensuring your evaluation is impactful.
Value: statistics that support society’s needs for information
Value means that the analysis is useful, easy to access, relevant, and supports understanding of important issues by effectively communicating evidence as required by various audiences.
The users and the intended use should be at the centre of the evaluation, by understanding the research question and acting on the insight.
Published data and analysis are made equally available to all, being open about the nature of the evaluation and how it serves the public good. It enables its replication and reuse by others.