When we talk about evidence, we recommend keeping the following two types of evidence in mind (WHO 2021):
1. Scientific or research evidence:
Is produced systematically through a formal, standardised research process following certain methodological standards. It can include primary research, synthesis of existing research (secondary research), and other products and guidelines based on evidence such as guidelines (tertiary research). Scientific evidence can be numerical (quantitative) or descriptive (qualitative), using a range of methodologies.
2. Experiential evidence
Constitutes more informal knowledge based on the opinions and experiences of people who work or live in the area being analysed. This type of evidence is often expressed in the media, expert reports, personal anecdotes, qualitative interviews, group discussions or deliberative dialogues.
Both types of evidence can be valuable and answer different research questions. Their relationship is complementary. For example, experiential research can highlight an area to focus scientific research on, or it can validate or question the scientific research already carried out.
Typical research questions that can be answered by each respective evidence type (Superu 2018):
Scientific or research evidence
- Does the evidence validate, or accurately describe, your initial hypothesis, the target population and/or the drivers for change?
- Is there evidence for potential programmes, practices or policies being effective in addressing the issue you want to resolve?
- Are there well-designed studies or evaluations available to support or validate the success factors or outcomes you are seeking?
- What positive, negative or unintended effects has a programme, practice or policy had on changing behaviour or outcomes?
- Is there any implementation guidance available? What does the evidence say about the resources, processes and capacity needed to successfully implement a (change in) policy?
- What are the characteristics of the people the policy will serve? How and to what extent are the people affected by the issue, problem or opportunity?
Experiential evidence
- How does the policy approach reflect the experiences and knowledge of the people it aims to serve?
- What can a target group’s earlier experiences tell you about the acceptability, importance and appropriateness of the policy or programme to them?
- What can the experiences and knowledge of stakeholders tell you about what has previously worked or not worked with the target group?
- What common goals do the stakeholders and target group have around the issue that the policy addresses?
- What does the evidence tell you about how well matched the experiences and aspirations of the target group are to the goals of the policy?
- What experiences have been captured from evaluations or reports from similar initiatives?How could they be used to strengthen your approach?
How can you judge good evidence?
When deciding which evidence to base decisions on, it is important to assess the robustness and quality of the available evidence. There are numerous guidelines from research associations and institutions that provide a detailed overview of quality standards for the production of evidence.
In the German market, the ‘Gesellschaft für Evaluation e.V.’ (DeGEval) provides a good overview of the standards for evaluations and ‘Arbeitskreis Deutscher Markt- und Sozialforschungsinstitute e.V.’ ADM e.V. offers a useful checklist for clients commissioning surveys. In practice, however, it is more useful to be guided by concise and clear frameworks highlighting key dimensions of good evidence, such as the following framework from evidence-based policy practice in the UK (Houses of Parliament 2017, Nesta 2022).
When trying to assess evidence from a variety of sources and methods, it is helpful to ensure that it fulfils the following criteria:
| Authoritative |
It is important to assess where the information comes from and who created it. Are the qualifications of the authors credible and is their approach transparent? |
| Objective |
To understand the aim behind any research and the possible biases that may occur, you should consider the purpose of the study and the likely motivations behind it. In this context, it is often useful to check who funded the research. |
| Relevant |
Not all research is useful to your question or situation. You should therefore check whether the context in which the research took place is comparable to your own, whether all relevant perspectives were included, and which target group the research was originally intended for. |
| Timely |
If you are basing your decision on evidence, it is important that this evidence is not out of date and is still valid. You should therefore check whether the information you have is up to date and whether anything important has happened in the meantime that could affect the results. |
| Accurate |
To assess the quality of a method, you should ensure that the methodology is described in detail and is appropriate. It is also helpful to check if similar studies have been conducted or whether the study has been repeated. As a rule, peer review or validation by other experts in the field is a good sign of strong evidence. |
References:
- Houses of Parliament (2017): Parliamentary Research Handbook
- Nesta (2022): Engaging with evidence toolkit – A practical resource to strengthen capabilities for evidence use and expert engagement.
- Social Policy Evaluation and Research Unit (Superu 2018): Making sense of evidence: A guide to using evidence in policy. Wellington, New Zealand.
- World Health Organisation (2021) Evidence, policy, impact. WHO guide for evidence-informed decision-making
.svg)
Back
.png)
