What evaluators need to know about optimism bias

Why do people get married when around a third of marriages end in divorce?

Optimism bias is our tendency to overestimate our likelihood of experiencing positive events and underestimate our likelihood of experiencing negative events. Most people experience optimism bias in one context or another.

1. Being optimistic – good or bad

In many ways, being optimistic is a good thing. Optimism improves our wellbeing because it reduces stress and anxiety, and it encourages us to persevere after failure. However, it can lead to poor decision-making.

At a personal level, we’ve all heard the disaster stories of friends who have underestimated the costs and timings of their home renovation. At a government level, optimism bias is one of the biggest causes of cost overrun on major projects like public infrastructure.

Sometimes the benefits of projects are larger than anticipated which makes the cost overrun worth it, but often the benefits are also overestimated. This is bad enough for taxpayers and other investors who fund major projects, but it can disproportionately disadvantage certain groups who were promised the benefits.

2. What this means for evaluation

We see the effects of optimism bias in many of the programs we evaluate. For example, one program we evaluated involved two vastly different organisations collaborating for the first time. Each organisation had a different culture, policies, and ways of working. The program implementer assumed the two organisations would figure out how best to work together, with little support to facilitate this. The program implementer’s optimism bias significantly impacted the program’s success.

As evaluators, we also need to be mindful that we face the same risk of optimism bias in how we design, deliver, and report evaluations. For example, we have designed evaluations assuming we would be able to analyse administrative data which turned out to be too hard to access. We have also underestimated how stakeholders would react to certain evaluation findings.

3. How to mitigate optimism bias

Many people struggle to see their own cognitive biases but can see other peoples’ biases very clearly. We need to recognise that we are all prone to optimism bias and build in mitigation.

During project planning – regardless of whether the project is a community service or an evaluation – there are a few things that can help:

1. Identify a similar past project and use information about how long it took to complete and how much it cost, to adjust your own project plan.
2. Conduct a premortem on your project before you implement it. A premortem involves a group brainstorm where you imagine the project has failed and work backwards to identify the potential causes. It’s a hypothetical exercise which helps to address the short-sightedness that exacerbates optimism bias.
3. Use adaptative management techniques during the project implementation. Monitor key indicators of success and risk; and change course if required in a timely fashion. 

The Australian Evaluation Team at Kantar Public has expertise in both evaluation and behavioural science. We integrate these approaches to actively look for optimism bias and other cognitive biases in our own work and in the programs we evaluate. This ensures we deliver high quality and timely evaluations with valuable insights about the hidden causes of program failure and innovative solutions to improve outcomes in the future.