Superforecasting for innovators
By Kate DowlerIn an increasingly uncertain and volatile world, we reflect on whether it is possible to forecast the future, and what this means for innovators; drawing learnings and inspiration from the book ‘Superforecasting’ by Dan Gardner and Philip Tetlock.
In an increasingly uncertain world with high levels of disruption from external forces, innovators are worried about the future, and how to make good decisions in terms of which direction(s) to take, what to invest in, and which ideas to back.
A lot of the teams and innovators we work with are focussed on forecasting change, for example Strategy and Marketing teams who want to predict specific events and changes that may have a big impact on business. For example competitor activity, regulatory changes or the availability of new disruptive technologies, and then using these predictions to make business decisions.
In this article, we’re going to be taking a look at the art of Forecasting in more depth, and reflecting on Dan Gardner and Philip Tetlock’s interesting book Superforecasting. What can we learn from those individuals who are better than average at predicting future events, and how can we apply these principles to innovation to make better decisions for the future?
The story of Superforecasting
The opening chapters of Superforecasting describe a twenty year study conducted by Tetlock at University of California measuring the accuracy of predictions made by the kinds of people who are paid to make them - journalists, forecasters, and industry experts. A total of 284 experts were recruited, and over 80,000 of their predictions were logged. Tetlock’s findings concluded that these experts were no better at predicting the future than the general public, or to use his now famous quip “a dart-throwing chimpanzee.” The study revealed how difficult indeed it is to forecast and predict the future, even in the short-term.
From this, perhaps disappointing, conclusion, the book goes on to introduce the Good Judgement Project (GJP), another long-term research project ran by Tetlock and Barbara Mellers at the University of Pennsylvania, to understand more about the science of prediction - specifically what makes some people better forecasters than others.
What was so interesting about the GJP team of research participants was that its members were not industry experts. They were selected from a pool of thousands of volunteer forecasters from all walks of life who were asked to forecast a likelihood of a range of geopolitical events. Those with the highest scores (known as Brier scores) were selected to be part of a research team who competed in a competition run in the US in 2011 by the Intelligence Advanced Research Projects Activity (IARPR) to identify cutting-edge methods to forecast geopolitical events. Tetlock tells the story of how the GJP team emerged as the undisputed victor in the tournament, outperforming expert predictions by around 30% and even outperforming intelligence analysts with access to classified data.
One of Tetlock's key findings from the GJP was that cognitive and personality traits were more important than specialised knowledge when it came to predicting the outcome of various world events. A number of key behaviours were identified, and what struck us most when reading the book was the similarity to key “creative behaviours” which we value as innovators, such as open mindset, the need for diverse thinking and the power of iteration.
It got us thinking - how could we cultivate these behaviours even more in the innovation process? For example, being better forecasters should enable us to make better decisions about the ideas we prioritize and take forward during idea evaluation, and it should make us better able to anticipate and respond quickly to external change.
Superforecaster behaviours
The book describes a total of 10 key behaviours that define the best Superforecasters. We’ve picked out 3 that are particularly interesting and relevant for innovators to cultivate further:
Strike the right balance between internal and external views
When faced with a new problem Superforecasters ask the question “How often do things of this sort happen in situations of this sort?”. Rather than treating the problem as unique, they also look for examples and evidence from the external world to help assign a probability, or forecast to the problem or question at hand.
Getting the right balance between internal and external views is something many of our clients struggle with. Some focus mainly on internal wisdom and judgement, and others rely too much on external metrics, such as using consumer testing alone to guide decision-making.
In our work, we consider this balance carefully, mining the internal wisdom of project teams, whilst seeking external views from consumers, experts and creatives to build and strengthen ideas. For example, in a recent project exploring the disruptive potential of a new technology, we used a range of sources. The internal team had plenty of opinions and thoughts which we interrogated as a starting point. We then used a diverse set of external views to explore the challenge, including interviewing subject-matter external experts, looking for analogous examples from parallel industries and seeking out case studies from the past that we could learn from.
Look for clashing forces
For every good argument or opinion, there is typically a strong counter-argument. Superforecasters pay good attention to different views, deliberately seeking out different views and perspectives, helping to overcome confirmation bias; our tendency to notice, focus on and seek out evidence that fits with our existing beliefs.
We are firm believers in the importance of diverse thinking, of building diverse teams who bring different perspectives to a challenge. If we want to behave more like Superforecasters then we should be taking things a step further - pushing individuals to deliberately take on a different viewpoint, to seek out contradictory evidence and information, to challenge their own assumptions and decisions.
A constructive example of this approach is the concept of integrative thinking, described in detail in Roger Martin and Jennifer Riel’s book Creating Great Choices. Martin describes the approach as “the ability to face constructively the tension of opposing ideas and, instead of choosing one at the expense of the other, generate a creative resolution of the tension in the form of a new idea that contains elements of the opposing ideas but is superior to each.” Essentially, it’s about articulating and exploring two extreme and opposing models to the problem, and using this tension to create new possibilities, ideas and ways of thinking.
The importance of belief updating
Superforecasters are particularly good at regularly updating their forecasts in the light of new information, and striking the right balance between under and over-reacting to new information and signals. This regular process of belief updating is a methodical and well-considered process - contrary to the human tendency to over-react to information which is particularly striking and engaging with high availability in memory - cognitive biases known as the Anecdotal Fallacy and Availability Heuristic.
The importance of iteration is well known amongst innovators. However, the concept of “belief updating” is subtly different from how we think about iteration generally. It suggests that we should be continuously reflecting on the decisions we make, and adjusting our beliefs in light of new external changes and evidence. For example, an idea that appears to have limited future market potential today, may have quite a different forecast in a year or so. Most organisations have banks of existing, “old” ideas lying around, which are rarely given the light of day once they are de-prioritized. It’s worth bearing in mind at any point in time that the answer to the changing world may already exist within your organisation.
Superforecasting for Innovators
What have we learnt from Superforecasters? Well, firstly that it is extremely difficult to predict the future, even in the short-term (and almost impossible for longer term horizons).
However, there are some key behaviours that can help us make better, more informed decisions when it comes to forecasting short-term change, which can be of great use to innovators and Marketing teams who want to predict specific events and changes that may have a big impact on business in the near-term.
In particular, balancing internal vs. external views, looking for clashing forces and belief updating are three key principles that we should cultivate further to make better decisions when it comes to planning for the future, as well as making us better and more considered innovators.