Held on Thursday 16 May, CGIUKI’s inaugural Annual Lecture brought together the Institute’s international board members, divisional CEOs and distinguished guests for an evening of discovery and discussion.
Kindly hosted by Deloitte, and Legal Entity Management partner there Daniel Connell, the evening kicked off with a warm welcome from his team. Our guest speaker for the evening was Professor Alex Edmans from London Business School, on the topic of ‘Facts, Data and Evidence: Knowing What to Trust’, based on his new book May Contain Lies: How Stories, Statistics, and Studies Exploit Our Biases – And What We Can Do About It. In an engaging and lively speech, he set out a framework for navigating the seeming onslaught of misinformation, bogus studies and misleading data which confronts us on a daily basis – in both our professional and our personal lives.
What breastfeeding can tell us about handling data and evidence
Edmans began with an attention-grabbing personal anecdote about how he and his wife went about deciding whether or not to exclusively breastfeed their son. He demonstrated that at first glance, the evidence suggests that breastfeeding has better outcomes than bottle feeding for a whole host of important characteristics, including overall health, intelligence and parental bond. However, when he and his wife dug deeper into the studies which supposedly supported these claims, they uncovered something unexpected.
Almost all of the benefits derived from breastfeeding could, in fact, be explained by other factors (known as ‘confounders’), such as family income, support received by the mother, education and personal attitudes to health. Despite the World Health Organization – a highly credible body – recommending that babies are exclusively breastfed for the first six months, the scientific evidence supporting this recommendation is shaky. With their newfound knowledge, Edmans and his wife felt more at ease finding a balance between breastfeeding and bottle feeding which worked for them.
What are the lessons that Edmans wanted us to take from this anecdote? He emphasised four key points:
- Research is everywhere. Both in our professional and personal lives, research influences our decisions and beliefs. The volume of research can, at times, be overwhelming, and can make it tricky to distinguish what is worthwhile from what is not.
- ‘The truth is not enough’. By this, Edmans meant that it’s not always enough just to check the facts. Facts can be accurate but still misleading, for example by implying causation when only correlation has been shown. What really matters about facts is the context in which they are presented, and the inferences which we draw from them.
- Beware your biases. Humans are prone to bias in our thinking – and some biases are simply useful shortcuts which our brains take to speed up our understanding of the world. However, biases can also be very damaging. One major bias is confirmation bias. This means that we are much more likely to believe something is true when we want it to be true (even if the evidence is shaky) and are much less likely to believe something which we don’t want to be true (even if the evidence is staring us in the face).
- The solution sounds simple: check everything. However, this is not always simple to put into practice. Edmans then set out a framework, named the ‘ladder of misinference’, to help us better interrogate information which we come across.
Interrogating information using the ladder framework
When presented with information, be that in our personal or professional lives, how can we check that it is accurate and not misleading? Edmans proposed climbing up a conceptual ladder – with the first rung being ‘statement’, the next ‘fact’, then ‘data’, and the final rung being ‘evidence’. If the information ‘passes the test’ at each level, it can then be given an appropriate weighting in our decision-making; we can give it the attention (or lack of attention) that it deserves.
- Statement. A statement is not a fact and may not be accurate. With a statement, we want to check about the broader context to ensure that it has not been selectively quoted. If the statement is about how an input affects an output, check how both have been measured.
- Fact. Facts are not data and may not be representative. A fact could be absolutely true, but only based on a very small selection of cherry-picked examples. We may only be seeing the cases that fit the desired narrative – and not any of the others.
- Data. Data is not evidence and may not be conclusive. Data may not point to one particular conclusion – there may be a variety of explanations for a particular correlation. It may be that the input caused the output, the output caused the input, or a third factor caused both.
- Evidence. Evidence is not proof and may not be universal. Evidence may have been gathered in one particular setting and not be applicable in other contexts: what makes for a successful violinist may not make for a great executive. Alternatively, the sample may have special characteristics, such as students at a top university, and may not be generalisable.
The ladder framework is useful for big decisions with lots of information to consider – such as those decisions happening at board level. It might not be practical, though, to apply it to every single piece of information we come across – and so there is also a shortcut. Edmans proposed a quick thought exercise which can activate critical thinking and help to overcome confirmation bias.
Imagine you found a piece of information or research study which contradicts your beliefs. Consider how you would critique it, and then apply the same scrutiny to the information which aligns with your views. Do any of the same criticisms still apply? If so, treat the information with caution. Ultimately, Edmans emphasised that this approach allows us to approach studies and research with far more balance.
The evening concluded with closing remarks from Charlie Brown, CGIUKI President, who thanked Edmans, Deloitte and all of the attendees. Guests then enjoyed networking over drinks and canapes in a packed-out room. Safe to say, the evening was a great success. Roll on next year’s lecture!
