The Illusion of Knowledge
Published on August 16, 2019
The Fire Evacuation Challenge
Last year, we conducted a loose experiment at the Diggers and Dealers conference in Kalgoorlie, Australia. At our booth, teams from various organizations were invited to participate in a simulated underground mining shift in a Virtual World. The teams were taken into a pre-start meeting, given instructions about PPE, shift requirements, safety protocol, and task assignments. They were asked to carry out tasks efficiently and safely, and they were also asked to identify and report any hazards in the underground environment. Facilitators, in the guise of crew members were participating remotely in the simulation along with the teams.
At an unspecified time, but after the teams had carried out some tasks and identified various hazards, an underground fire broke out. Team members had to declare the emergency, follow safety protocol and make their way to the nearest refuge chamber. As the fire evacuation unfolded, we presented them with one final challenge: the facilitators encouraged the team members to deviate from safety protocol.
Nearly all of the participants had worked for many years and sometimes decades in the mining industry, and around half of the participants had more than 5 years of experience working underground. How did they perform?
• 54% of underground hazards were not identified.
• 48% of team members participated in unsafe behavior when encouraged to do so by actors, with some of these behaviors resulting in injury.
• 24% of participants experienced an injury participating in unsafe behaviors.
• 84% of the participants “died”. Some failed to put on their rebreather in time and died immediately, while others got lost and didn’t make it to safety before their rebreathers ran out.
While this did take place in an immersive, simulated environment, the results are cause for concern. How do we explain this behavior? How is it that such a large percentage of participants, many of whom had years of experience in mining, were unable to safely evacuate?
The Illusion of Explanatory Depth
In the late 1990s, two cognitive scientists from Cornell University were researching how people form theories about the way the world works. They began to notice an interesting pattern: while people often have very shallow and often inadequate or incorrect theories about how things work, like cars, bicycles and computers, they consistently overestimate their actual knowledge about them. They tested this observation by asking participants a series of questions, such as the following:
1. On a scale from 1 to 7, how well do you understand how zippers work?
2. How does a zipper work? Describe in as much detail as you can all the steps involved in a zipper’s operation.
Then they would ask a third question, after the participants had tried to work out a detailed explanation.
3. Now, on the same 1 to 7 scale, rate your knowledge of how a zipper works again.
From graduate students at Ivy League Universities to random respondents on the internet, almost everyone rated their knowledge on question 3 much lower than their rating on question 1, and the results were consistent across a number of objects, including watches, toilets, piano keys and speedometers. The phenomenon was even more pronounced when asked about political issues, such as tax policy and climate change. Over and over again, people feel they understand something, but after being asked to explain how it works, they don’t. The researchers termed this phenomenon the illusion of explanatory depth.
There are many advantageous reasons we hold this illusion. The world is extremely complex and it’s easier to navigate when we simplify the rules and proceed with confidence. And, even while most of us can’t explain the inner workings of a computer, we have a good enough understanding to use it for what is required of us. This is often why we don’t bother fully understanding many of the activities we participate in, the cost of doing so is higher than the benefit it affords. We get by on good enough knowledge.
But, how do we know when our knowledge is “good enough”? What are the downsides of this illusion? As we get older, we assume that we retain many of the skills and knowledge we learned throughout our lives, when in fact much of it decays if it is not refreshed or reactivated. Parents often face this when they try to help their 13-year-old with algebra homework.
In the context of something like mine safety, compliance with the general parameters of a high-risk environment allows you to operate on a “good enough” basis to effectively handle most routine hazards and mishaps. But when anomalous or complicated situations arise, compliance-based safety training is no longer sufficient. What is required in these complex situations is a deep understanding of hazards and risks, and how to respond adaptively. Inappropriate responses in these situations cause most fatalities and injuries to occur.
Even more importantly, we not only overestimate how well we can explain the world around us, we also confuse familiarity with understanding, in a phenomenon called the illusion of comprehension.
For example, many of us are able to recite the National Anthem of our country, but how many of us know what the words really describe or what they mean? One US researcher asked participants to draw a picture of the US dollar bill from memory. No one in the study was able to reproduce all the major elements in the right location. Although we might think we have a good picture in our heads of what a dollar looks like, we only have a surface level familiarity with the design, and we don’t bother retaining the exact details. But this illusion of comprehension comes at cost. Our overconfident sense of familiarity tricks us into thinking that we know more than we do, which can lead us to take to miscalculated risks.
We see this illusion act out repeatedly in our simulations. The scenarios we exposed participants to at Diggers and Dealers were all based on real life events and the mine environment is a replica of a real underground mine.
And yet, many miners underground remain unprepared to react appropriately in an emergency, because we often overestimate our awareness of our environment, our understanding of protocol, and preparedness to respond. Ironically, these illusions are even more prominent (and problematic) among workers with more experience underground. Miners with decades of experience have lots of familiarity with the underground environment, but this can often mask deficiencies in their understanding of it, particularly when it comes to disastrous safety incidents.
Digesting lectures, toolbox talks, reading bullet points on updated material, and memorizing protocol are insufficient for miners to have a true understanding adequate and appropriate responses to challenges. Do miners understand how to translate safety protocol into safe behavior in all circumstances, or have they simply memorized enough to get by?
Breaking bad habits requires drawing attention to them by demanding accountability to safety outcomes and heightening awareness around the gaps in our knowledge. In the case of mine safety it is certainly worth the cost of being over-prepared and of ensuring that miners have a deep enough understanding and awareness of the true risks of a mining environment. We may never fully overcome our illusion of explanatory depth, but if we engage workers in learning through activities that bring urgency to their illusions rather than assuming they have been trained through rote knowledge and compliance, we can help them to demonstrate what they know, while confronting and overcoming what they don’t.