To create strategy in the face of uncertainty, leaders need to recognize the limitations of the scientific method
Writing: Haywood Spangler
Evidence-based practices and data-based decision-making are intended to reduce uncertainty and minimize risk. They ground choices in empirical evidence. But how can you trust the empirical evidence itself?
Assessing evidence presented as scientific findings is foundational to planning and making decisions with foresight. To avoid oversimplifying complexity and falsely reducing uncertainty, leaders must adopt the mindset and practices used by researchers themselves.
Consider this example: David sought to develop policies that would encourage commuters to reduce driving. Several credible sources told him that commuters would change their behavior if they had the option of rapid bus transit. Local public-private partnerships spent millions of dollars creating such a system – yet ridership was low and freeway usage remained almost unchanged.
Similarly, Maria’s business was poised to develop a subdivision of detached homes near a wetland. They would have fantastic views of the wetlands and wildlife. Well-conducted demographic research indicated her region was set for an influx of young professionals who would be eager to buy such homes. Her company started to develop the subdivision, and the young professionals were moving into the region, as predicted. Yet they weren’t buying in Maria’s subdivision: the newcomers tended to prefer infill row houses that had first been developed in the 1950s.
Both David and Maria fell short in their strategic planning because of their overconfidence in scientifically based claims about social trends and human behavior. They equated scientific findings with certainty. Like David and Maria, many of us may assume that mathematically derived findings using empirical methods are indisputable.
Cultivating a scientific mindset
David and Maria’s stories illustrate that, ironically, we may adopt an unscientific mindset toward information labeled as scientific. An unscientific mindset isn’t irrational, but it can be less discerning and over-credulous.
Defaulting to an unscientific mindset involves accepting information without seeking disconfirming evidence to verify or discredit that information. It also involves relying on conventional, received wisdom, sometimes represented as ‘best practices,’ and getting on the bandwagon with solutions that peers and competitors seem to employ to achieve success.
These undiscerning habits may lead us to embrace scientifically validated information without asking whether the information is, in fact, relevant to our specific situation.
You can counter these tendencies by deliberately developing your own scientific mindset, and ascertaining the quality and relevance of the scientific data you encounter. Here are four ways to develop a scientific mindset.
1. Recognize what defines science and the scientific method
Science involves a process of developing hypotheses (theories to answer a question or solve a problem) and testing them, statistically and through controlled experiments. Moreover, it is a process of inquiry, relying on the refinement of theories, generating an evolving body of knowledge.
Be clear that scientific research is an iterative, communal process. A finding based on a few experiments is provisional and likely to be altered by further research. More conclusive findings result from a lengthy process of replication involving many researchers. Unless findings have been robustly replicated, they may not provide sound evidence for planning and decision making.
2. Look for findings that have been replicated
Just because a specific researcher says, “I conducted a series of experiments. Therefore, I have concluded x, y, and z about human behavior,” doesn’t mean the researcher reached a definitive conclusion.
To be credible, specific findings have to be replicated by multiple people, at different times, in different contexts. Investigate how frequently a particular finding you want to use as evidence has been replicated. Just as you might seek a second medical opinion upon receiving a physician’s diagnosis, seek multiple sources of data to ascertain the degree to which the data is accepted by multiple experts.
3. Realize that methods used in research rely on induction
Researchers test theories about human behavior through statistical hypothesis-testing and controlled experiments – such as experiments examining how we make decisions under conditions of uncertainty. The information we use for evidence-based practice and data-driven decision-making often relies on induction, which involves drawing inferences from features in a sample of data to reach a broader conclusion about a category of events or a population.
Yet induction has a fundamental limitation: we can’t know unobserved things based on what we know of observed things. Notions that the future looks like the past, or that a whole population looks like its sample, are always assumptions that could be disproven at any time by unforeseen events.
For the purposes of strategic planning, information based on induction provides good but not conclusive grounds for accepting a conclusion or prediction.
4. Enhance your uncertainty tolerance
One reason we may misinterpret forecasts and scientifically based findings is a need for certainty. Yet as David and Maria’s scenarios illustrate, when we project certainty onto situations that actually have indeterminate elements, we can make unsound strategic decisions.
Enhancing uncertainty tolerance is foundational to developing a scientific mindset. Recognize that being uncertain doesn’t undermine your credibility.
Identify your areas of adaptive expertise – your core knowledge and skills that can be applied in different situations where there’s uncertainty. Apply what you know: insights from previous experiences, interactions with peers, and sense of purpose.
Scientific thinking
Adopting these four elements of a scientific mindset could have helped David and Maria examine the research they received more critically. They could have developed strategies better aligned with the reality of populations they sought
to engage.
In a world where every category of business decision is based on research, analytics, and claimed insights about what different datasets tell us, leaders would do well to think like scientists more often.
Haywood Spangler is the founder and principal of Work & Think and author of Reasoning for Business: The Inquirer’s Guide to Decision Making (Routledge)
