Business decisions have always been made with a mix of intuition and evidence – but technology is forcing a new approach
The digital revolution and the rise of analytics has fundamentally transformed decision making processes. Data and analytics literacy has become an integral part of a modern manager’s skill set. For organizations, a key differentiator of future winners will be the ability to navigate the complexities brought by an abundance of structured and unstructured data. It will be built on investment in data analytics and the development of leaders with data and analytics acumen. It means understanding the new role for leaders in business decision making – and how decision making has fundamentally changed.
The evolution of decision making
In the digital era, the speed at which decisions need to be made – and the complexities involved – are often an obstacle to effective decision making, even with the most skilful use of classical critical thinking and decision making approaches. Looking at how business decision making has evolved over past decades, we can identify three broad phases.
In what we might call ‘Decision making v1.0’ – the foundational approaches to business decision making which have been a staple of management education – leaders leverage decision analysis to structure problems, use utility theory to think about values, and optimize with respect to desired objectives (profit, cost, market share, risk mitigation and so on). In this model, being ‘data driven’ meant making rational decisions, unclouded by emotion or prejudice. The leader’s industry knowledge and experience is recognized, but fundamentally, Decision making v1.0 assumes a fully rational super-human, somehow capable of collecting and immediately processing all available information.
Enter ‘Decision making v2.0’, which recognizes the limits of human cognition and the inherent irrationality of the mind’s shortcuts for analysing information. Here, there is an awareness of the flaws of intuitive thinking and ways of correcting for them, such as debiasing techniques and nudges. Decision making v2.0 allows for leaders’ intuition in complex decisions – for instance, in choosing what is relevant and in framing the task at hand.
Organizations need to embrace the power of analytics to supplement and complement the old ways of decision making
Today, however, these human-only approaches are increasingly inadequate, thanks to the abundance of information and the speed and sheer volume of decisions needed. Organizations have to embrace the power of analytics to supplement and complement the old ways of making decisions. We are in the era of ‘Decision making v3.0’. Assuming that they are well designed and employed, data-driven algorithmic analytics tools are well-suited to the complex demands of decision making in data-rich interconnected environments. The new model requires human adaptability: not letting go of our intuition, but accepting that intuitive insights are imperfect and can often be enhanced (or corrected) by new evidence, information or experiences.
While human intuition built on a career’s worth of expertise and experience can be formidable, data also represents experiences and outcomes. Each individual data point alone may only be a minuscule piece of evidence, but analytics using large amounts of such data can easily outweigh one person’s insights.
Decision making at speed and scale
Analytics have always been important for business functions which heavily rely on information management, such as IT, accounting and finance, but in recent years they have profoundly impacted other business functions and sectors.
Marketing, for instance, has been forever changed by the emergence of digital ad markets, which have disrupted the media industry and transformed how media content is disseminated and consumed – and it has only been possible thanks to algorithmic tools. With billions of allocation and pricing decisions processed daily, human control of market-clearing processes is impossible. Plus, human assessment of pre-selected user characteristics is no match for algorithmic estimates of user value, discerned from their digital footprints.
Further, as my research has established, optimal decisions can have seemingly counterintuitive features that human decision-makers would miss. For instance, an advertiser might find their own ad more valuable if no competitor’s ad is displayed concurrently: businesses are sometimes willing to pay significantly more to prevent the display of a competitor’s ad.
Another key domain for algorithmic decision making is in on-demand service platforms. Their success is partly due to value created by optimizing the coordination of supply and demand. Decisions made by an algorithm for a ride-hailing platform, for instance, can be more efficient than those of a human taxi dispatcher – and higher utilization of drivers is key to lower prices for riders. Algorithms can take decision making in unexpected directions. The concept of riders rating their drivers is well established, but drivers’ ratings of riders also have a critical function in the absence of price discrimination, allowing the platform to prioritize some riders over others. It enables a level of optimization that is otherwise impossible.
Data-driven approaches are especially valuable in complex systems, like large corporations that own multiple products with interrelated production inputs and demand – think of the varied product lines of companies like Nestlé or Unilever. Businesses have generally used the strategy of ‘decomposition’, managing production and sales separately for each product. Yet these products are often highly interrelated (consider foods with common ingredients, such as sugar or wheat) – and this strategy can result in a significant loss of overall profitability, even if optimal decisions are made at the product level. In contrast, analytics tools allow for optimal decisions to be determined at the system level, even if they seem suboptimal from a product manager’s perspective.
The more complex the system and decision making problem, the greater the need for analytics.
Evidence vs intuition
The digital revolution lays bare the inherent limitations of intuition-based decisions. Decision making v3.0 requires leaders to learn new skills and adopt a new role in the process, changing where they focus their time and attention. The algorithms and analytics tools can be outsourced to specialists, but leaders need a basic grasp of the types of problems analytics can and cannot handle, what data is available and helpful, and whether algorithmic implementation is feasible. Today’s managers need to be able to communicate and collaborate effectively with the engineers in charge of these crucial tools.
With digitization and interconnectivity accelerating in all aspects of life, the winning organizations of the future will be those that ramp up their business leaders’ data acumen, shift their role in the decision making process, and incorporate data analytics sooner rather than later.