Daniel Kahneman is a psychologist who was awarded the 2002 Nobel Prize for his influence on the burgeoning field of behavioral economics. I recently read his bestselling 2011 book “Thinking Fast and Slow” . The book begins with a set of chapters collectively entitled “Two Systems.” This is where the book’s title comes from: System 1 [the “Thinking Fast” from the book’s title] “operates automatically and quickly, with little or no effort and no sense of voluntary control,” while System 2 [“Thinking Slow”] is engaged in “the effortful mental activities that demand it, including complex computations …” .
Kahneman then proceeds to illustrate how these Systems interact. He presents several examples in which System 1’s assessment processes are simplistic and biased. System 2, while capable of making much better decisions, is shown to be “lazy” as a result of the volume and variety of demands that leave it in a busy and depleted state. The tendency toward lazy System 2 processes, it turns out, is also why so many people turn out to be quite unskilled at probabilistic reasoning and associated decision-making; it is simply much, much easier for System 1’s automatic (and often incorrect) heuristics to be deployed than for System 2 to break away from its many other demands.
My System 2 was exhausted by the time I finished “Thinking,” so I simply started reading the next book that was sitting on my nightstand, which was Eric Siegel’s “Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die” . Siegel is a former computer science professor, an experienced analyst and more recently the founder of the Predictive Analytics World conference series. As its title suggests, he has written a book that focuses on data-driven predictions, which he collectively labels as “predictive analytics” (PA).
The centerpiece, or rather centerfold, of the book is a list of more than 100 success stories that involve PA, grouped into categories ranging from “Financial Risk and Insurance” to “Family and Personal Life.” In turn, each chapter tells its own tale through these PA success stories. For example, in the chapter that explores the ethical and privacy implications of using data for prediction (“With Power Comes Responsibility”), Siegel illustrates the key ideas through the story of HP’s model for predicting the likelihood of employees leaving the company and Target’s algorithm and processes for predicting which customers were likely to be pregnant, while in the last chapter (“Persuasion by the Numbers”) he shines a bright light on U.S. Bank, Telenor (a Norwegian telecommunications company) and the Obama 2012 campaign.
When I started reading this book, I really did not know much about machine learning and so it was a nice window into its key ideas (Siegel views machine learning and predictive analytics as almost synonymous). Though this is a decidedly non-technical book, Siegel provides a clear illustration of the concepts behind machine learning, while also naturally introducing some of its terminology (“training sets,” “test sets,” “ensemble methods,” etc.).
Despite Siegel’s best efforts at levity – he routinely makes puns, includes pop culture references and even offers up his own tongue-in-cheek poetry to keep things light – my head began to hurt after reading a few chapters. At first, I could not figure out what my problem was, but eventually, it came to me: the juxtaposition of these two books was the source of the pain.
In the introduction to “Thinking,” Kahneman writes that “this book presents my current understanding of judgment and decision-making, which has been shaped by psychological discoveries of recent decades”; indeed, many of these discoveries were his own, and it appears that the desire for continuing to improve this understanding still drives him to this day. Siegel, on the other hand, describes the world of predictive analytics as ruthlessly pragmatic: “We usually don’t know about causation, and we often don’t necessarily care…prediction trumps explanation.”
Kahneman clearly relished the regular and frequent informal conversations through which he and the late Amos Tversky (with whom he published many seminal papers prior to Tversky’s death in 1995 and to whom “Thinking” is dedicated) evolved their thinking. For his part, Siegel writes with excitement of “one of the coolest things in science, the most audacious of human ambitions: the automation of learning.”
The theories and discoveries that Kahneman describes nearly always involved intricate and careful data collection to test specific hypotheses that were framed within a large body of previous research, while Siegel asserts that, “PA’s aim isn’t only to assess human hunches…but also to explore a boundless playing field of possible truths beyond the realms of intuition.”
When I finally finished Siegel’s book, there was no doubt in my mind that there is a bright and growing future for predictive analytics, and that Siegel is a capable and passionate spokesman with a compelling vision for this future. Indeed, his book illustrates that one virtue of PA in the business world is that it provides objective insights based on data, rather than simply leaving managers and executives free to make complex decisions by intuition alone. As it happens, several Kahneman and Tversky papers  have had a huge impact on exposing the very real limitations in the quality of intuitive decision-making, which has helped legitimize the practice of predictive analytics.
Yet if the future belongs to analytics professionals, as Siegel and many others are fond of suggesting these days, I do hope that they understand not only the power of the computer to make discoveries by relentlessly sifting through increasingly large piles of data but also the value of developing deep domain knowledge, participating in inspired discussions, and being both persistently curious and curiously persistent. These lessons, in addition to their many insights into the psychology of decision-making, are also a huge part of Kahneman’s (and Tversky’s) legacy.
Vijay Mehrotra (vmehrotra [at] usfca [dot] edu) is an associate professor in the Department of Analytics and Technology at the University of San Francisco’s School of Management. He is also an experienced analytics consultant and entrepreneur, an angel investor in several successful analytics companies and a longtime member of INFORMS.