Originally published in Forbes
Executives know the importance of predictive AI. As Unilever CDO Morgan Vawter wrote, “Its practical deployment represents the forefront of human progress: improving operations with science.”
But there’s bad news for data scientists: Your predictive AI project will probably fail. Your customer probably won’t operationalize the machine learning model you deliver. They won’t use it, act on it or integrate it. Sadly, most models developed for deployment wind up on the shelf.
But you probably already knew that. A plethora of industry research and anecdotal wisdom has let that cat out of the bag.
And yet what most data professionals haven’t come to understand is the true reason why. After all, shouldn’t a great model be a sure bet to deploy?
No. It won’t deploy because you haven’t closed the sale. When you deliver a model, you haven’t necessarily finished selling, no matter how much of a done deal it may seem to be. Until your model actually deploys, you must continue to actively sell it to stakeholders – in compelling, concrete business terms like improved profit – even if they’ve already agreed, signed and paid.
Only a concrete projection of value compels a business. Your customer and other decision makers may well have been excited about the project’s intent at its outset. But without visibility into the business value – a view that plainly displays how the model will drive decisions and the expected value to be gained – you’ll ultimately begin to hear excuses not to deploy. Put another way, without the enthusiasm that only a bottom-line promise can produce, the project will be among those first cut when the next inevitable financial crunch arrives. People don’t buy what they don’t understand.
As with any operational improvement, with or without analytics, a business can’t move forward until there’s a credible estimation of how much it’s going to improve those operations – a calculation of the business improvement you stand to gain – in straightforward terms like profit or other KPIs.
Yet most predictive AI projects don’t move beyond standard technical metrics – such as precision, recall, AUC or F-score. These represent the data scientist’s training, tools and comfort zone. And they serve well to establish model soundness, assessing whether the model performs relatively well, substantially better than guessing. If so, the model is potentially valuable.
But these metrics provide little to no insight into how valuable the model would be if used. They are arcane from the standpoint of business professionals and business objectives.
You may be the technical heavyweight in the room, but if you’re presenting only standard technical performance metrics, you’re committing a cardinal business sin and deep down your colleagues know it: You’re proposing a systematic operational change with no concrete estimate of its upside.
After all, when you build something, you’ve got to check out how good it is before you use it. You can’t launch a new rocket until you’ve stress-tested it in its intended usage, according to the KPIs that matter. Without a credible estimate of the potential value, the launch would be a shot in the dark. Sensible decision-makers would scrub the launch. Indeed, most predictive AI deployments are scrubbed.
No matter how advanced your analytical method may be, the decision maker’s gut skepticism rings true: If you haven’t measured the potential business value, then how could the project be pursuing business value? They may not feel that they hold the “tech authority” to say so, but they’ll find any of a myriad reasons to not move forward with operationalization.
Instead, make it a no-brainer for your customer: Sell the potential business value. This will give decision makers essentially no choice but to deploy, and will give the fruits of your number crunching a chance to realize a business impact. Everyone already understands, in general terms, the potential value of driving decisions with model predictions – that’s what has brought the project this far. Now it’s time to follow through on your business-oriented swing by establishing how much business value the deployment stands to deliver. This way, you’ll knock it out of the park.
About the author
Eric Siegel is a leading consultant and former Columbia University professor who helps companies deploy machine learning. He is the founder of the long-running Machine Learning Week conference series, the instructor of the acclaimed online course “Machine Learning Leadership and Practice – End-to-End Mastery,” executive editor of The Machine Learning Times and a frequent keynote speaker. He wrote the bestselling Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, which has been used in courses at hundreds of universities, as well as The AI Playbook: Mastering the Rare Art of Machine Learning Deployment. Eric’s interdisciplinary work bridges the stubborn technology/business gap. At Columbia, he won the Distinguished Faculty award when teaching the graduate computer science courses in ML and AI. Later, he served as a business school professor at UVA Darden. Eric also publishes op-eds on analytics and social justice. You can follow him on LinkedIn.
