Machine Learning Times
Machine Learning Times
Elon Musk Predicts Artificial General Intelligence In 2 Years. Here’s Why That’s Hype
 Originally published in Forbes, April 10, 2024 When OpenAI’s...
Survey: Machine Learning Projects Still Routinely Fail to Deploy
 Originally published in KDnuggets. Eric Siegel highlights the chronic...
Three Best Practices for Unilever’s Global Analytics Initiatives
    This article from Morgan Vawter, Global Vice...
Getting Machine Learning Projects from Idea to Execution
 Originally published in Harvard Business Review Machine learning might...

7 years ago
If You Build It, Will They Come? Lessons Learned To Improve Model Adoption


By James Gottshall, Vice President, Strategic Analytics at XL Catlin

This article covers a similar theme to the Keynote presentation James Gottshall  will deliver at PAW Financial, Oct 29-Nov 2, 2017 in New York.

Think about a time when you built a predictive model for one of your business partners that you were proud of, but for some reason, their excitement level wasn’t matching yours. If you are anything like me, you probably were overcome by feelings of frustration and confusion. “How is it possible to not understand the power and competitive advantage I just hand-delivered to you?! How else do I need to explain this to you for you to get it?!” I have come to realize that in those scenarios, the only person I need to ask those questions is standing in the mirror.

In today’s world of data and analytics, it is easy to get caught up in all of the hype – and rightfully so, to some degree. There aren’t that many people in this world that have the requisite skills to transform an organization’s information assets into engines that can help drive its future outlook. There is a lot to be said for those skills, and it’s easy to understand why organizations are enamored with visions of grandeur. But there is a catch.

In order for an organization to fully realize the value of predictive analytics (or big data, or machine learning, or AI, or [insert current buzz word here]…), it needs to be ready and willing to go through a transformation process. The insights and foresight that will be created are only valuable if they are used consistently. Building the tools is critical to the process, but gaining adoption by navigating your way through the change process is equally as important.

I’ve learned more through my failures than I have in my successes, so I thought I would share some valuable lessons I have learned through a painful experience that took place a number of years ago. Please don’t misinterpret my points as fully comprehensive or even appropriate for every situation. Also, please understand that I take full responsibility for the lack of adoption. I now realize I should have approached things differently – very differently, in fact. My simple hope is that you can benefit from my mistakes without having to experience them yourself.

So, how can you increase the likelihood that the tools you build and insights you generate will be used by the business?

Align your priorities to the business’ priorities.

In a prior role, I helped one of my employees build a model that predicted additional premium and return premium audits for our Workers Compensation line of business. All of the statistics proved that the model was accurate – actually, it was very accurate, especially on the bookends. We were sure that the business would jump all over the opportunity we just presented them with. The exact opposite happened.

When we presented our results to the business leaders, we were met with an underwhelming response. I won’t go into the details, but we were highly disappointed to say the least. With the benefit of hindsight and time, I realize that we approached the problem completely backwards. We were trying to prove how smart and capable we (thought we) were. Meanwhile, the business leaders were trying to figure out why we were spending our time solving that problem.

If we approached the business leaders first to understand their priorities, we could have spent our time demonstrating what we were capable of on a business problem that was a priority to them at that particular point in time. As simple as this sounds, this is a common pitfall that dooms many projects before they even start. Spend your time wisely. A simple meeting or two to better understand the business’ priorities will go a long way. You can start to bring your own ideas to the table after you’ve first established trust.

Start with the end. Demonstrate how to operationalize the insights before starting a project.

When I take the time to reflect on the underwhelming response we received when we presented the results of our premium audit model, I know a major problem was that we weren’t aligned with the business’ priorities. However, in spite of that initial roadblock, I think we still would have had a fighting chance if we demonstrated how the insights would generate additional business value within an operational context. Unfortunately for us, we didn’t do that.

We structured our presentation around the results of the model, using statistical terms and visualizations to show how accurate the predictions were. I’m sure you can guess how well that was received by the business. While it’s important to demonstrate that the model is powerfully predictive (and therefore useful), we went too deep – way too deep.

We should have demonstrated how the business could operationalize the new predictive insights we created. Presenting examples that showed how to optimize our premium audit resources, or how to recommend different payment plans, probably would have piqued their interest. We would have been speaking in their terms, addressing the important, “What’s in it for me?” question. We didn’t get a chance to discuss any deployment and integration options because we were too focused on presenting results that showed how smart we thought we were. Cue Alanis Morissette’s song “Ironic.”

Take the time to understand how the new insights will improve business outcomes. Partner with the business to define success in their terms before you start the project, which has to include discussions on enhanced future-state workflows and integration technologies. Not only will this improve adoption rates, it will also enable you to monitor the results post-implementation so you can measure your level of success (and close the feedback loop). You will be viewed as a problem solver, not just a technical SME.

Co-author the solutions with your business partners.

Let’s pretend that the premium audit problem was a top priority for the business at that point in time. And let’s also pretend that we presented our modeling results in a way that demonstrated the business value. (I know, I know…hindsight is 20/20!) I’m willing to bet that we would have still been met with some resistance from the business. The reason is because we didn’t engage them as early and as often as we should have.

I mentioned earlier that the organization needs to go through a transformation process in order to accept and adopt analytical solutions. The main reason comes down to simple human nature. As professionals, we all take pride in our accomplishments. Many of us have gotten to this point in our careers because of those accomplishments. And those accomplishments were achieved by doing things a certain way. Analytics could be seen as a threat to that.

The natural human response is to reject the analytical insights if they don’t align with our own intuition. Therefore, feelings of threat and inadequacy are to be expected. With that in mind, as analytical professionals, it is our obligation to go on a change management journey with our business partners to help them overcome those initial feelings. If you remain humble and open to new ideas throughout the journey, you will probably find that you yourself were stretched and grew even more than your business partners. Give yourself permission to step out of your comfort zone.

Strategically create project teams that include a healthy mix of early adopters, naysayers, and everyone in between. Select employees from various levels that are well-respected by their peers and that have a lot of influence in the organization – regardless of their views on the project. Empower the team to define how the tools should be used within their operations in the future-state; add the project to everyone’s business objectives. At the end of the day, you don’t want the business to be an audience member. You want them to be co-authors that have just as much accountability to the project’s success as you do.

If you build it, will they come?

I can’t promise you that you’ll achieve 100% adoption by implementing the tips I mentioned above. But if you do, I’m fairly certain that you will be positioning yourself for much a higher chance of success than if you made the same mistakes that I made.

I know you have a lot of your own ideas and experiences to share that I (and the collective we) can learn from. Please post them in the comments section below. Depending on the responses that I receive, I may aggregate them in another post for everyone’s benefit.

Author Bio

 James GottshallJames joined XL Catlin in January 2017 as a Vice President and Business Solutions Consultant on the Strategic Analytics team. He is responsible for leading large and complex analytics projects that enhance decision-making processes and that add tangible business value. James liaises with business leaders, data engineers, and data scientists to create highly impactful business solutions that drive behavioral changes. Prior to joining XL Catlin, James worked at Selective Insurance as an Assistance Vice President of Analytics Strategy. In that role, he was responsible for creating and subsequently executing the analytics strategy to infuse a culture of analytics across the entire organization. James also served as an Assistant Vice President of Claim Analytics Integration at Chubb, where he led the integration efforts within claims advanced analytics.

James earned his B.S. degree in mathematics from Pennsylvania State University. He also serves the United Way of Northern NJ as a Board Trustee.

Leave a Reply