We have all heard and seen the diagrams that relate to the evolution of analytics within the marketplace. Listed below is one such diagram which is probably familiar to most of you
The above diagram represents this evolution as encompassing 4 stages (Descriptive Analytics, Diagnostics Analytics, Predictive Analytics, and Prescriptive Analytics). Much of the literature attempts to discuss the proportion of companies that are within each stage. Not surprisingly, most companies are still not at the normative stage of predictive analytics(well under 50%). By normative, I mean that the discipline of predictive analytics has become a critical business process within the organization. Even within those organizations that consider themselves predictive analytics practitioners(i.e. normative), many of them are skeptical that their efforts are truly being optimized.
These findings and comments just simply reinforce the notion that there is a high degree of immaturity regarding predictive analytics as a business discipline. Many people might refute this statement but the counter argument is a simple one. The demand for experienced predictive analytics practitioners far outweighs the supply. International Data Corporation (IDC) predicts a need for 181,000 people with deep analytical skills in the US by 2018 and a requirement for five times that number of positions with data management and interpretation capabilities. In many other disciplines, small gaps do exist from time to time but this gap within the predictive analytics discipline is both wide and persistent. Organizations are attempting to fill the gaps by hiring data scientists that have the requisite technical skills and yet there are still shortages even when hiring for roles with minimal practical experience. But this gap in talent is further compounded when organizations attempt to identify individuals with high levels of practical experience.
As I have stated before in previous articles, there are few predictive analytics practitioners with any reasonable longevity of 10 years plus. Yet, it is these seasoned practitioners who will be critical in ensuring that the best practices and processes of our discipline are enacted within the organization. The resolution of this dilemma is ultimately time as more and more practitioners gain that vital “level” of experience. But at this point in time, the discipline is still in early stages.
Despite its current level of immaturity as a regular business discipline, predictive analytics and the subsequent stage of prescriptive analytics have been used by certain organizations for many years. Indeed I was very fortunate to work at one of these organizations such as American Express back in the late eighties and early nineties where the basic principles of data mining and data science were essential in developing successful predictive models. As Amex has always been at the forefront in adopting new processes and methodologies, the organization had actually built a variety of models which could be used to create an overall net benefit score for each cardmember. These models encompassed levels of credit risk as well as levels of marketing performance such as retention and cross sell/upsell which were all then integrated to create an overall net benefit score. The net benefit score was essentially a composite measure of the customer’s overall profitability to Amex. The creation of this net benefit score was to provide information that was prescriptive or resulting in actions based on this score. Customer service reps on the front line could conduct certain activities to a given customer based on the net benefit score. For instance, certain products or services might be offered to customers with high net benefit scores while services to customers with low net benefit scores might involve options to improve card payment.
However, achieving this stage of prescriptive analytics requires a number of stakeholders from key areas such as finance, I/T, and operations. Extensive collaboration between these three levels of stakeholders was critical in ensuring that predictive analytics could be transformed to prescriptive analytics. With finance, our data science team needed to better understand the mechanics of the P&L statements and how our models would directly impact a particular line item. At the end of this process, we needed to understand how all the models and their impact to various line items in the P&L would translate to a given net benefit score. In the I/T area, we needed to work closely with the programmers in ensuring that our logic in creating the models and the actual net benefit score could be operationalized within the current Amex systems infrastructure. We also needed to ensure that these net benefit scores with potential defined activities could be hard-coded into the system and be available to the customer service reps on the front lines. This collaboration required extensive meetings and phone calls to ensure that the correct steps and tasks were executed correctly. The last stage involved one of socialization where the data scientists along with the business people from finance and marketing had informal meetings as well as lunch and learns to educate the front-line CSR’s on what this new information meant and how it could be actioned.
In the above Amex case, this collaborative effort among st all these stakeholders was very time-intensive. But without this commitment, the ability to enact real prescriptive behavior is limited. Transforming organizations into this last stage of prescriptive behavior involves a corporate culture that is receptive to change management. Organizations, that remain uncommitted to change, will develop predictive analytics solutions that work very well but are not being deployed in the most optimal manner. Commitment and change philosophy are not easy but well worth it if prescriptive behavior is the outcome.
Richard Boire, B.Sc. (McGill), MBA (Concordia), is the founding partner at the Boire Filler Group, a nationally recognized expert in the database and data analytical industry and is among the top experts in this field in Canada, with unique expertise and background experience. Boire Filler Group was recently acquired by Environics Analytics where I am currently senior vice-president.
Mr. Boire’s mathematical and technical expertise is complimented by experience working at and with clients who work in the B2C and B2B environments. He previously worked at and with Clients such as: Reader’s Digest, American Express, Loyalty Group, and Petro-Canada among many to establish his top notch credentials.
After 12 years of progressive data mining and analytical experience, Mr. Boire established his own consulting company – Boire Direct Marketing in 1994. He writes numerous articles for industry publications, is a well-sought after speaker on data mining, and works closely with the Canadian Marketing Association on a number of areas including Education and the Database and Technology councils. He is currently the Chair of Predictive Analytics World Toronto.