8 years ago
Misconceptions About Predictive Analytics

Predictive analytics is often cited as a key business driver for Big Data. It is easy to see how predictive analytics, when done right, can represent a strong competitive advantage for companies. Such as:

• Enabling companies to be proactive which translates to reduced cost, improved service and/or being the first to market
• Making customers feel you’ve read their mind which improves customer loyalty and retention

So let’s discuss the common misconceptions about predictive analytics and how to overcome them:

1. We need to start small and iterate, so let’s start with a limited feature set

A key roadblock for implementing analytics projects is the availability of quality data.  In the initial phases of analytics projects, obtaining a wide feature set (a.k.a. attributes) on a reasonable number of data points is more important than having a limited feature set on a large number of data points.

Why? Because starting off with a sizable feature set helps us triangulate the data better, identify the key features that most reliably drive the predictions and cull out the others. So what does it mean to start small in analytics? It means:

• Focusing on a limited set of the most pressing business problems and associated use cases
• Using a representative sample of data points as input into the analytics (as opposed to trying to acquire all the data points)
• Applying the business actions to a limited set of instances to minimize business risk and build a feedback loop for iterative model refinement

For example, when modeling customer behavior and personalizing customer experience, it is better to consider all attributes of a representative set of customers, rather than a limited number of attributes of the whole customer base. Similarly, you may want to personalize experience for a subset of customers and iteratively refine the personalization toward the larger customer base.

2. We are predicting outcomes reliably, so we are done

Generating meaningful insights, as difficult as it is, is easier than acting on them. After all, insights have no business value unless acted upon. Why is it more difficult to act on the insights? Because the opportunity cost of not acting, the cost of acting on incorrect insights and/or the unforeseen side effects of action have to be well understood.

So how do we make the process of acting on the insights less risky? By starting small and iterating quickly. Recall how Starbucks introduced oatmeal breakfasts in just a few carefully picked stores before expanding to a large market?  This is a technique also used routinely in the retail industry.

3. We are successfully taking actions on our predictions, so we are done

Predictive models require constant care. All analytic models start off with environmental assumptions. These assumptions need to be constantly re-evaluated and the models need to be updated as the environmental factors change.

Lacking this, the models degrade over time and lose their predictive power. While lifecycle management of enterprise software protects existing enterprise assets from degrading in business value, lifecycle management of predictive models also preserves customer loyalty, competitive edge and revenue. Budgeting for the lifecycle management of predictive models is key when embarking on Big Data projects.

About Sai Devulapalli

Sai Devulapalli is an accomplished leader in defining and driving product strategies, business development and go-to-market execution in disruptive technology domains such as Data Analytics, Internet of Things and Enterprise PaaS in telecom, database and storage industries over the past 20 years. Devulapalli is currently responsible for global business and partnership development for data analytics product portfolio at EMC’s Emerging Technology Division and is a Writer for icrunchdata News Dallas, TX.