Machine Learning Times
Machine Learning Times
Effective Machine Learning Needs Leadership — Not AI Hype
 Originally published in BigThink, Feb 12, 2024.  Excerpted from The...
Today’s AI Won’t Radically Transform Society, But It’s Already Reshaping Business
 Originally published in Fast Company, Jan 5, 2024. Eric...
A University Curriculum Supplement to Teach a Business Framework for ML Deployment
    In 2023, as a visiting analytics professor...
The AI Playbook: Providing Important Reminders to Data Professionals
 Originally published in DATAVERSITY. This article reviews the new...

By: Rob F. Walker, Ph.D., Vice President of Decision Management, Pegasystems
Originally published at computerworld 


If we were able to predict the future, we would be able read the minds of our customers and know exactly what they wanted.  The closest we can get to this supernatural ability is utilizing predictive analytics to tap into the hidden treasures that reside in big data that every organization is wrestling with nowadays. Predicting the future is often seen as a sort of “dark art” that has its fair share of doubters.  Predictive analytics, however, is a proven technology which mines data for repeatable patterns that are reliable enough to use as a basis for predicting future events.  When done right (and more about that later), predictive analytics can become a crystal ball that can enable organizations to make better decisions.

Providing today’s organizations with the ability to utilize predictive analytics takes robust technology, as personnel alone can not begin to even tackle such an objective.  It will take automation.  Research has shown that by 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills, as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.   Automation means that the current structure of very manual predictive laboratories will need to evolve into much higher throughput analytics factories that reliably analyze big data to predict the next best action for given business functions.

The uses for predictive analytics are plenty – banks use predictive analytics for loan applications and can calculate the probability of default.  Many companies use the capability to increase customer lifetime value by making relevant recommendations that enable businesses to maintain and evolve their relationships with customers.  Today, we are able to predict customer behavior to a degree of accuracy that’s vastly superior to human judgement or assumptions with subsequent growth in customer lifetime value. But… predictive analytics is dependent on historical data to build models  that are able to reliably predict the future when applied to new data.

Challenges that can sidetrack predictive models

Like any science, predictive analytics can run off the tracks based on some of the challenges that practitioners need to be aware of.  These challenges are:

1.       Data volume – According to estimates, the volume of business data worldwide, across all companies, doubles every 1.2 years. Insofar these volumes are an indication of change predictive models need to keep up or they can risk becoming obsolete. Best practice is to have models monitor themselves, continually checking whether the data that was used to develop them is still representative of the data they are being applied to. On the opposite side, if a model requires too much data to find a pattern, it may be too complex to provide reliable predictions.

2.       Data quality – Data needs to be useful in order to produce the desired results.  Data must be automatically analyzed and manipulated in order for it to be safe for vigorous modeling. Predictive analytics solutions need to address two key modeling objectives in order to be successful – they need to balance both accuracy and stability in order to yield high data quality.  That said, many companies waste precious time and money on grooming perfect data where they could easily have monetized what they already have. State of the art predictive analytics is quite ‘reflective’ of its own performance; if the data cannot reliably be mined for an actionable pattern, modern tools will admit defeat rather than generate garbage.

3.       Model complexity – It is easy to get carried away when developing data models.  Data miners need to keep their predictive models simple as simple as possible, just not simpler. Modern predictive analytics tools can give proper guidance. Businesses need to take a top-down modeling approach to achieve the best results – simpler models are less prone to changes in the data and are more robust.  In the long run, they will prove to be much more cost effective and quicker to implement.

4.       Model usability – Businesses need to be able to use models in order for them to be effective. Otherwise data mining is just an expensive hobby. The end goal, the desired improvement in outcomes, must be clear in one’s mind when developing models.  This goes back to the model factory, versus laboratory, concept discussed earlier. Businesses cannot afford scarce resources to do anything but delivering actional intelligence. IDC estimates that by 2020,business transactions on the internet, business-to-business and business-to-consumer, will reach 450 billion per day.  Those transactions  demand  better decisions and many of those decisions will require high quality predictions.

Delivering value to the business

Predictive analytics can only be actionable  if incorporated into operational processes.  Businesses do not just need the power to know, they need the power to act.  If they are able to work through the challenges highlighted above, they will be well on their way to developing the crystal ball that is proven to deliver tremendous value and insight that drives growth.

By using predictive models to assess and anticipate needs, risks and aspirations, organizations are able to provide next-best-action recommendations during customer interactions in real-time, either in-person or through self-service interactions.  Such powerful capabilities enable businesses to reduce churn by addressing customer needs, or assess risk moving forward (such as for mortgage or credit applications or insurance underwriting). How could such a crystal ball affect your business?  In the next installment, we’ll discuss how predictive analytics can provide next-best-action capabilities to help fuel business growth.

By: Rob F. Walker, Ph.D., Vice President of Decision Management, Pegasystems
Originally published at computerworld

Leave a Reply