Archive for January, 2015

January 29th 2015

Wise Practitioner – Predictive Analytics Interview Series: Sarah Holder of Duke Energy

By: Eric Siegel, Founder, Predictive Analytics World

In anticipation of her upcoming conference presentation, It’s Not a Black Box! Explaining Sarah_HolderPredictive Models to the Masses, at Predictive Analytics World San Francisco, March 29-April 2, 2015, we asked Sarah Holder, Senior Market Research Analyst at Duke Energy a few questions about her work in predictive analytics.

Q:  In your work with predictive analytics, what behavior do your models predict?

A:  Besides being a utility, Duke Energy also offers Warranty and Energy Efficiency Programs.  In the Marketing Analytics Group, we currently use Predictive Analytics to target Direct Marketing offers through Direct Mail, Email, and our Call Center.

Q:  How does predictive analytics deliver value at your organization? What is one specific way in which it actively drives decisions?

A:  Predictive Models drive our “Smart Window.”  This Window pops up when a Call Center Representative pulls a Customer’s information.  A customer may be calling in to star service at an account, or to inquire about a change in their bill amount.  In the Smart Window, there is a list of top products for which the customer qualifies and star ratings to signify the probability of the customer’s interest.  Because the representatives have information on the customer, it helps them make a knowledgeable sales pitch.

Q:  Can you describe a successful result, such as the predictive lift of your model or the ROI of an analytics initiative?

A:  Duke Energy previously used external vendors for Targeting Direct Mail lists.  Once we moved the process to our internal Analytics group and used Logistic Regression to predict the best customers, our Load Control program’s direct mail response rates increased by 261%!

Q:  What surprising discovery have you unearthed in your data?

A:  Contrary to a previous belief, we found that households with two occupants tend to use more energy as a whole when compared to households with 3 or more occupants.  The number of people in the household may be a clue towards the life stage of the occupants.  When there are children present, the home may not be occupied as frequently during the day, leading to lower energy use overall.

Q:  Sneak preview: Please tell us a take-away that you will provide during your talk at Predictive Analytics World.

A:  I will briefly explain the success story of predictive modeling within our marketing department.  I will also review some tips for communicating the benefits of predictive models verses purchased segmentation systems for targeting customer direct marketing lists.

————

Don’t miss Sarah Holder’s conference presentation, It’s Not a Black Box! Explaining Predictive Models to the Masses, at Predictive Analytics World San Francisco, on March 31, 2015, from 3:55-4:15 pm. Click here to register for attendance.

By: Eric Siegel, Founder, Predictive Analytics World

No Comments yet »

January 20th 2015

Wise Practitioner – Predictive Analytics Interview Series: Mohamad Khatib of Nielsen

By: Eric Siegel, Founder, Predictive Analytics World

In anticipation of his upcoming conference presentation, Pizza Analytics & Optimization, at Mohamad_KhatibPredictive Analytics World San Francisco, March 29-April 2, 2015, we asked Mohamad Khatib, Sr. Project Manager at Nielsen, a few questions about his work in predictive analytics.

Q: In your work with predictive analytics, what behavior do your models predict?

A: The work I have done focused on predicting customer purchases, in response to targeted advertisements and marketing promotional offerings. The aim is to help manufacturers optimize their product marketing promotional spend by predicting customer responses to different promotions.

Q: How does predictive analytics deliver value at your organization? What is one specific way in which it actively drives decisions?

A: By utilizing predictive analytics, the small business we worked with (pizza shop: manufacturer and retailer) were able to better schedule promotional campaigns that will yield the best results, and attract targeted customers.

Having informed predictions enables these pizza outlets to optimize their inventories of promoted products. In addition, they guide production staffing plans so that pizza outlet managers can schedule resources appropriately to meet the changing demands. This will help to ensure effective delivery of these products to clients.

Q: Can you describe a successful result, such as the predictive lift of your model or the ROI of an analytics initiative?

A: By applying predictive analytics, the engaged pizza shops were able to effectively plan for their staffing needs, and align that with targeted promotions carried out to clients.

As predicted, one pizza shop was prepared to respond to increased demand in response to the post card promotions. As responses exceeded initial projected rates in some cases, additional staffing plans were in place to meet the increased demands and carry out the required service levels.

Also, they were able to compare effectiveness of types of promotional campaigns as applied to their targeted demographics.

Q: What surprising discovery have you unearthed in your data?

A: Even though we utilized simplified models to provide predictive analytics, the correlation between promotional activities and client responses was easily visible and measurable. This ease of quantification of results in the pizza business produces great benefits in directing promotional activities to maximize returns on spends.

Q: Sneak preview: Please tell us a take-away that you will provide during your talk at Predictive Analytics World.

A: A simplified predictive analytics model can be easily developed and applied to a small business. Such limited efforts and investment will bring positive returns to manufacturers.

This simplification does not trivialize the approach, and has proven to produce valuable results and insights to predict responses to targeted pizza promotions.

In addition, it is clear that successful deployment of predictive analytics will have cross-functional impact throughout the organization. This impact applies to large as well as small organizations. Respective business processes will be impacted accordingly.

—————

Don’t miss Mohamad Khatib’s conference presentation, Pizza Analytics & Optimization, at Predictive Analytics World San Francisco, on Wednesday, April 1, 2015 from 10:25-10:45 am. Click here to register for attendance.

By: Eric Siegel, Founder, Predictive Analytics World

No Comments yet »

January 16th 2015

Wise Practitioner – Predictive Analytics Interview Series: Dominic Fortin of TD Insurance

By: Eric Siegel, Founder, Predictive Analytics World

In anticipation of his upcoming conference presentation, A Success Story: Sales and Revenue Forecasting through Predictive Analytics, at Predictive Analytics World San Francisco, March 29-April 2, 2015, we asked Dominic Fortin of TD Insurance a few questions about his work in predictive analytics.

Q:  In your work with predictive analytics, what behavior do your models predict?

A:  The models are predicting sales, cancellations, renewals for general insurance.  Units and premiums are forecasted, as well.  Twenty-three variables are predicted in 92 variances (region, product, sales channel, insurer, etc…), each level with its own specificities, creating 2,116 different models.  The models encompass all the changes to the business (rate change, new project, marketing investments). The innovation was that we were able to create generalized models without having to create 2,116 models independently.

Q:  How does predictive analytics deliver value at your organization? What is one specific way in which it actively drives decisions?

A:  It allows us to not only to do our sales & revenue budget, to forecast our upcoming results but also to do scenarios on initiatives (e.g. effect of a rate change, etc…).  This capability helps the company to make sound decisions on business initiatives.

Q:  Can you describe a successful result, such as the predictive lift of your model or the ROI of an analytics initiative?

A:  We had models in the past based on excel, they were doing "the job" on predicting our sales & revenue.  The new models add more flexibility and rapidity and lower the risks associated with a manual excel based forecast.

Q:  What surprising discovery have you unearthed in your data?

A:  There was no real surprise.  We mastered our data well, it is more the capability to forecast at a more granular level that allows us to be more precise rather than trying to predict at high level.

Q:  Sneak preview: Please tell us a take-away that you will provide during your talk at Predictive Analytics World.

A:  A take-away is the assistance to think about how this development method could be used to develop a large number of models for your business domain.

Don't miss Dominic Fortin’s conference presentation, A Success Story: Sales and Revenue Forecasting through Predictive Analytics, at Predictive Analytics World San Francisco, on April 1, 2015 at 3:55-4:15 pm. Click here to register for attendance.

 

By: Eric Siegel, Founder, Predictive Analytics World

No Comments yet »

January 6th 2015

Wise Practitioner – Predictive Analytics Interview Series: Pasha Roberts at Talent Analytics

By: Greta Roberts, Conference Chair, Predictive Analytics World for Workforce 2015

 

In anticipation of his upcoming conference presentation, A Transaction-Based Approach to Understand Sales Representative Growth, Performance, and Gaming, at Predictive Analytics World for Workforce, we interviewed Pasha Roberts, Co-Founder and Chief Scientist at Talent Analytics Corporation. View the Q-and-A below to see how Pasha has incorporated predictive analytics into the workforce of Talent Analytics Corporation. Also, glimpse what’s in store for the new PAW Workforce conference.

Q: In your work with predictive analytics, what specific areas of the workforce are you focused on?

A: We work with clients to solve workforce problems, using a data science approach to predict the ways that an employee will behave and perform. Only rarely are the problems neatly defined, but it usually comes down to estimating tenure or performance factors. We develop and deploy these models in the cloud to inform candidate selection and internal operations.

Q: Do you primarily work inside of HR – or inside of the Line of Business? If Line of Business – which one(s)?

A: Occasionally we find an HR department that is in sync with the vast potential of predictive analytics. These people are wonderful, and are earning their place at the future of their business; several are speaking at PAW for Workforce this year. Most HR departments have not crossed this chasm; they feel that their scope is more limited.

The line-of-business tends to actually feel the actual pain, and is typically more facile with data. We often work with the Sales, Service, or Call Center Lines of Business. These operations managers are on the front line and are accountable.

Q: What workforce outcomes do your models predict?

A: We create models that predict tangible outcomes – such as dollar sales, or calls per day, or hours worked, or error rates, or tenure/survival. The business leaders decide what is important to success in a specific role, and then we build models for those factors. For example, we may model the likelihood of a job candidate to achieve a top sales level within 6 months.

I am not a big believer in working with intermediate variables, such as engagement or job satisfaction. You can’t eat engagement. People may assume that engagement drives business performance, but that link needs to be proven case by case. At that point, you might as well predict the business performance directly.

Q: What is one specific way in which predictive analytics actively drives decisions?

A: We recently deployed a model to reduce bank teller attrition at a large bank. The model predicts the Cox proportional hazard for survival in the role, based on our aptitude tool metrics.

Advisor, our deployment platform, displays the probability of a job candidate to be on the job in one year. Recruiters use this number (actually cutoff levels of this number) to move candidates forward, or not, during hiring.

Q: Can you describe a successful result, such as the predictive lift of one of your models or the ROI of a predictive analytics initiative?

A: The AUC of the attrition model above is quite good, well over 0.70, with lift over 2.0 in the important regions.

The business benefit of this is two-fold – (1) fewer bad fits, which means less wasted training due to early attrition, and (2) more good fits, which means longer employee runs and higher lifetime value. The total benefit of this incremental change will be over half a million dollars.

Q: What is an example of surprising discoveries you have unearthed in your data?

A: Cluster analysis of a recent sales assignment revealed a clear group of sales reps who were succeeding largely by cheating. They were “poaching” sales from regions assigned to other reps, pulling low-hanging fruit away from others. Reps in this cluster barely worked any hours, but had very high performance numbers.

It just goes to show that you can’t use just one KPI. And yes, we had great lift in predicting job candidates who were likely to fall into this poaching cluster.

Q: What area of the workforce do you think has seen (or will see) the greatest advances or ROI from the use of predictive analytics?

A: The best predictive analytics can be done with largest sample sizes and quantified, quality output variables. We see the most of this in high volume roles, such as call centers, sales, retail banking, and insurance.

The costs and benefits of attrition/under-performance drive the ROI. In high volume situations, even a small increment can make millions of dollars of difference.

Q: Why do you think Business Leaders, HR Leaders and Analytics professionals should attend Predictive Analytics World for Workforce?

A: We need business leaders who understand what predictive analytics is, what it can do, and how to leverage the methods. It requires thinking about the world in a slightly different way, in terms of probability. This is not as hard as it may sound, and it is the key to unlocking a whole new level of corporate quality and performance.

Analytics practitioners should come to PAW for Workforce to learn – not only methods and technology, but to the sometimes-harder problem of business application and deployment. It’s the difference between solving equations and solving word problems.

Q: Do you feel any urgency you want to pass along to your fellow HR and Business Executives to implement predictive analytics to help solve employee challenges? Why?

A: I would like practitioners to realize the fact that there is a rare opportunity in employee analytics. Analytics can be so much more effective in hiring, because we directly choose employees, therefore predictions can directly drive results. This is a luxury that most other forms of analytics do not have – we don’t choose our customers, for example, so our ability to drive marketing results is indirect at best.

Q: What is one misunderstanding people have about using predictive analytics to solve employee challenges?

A: We predict patterns that tend to happen over time. We don’t predict what will happen in each specific case at each moment.

For example, if we predict that an employee has a 36% chance of staying on the job for one year; it’s still possible that they will last on the job for years. This doesn’t mean we’re wrong, because other employees with the same pattern may compensate.

As a machine learning geek, I love being “wrong” in this way, and love it more when managers disobey the models, because these variances only make new models stronger on the next iteration.

Q: How involved has the business unit been in the work you’ve done inside of your organization?

A: So far, very involved. The impact of our work boils down to understanding risk and tradeoffs, which is something senior managers often understand better than line managers.

You just have to speak their language, instead of going on about your latest foray into conditional random forest algorithms or how many Hadoop nodes were used.

Q: SNEAK PREVIEW: Please tell us a take-away that you will provide during your presentation at Predictive Analytics World for Workforce.

A: I am working with millions of geo-located sales transactions to discover and predict patterns in how they learn to sell. It is a fascinating dataset that chronicles thousands of reps as they sink or swim, including some who cheat their way to the top. I hope to deliver insights into how employees learn, as well as techniques to analyze large unaggregated data sets.

 

Don't miss Pasha Roberts’ conference presentation, A Transaction-Based Approach to Understand Sales Representative Growth, Performance, and Gaming, at PAW Workforce, on Wednesday, April 1, 2015, from 11:15 am – 12:00 pm. Click here to register for attendance.

 

By: Greta Roberts, CEO, Talent Analytics, Corp. @gretaroberts and Conference Chair of Predictive Analytics World for Workforce.

 

No Comments yet »