Wise Practitioner – Predictive Analytics Interview Series: Josh Hemann of Activision
By: Eric Siegel, Founder, Predictive Analytics World
In anticipation of his upcoming conference presentation, Cheating Detection in Call of Duty, at Predictive Analytics World San Francisco, March 29-April 2, 2015, we asked Josh Hemann, Principal, Game Analytics at Activision, a few questions about his work in predictive analytics.
Q: In your work with predictive analytics, what behavior do your models predict?
A: My team focuses on integrating analytics into large scale, operational processes for our online, multiplayer games, and mostly for the Call of Duty franchise. A lot of that work is more about optimization rather than prediction per se. But one area that is certainly prediction/classification is algorithmically detecting cheating, which can encompass a lot of different behaviors to identify in various settings. In some settings, the prediction absolutely must happen in real-time; in others, we can do batch processing and build evidence over time.
Q: How does predictive analytics deliver value at your organization? What is one specific way in which it actively drives decisions?
A: We have various statistical models to characterize players’ engagement, their likelihood to quit playing, and even their styles. The process of fitting and testing these models teaches us a lot about how different elements of game play affect our players, which in turn helps inform design decisions for future game features.
Q: Can you describe a successful result, such as the predictive lift of your model or the ROI of an analytics initiative?
A: In algorithmically detecting cheating the main benefit has been scale. Before using algorithms the manual review of player telemetry data could lead to at most a couple hundred cases being identified each day. Now we can act on thousands of cases per day, leading to a healthier player community while also freeing up valuable time for my colleagues to focus on other areas.
Q: What surprising discovery have you unearthed in your data?
A: We collect a lot of telemetry when people play online, everything from where you are on a map at a particular point in time to how many shots you fired with a particular weapon. So the following is not so much a surprise as it is my constant amazement: I only get to work with these data because there is physics code, graphics code, server code, etc. that is rendering many tens of thousands of events for each player in a single game, at 60 frames per second, and this is happening for millions of games played every day all over the world.
Q: Sneak preview: Please tell us a take-away that you will provide during your talk at Predictive Analytics World.
A: It’s easier to make predictive analytics useful when you give a lot of attention up front to the business decisions you are trying to affect and what monetary value there is in improving them.
Don't miss Josh Hemann’s conference presentation, Cheating Detection in Call of Duty, at Predictive Analytics World San Francisco, on Wednesday, April 1, 2015, from 10:00-10:20 am. Click here to register for attendance.
By: Eric Siegel, Founder, Predictive Analytics World
Eric Siegel, Ph.D., founder of Predictive Analytics World and Text Analytics World, and Executive Editor of the Predictive Analytics Times, makes the how and why of predictive analytics understandable and captivating. He is the author of the bestselling, award-winning Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, a former Columbia University professor, and a renowned speaker, educator, and leader in the field.