Machine Learning Times
Machine Learning Times
Three Best Practices for Unilever’s Global Analytics Initiatives
    This article from Morgan Vawter, Global Vice...
Getting Machine Learning Projects from Idea to Execution
 Originally published in Harvard Business Review Machine learning might...
Eric Siegel on Bloomberg Businessweek
  Listen to Eric Siegel, former Columbia University Professor,...
Effective Machine Learning Needs Leadership — Not AI Hype
 Originally published in BigThink, Feb 12, 2024.  Excerpted from The...

9 months ago
Experimenting with Machine Learning to Target In-App Messaging

Originally published in Spotify R&D Engineering, June 28, 2023.

Messaging at Spotify

At Spotify, we use messaging to communicate with our listeners all over the world. Our Messaging team powers and creates delightful foreground and background communications across the Spotify experience, experimenting with and tailoring the perfect journey for each user across our platform. Today we are able to send messages through WhatsApp, SMS, email, push notifications, contextual in-line formats, and other in-app messaging formats, like modals and full-screen takeovers.

In-app messaging at Spotify

What we refer to as in-app messaging covers a range of different message formats, with the unifying trait that they all appear when the user is using the app. See below for some example in-app messages.

This is possibly the most direct method we have for communicating with our users, which also means we need to be careful not to interrupt their listening experiences unnecessarily. We deliberately withhold in-app messaging for some users so that we can measure its overall effectiveness. These holdouts show that in-app messages have a mixed effect on user behavior when looking at the whole population, demonstrating that we would benefit from targeting users more selectively for in-app messaging.

We believed that we could use ML to decide in-app messaging eligibility and that by doing so we could improve user experience without harming business metrics. We discussed a number of possible solutions to this problem but decided to focus on uplift modeling, where we try to directly model the effect of in-app messaging on user behavior.

Heterogeneous treatment effect & uplift modeling

It’s clear that in-app messages have a different effect on different users, which is known as a heterogeneous treatment effect. We have some users that might enjoy Spotify Premium and would benefit from receiving a message prompting them to subscribe. We also have users that are happy with their current product offering, where messaging wouldn’t benefit either the user or Spotify. Our task, then, is to predict the effect of in-app messages on users. In particular, we wanted to understand the causal effect of sending in-app messages.

To continue reading this article, click here.

One thought on “Experimenting with Machine Learning to Target In-App Messaging

  1. Mapquest Directions is more than just a map service. Users can also search for nearby places, such as hotels, restaurants, shops, gas stations, and more. Users can also explore the world with live traffic and transit updates, satellite and street views, and 3D maps. Whether users need to drive, walk, or take public transportation, mapquest can help them get there faster and easier.


Leave a Reply