Machine Learning Times
Machine Learning Times
Video – How to Use AI Ethically from Natalia Modjeska of Omdia
 Event: Machine Learning Week 2021 Keynote: How to Use AI...
Video – Alexa On The Edge – A Case Study in Customer-Obsessed Research from Susanj of Amazon
 Event: Machine Learning Week 2021 Keynote: Alexa On The Edge...
Why AI Isn’t Going to Replace Data Scientists Any Time Soon
 Should data scientists consider AI a threat to their...
“Doing AI” Is a Mistake that Detracts from Real Problem-Solving
  A note from Executive Editor Eric Siegel: Richard...

12 months ago
Switchback Tests and Randomized Experimentation Under Network Effects at DoorDash

Originally published in DoorDash Engineering Feb 13, 2018.

To A/B or not to A/B, that is the question


On the Dispatch team at DoorDash, we use simulation, empirical observation, and experimentation to make progress towards our goals; however, given the systemic nature of many of our products, simple A/B tests are often ineffective due to network effects. To be able to experiment in the face of network effects, we use a technique known as switchback testing, where we switch back and forth between treatment and control in particular regions over time. This approach resembles A/B tests in many ways, but requires certain adjustments to the analysis.

Dispatch at DoorDash

The core responsibility of the Dispatch system at DoorDash is to power fulfillment of every delivery in Doordash’s three-sided marketplace of Consumers, Dashers and Merchants. To effectively achieve this, we focus on

  1. the core assignment problem — deciding which dasher is the best suited to fulfill a delivery, in an efficient, robust and scalable way.
  2. machine learning algorithms to predict the numerous timepoints in the life of a delivery — “when can this order reach the consumer”, “when will this order order be ready for pickup?”, etc.
  3. algorithms to decide how and when to group multiple deliveries headed in similar directions at similar times
  4. how to leverage various marketplace shaping strategies to balance supply and demand, including pricing changes.

To continue reading this article, click here.

Leave a Reply