3 years ago
Machine Learning is Going Real-Time

Originally published in Chip Huyen, Dec 27, 2020.

After talking to machine learning and infrastructure engineers at major Internet companies across the US, Europe, and China, I noticed two groups of companies. One group has made significant investments (hundreds of millions of dollars) into infrastructure to allow real-time machine learning and has already seen returns on their investments. Another group still wonders if there’s value in real-time ML.

There seems to be little consensus on what real-time ML means, and there hasn’t been a lot of in-depth discussion on how it’s done in the industry. In this post, I want to share what I’ve learned after talking to about a dozen companies that are doing it.

There are two levels of real-time machine learning that I’ll go over in this post.

• Level 1: Your ML system makes predictions in real-time (online predictions).
• Level 2: Your system can incorporate new data and update your model in real-time (online learning).

I use “model” to refer to the machine learning model and “system” to refer to the infrastructure around it, including data pipeline and monitoring systems.

To continue reading this article, click here.