Machine Learning Times
Machine Learning Times
EXCLUSIVE HIGHLIGHTS
MLW Preview Video: Dean Abbott, President at Abbott Analytics
 In anticipation of his upcoming pre-conference workshop and presentations...
MLW Preview Video: Evan Wimpey, Director of Strategic Analytics at Elder Research
 In anticipation of his upcoming presentation at Predictive Analytics...
Wise Practitioner – Machine Learning Week Interview: Mack Wallace, Head of Financial Products at MPOWER Financing
 In anticipation of his upcoming presentation at Predictive Analytics...
Wise Practitioner – Machine Learning Week Interview: Alejandro Jesús Castañeira Rodriguez, Principal Data Scientist at JANZZ Ltd.
 In anticipation of his upcoming presentation at Predictive Analytics...
SHARE THIS:

2 months ago
Distributed Machine Learning at Instacart

 
Originally published in Tech At Instacart, March 17, 2023.

At Instacart, we take pride in offering a diverse range of machine learning (ML) products that empower every aspect of our marketplace, including customers, shoppers, retailers, and brands. Along with the growth of business and ML applications, we have encountered an increasing number of use cases that require distributed ML techniques to effectively scale our ML products.

To support the emerging requests, we have designed the distributed ML system with the following key design considerations:

  • Scalability: We must be able to scale a large number of distributed ML workloads on both CPU and GPU instances with a robust system that maintains high performance and reliability.
  • Resource Efficiency: We aim to fully utilize distributed computation resources to maximize system throughput and achieve the fastest execution at optimal cost.
  • Diversity: The selected computation framework should be as extensible as possible to support diverse distributed ML paradigms. In addition, we should be able to handle diverse environments that are specific to different ML workloads.

Our team is currently working on a variety of ML applications that utilize the power of distributed computing to efficiently solve complex problems.

To continue reading this article, click here.

3 thoughts on “Distributed Machine Learning at Instacart

Leave a Reply