5 years ago
What Twitter Learned From The Recsys 2020 Challenge

Originally published in Towards Data Science on Oct 26, 2020.

This year, Twitter sponsored the RecSys 2020 Challenge, providing a large dataset of user engagements. In this post, we describe the challenge and the insights we had from the winning teams.

Recommender systems are an important part of modern social networks and e-commerce platforms. They aim to maximise user satisfaction as well as other key business objectives. At the same time, there is a lack of large-scale public social network datasets for the scientific community to use when building and benchmarking new models to tailor content to user interests. In the past year, we have worked to address exactly that problem.

Twitter partnered with the RecSys conference to sponsor the 2020 challenge. We released a dataset consisting of tweets and user engagements over a period of two weeks, with 160 million public tweets for training and 40 million public tweets for validation and testing over a period of two weeks.

In this post, we describe the dataset and the three winning entries submitted by Nvidia, Learner, and Wantely teams. We try to make general conclusions about the choices that helped the winners achieve their results, notably:

• most important features
• extremely fast experimentation speed for feature selection and model training
• adversarial validation [1] for generalisation
• use of content features
• use of decision trees over neural networks

We hope that these findings will be useful to the wider research community and inspire future research directions in recommender systems.

To continue reading this article, click here.