As every year since 2014, Criteo went to RecSys 2022! We were happy and proud to be a gold sponsor of this conference, especially as it had a lively onsite part in the beautiful city of Seattle.

Criteo and Recommendation Systems

Many know Criteo as a world leader in AI-driven Performance Advertising (now subsumed in our grand vision of Commerce Media). Still, few know that even before Criteo, there was a recommendation system. Indeed, the company was founded on a shiny new movie recommendation system. The tech worked, but the business outcomes were limited, and after several pivots, the company settled in the ad tech domain, where it thrived.

More than 15 years later, the recommendation engine is still very much at the heart of our systems. Presenting relevant products throughout their shopping journey to internet users is at the core of our vision on Commerce Media. (Personalized Product Recommendations | Criteo ) We deploy many state-of-the-art technologies at an internet scale for fast and accurate product recommendations. Most of the key technologies we use are open source and available on our GitHub

• R-SVD (SparkRSVD open-sourced by Criteo for large scale recommendation engines )
• DeepR (The deepr module provide abstractions to help build tensorflow models on top of tf estimators )
• AutoFaiss (Introducing Autofaiss: An Automatic K-Nearest-Neighbor Indexing Library At Scale )
• RecoGym (Code for reco-gym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising )

Share and enjoy! And send pull requests our way!

Key Take Aways

Our team very much enjoyed the conference, the main scientific track, the Tutorials, and the Workshops. All of those are of the highest quality, and we returned with stars in our eyes and ideas in our brains.

Let’s start with our favorite not-technical keynote: AI-Mediated Communication by Mor Naaman!

AI often operates on behalf of a human communicator by recommending, suggesting, modifying, or generating messages to accomplish communication goals. We call this phenomenon AI-Mediated Communication (AI-MC)

In his talk, the speaker presented his research and showed that AI-MC could have several negative impacts:

• Language shifting towards positivity: Autocomplete messages are often positive and therefore switch the initial writer’s intent to something more positive. AI is biasing communications.
• Impact on the evaluation of others: Sending AI-assisted or auto-generated messages makes one more trusted and liked by the recipient of his message compared to not AI-assisted messages. However, the simple suspicion from the recipient that the sender used AI to write his message will lead to a loss of trust from the recipient. This phenomenon is called the replicant effect (yes, Blade Runner!). For Criteo, it is important to keep the users’ trust so that they feel confident to click on the ads they like, so we need to make sure that users don’t think the ad looks fake or that the text/image was generated with AI.
• Change the extent to which we take ownership over our messages: With biased AI-based autocomplete suggestions, you can increase the proportion of users that write that they agree with a specific topic. Also, people writing AI-assisted messages tend to write twice more as others: it brings imbalance in the discussions and shows that humans easily let AI write for them more than what they would have written. Combined with the positivity bias, changing a question in a pool to a negative form can change the answers, scarring, right?
• Shift assignment of blame for communication outcomes: When an issue happens in a group, a user writing AI-assisted messages will more often share the burden for the failure. In contrast, a non-AI-assisted user would more often blame another person.

This doesn’t look good, and everyone in the AI world should pay great attention to these topics. Fortunately, there are ways to improve the situation:

• Transparency/disclosure
• Define and prevent harmful use
• Measure and monitor bias
• More regulations

Good to know: Criteo is aware of this kind of bias and monitors them! Our policy is to be transparent to our clients and users so that they gain trust in us. We also pledge to display ads only on publishers’ websites that respect the Criteo values (No porn, no propaganda, no gambling, no firearms, …), and we don’t recommend sensitive products. Fairness is also a hot topic in our recommendation teams, and we are hiring.

I thank RecSys for bringing such a subject to the table. AI should not always be about maximizing a score…

… and sometimes it is! Let’s look at some cool insights from the REVEAL workshop talks!

Here is a list of the key takeaways we got:

• Most companies use a mix of online and offline/batch recommendations to improve availability and robustness and optimize the cost. This is the case of Amazon’s book, which presented their recommendation architecture briefly. And the case of Criteo too!
• Warming-up models with off-policy data speeds-up greatly training time.
• For recommendation systems, only a few user clicks are required to get a high NDCG metric for the recommendation engine. This property is great for Criteo because it proves we can do very good ads with very little data.
• Using a Polyak averaging optimizer seems very efficient for Amazon book Papers with Code — Polyak Averaging Explained.
• For Google, their favorite optimizer is Shampoo: Preconditioned Stochastic Tensor Optimization. It can converge considerably faster than commonly used optimizers while having a runtime per step comparable to simple gradient methods such as SGD, AdaGrad, and Adam. It involves a more complex update rule tough, and Google’s overhead is +10% computation cost.
• Cleaning and preprocessing data is key to speeding up and improving NN models. Google is sampling data to keep only the examples that are significantly different from one another. At the same time, Amazon book regroups variants of the same book (audio, e-book, book) under the same label to get more signals from the training data.
• DCN layers are very efficient bricks to handle dense and discreet features with the same model
• Distillation methods based on the teacher-student model are used at google and speed up their models.
• Google uses autoML libraries to approximate the weight matrix into the low-rank matrix. This makes it possible to reduce the memory usage of the models.
• The best trick to reduce training time is to sample training data in a relevant way

Here are also some presentations that we liked:

Merlin

NVDIA introduces its framework Merlin to build four-stages (retrieval, filtering, scoring, and ordering) recommender systems at scale and intuitively. The framework provides benefits for model development, evaluation, and deployment with its libraries NVTABULAR for feature engineering and preprocessing, HugeCTR for distributed Click-Through-Rate estimation, and Models that offer implementations of state-of-the-art models for retrieval (e.g., matrix factorization, two tomer, …) and ranking (e.g., DLRM, DCN-v2 …) in TensorFlow and PyTorch and Systems to ship pipelines to production.

RecSys challenge 2022

We’ve attended the 2022 RecSys Challenge workshop.

The challenge was organized by Dressipi (Dressipi: Personalized Ecommerce ), an AI expert company that builds recommender systems for the fashion industry. The challenge’s goal was about accurately to predict which fashion item would be bought at the end of a user’s session from a 1.1 million per-session samples dataset that includes items viewed in a session, purchases at the end of a session, and items features (color, sleeve length, etc.). For more information about the dataset, visit http://www.recsyschallenge.com/2022/dataset.html.

A big congrats to the winner who successfully tackled the task using GNNs and Lightgbm (https://dl.acm.org/doi/abs/10.1145/3556702.3556850).

Criteo Party

Work hard, play hard! In addition to its strong technical and scientific expertise, Criteo is proud to share its culture with the RecSys attendees!

Criteo’s values are Open, Together Impactful, and during RecSys, we wanted to share an OPEN evening with all participants: the Garage Bowling.

The first part of the evening was dedicated to networking, meeting, and exchanging. A TOGETHER moment to break the ice, get to know each other, exchange on the first days of RecSys, the conferences…

The second part of the evening focused on bowling, with a space privatized for the group. Many strikes were counted — to show the IMPACTFUL side of the evening 😉

We were thrilled to be able to share this moment with the RecSys participants and look forward to next year’s event!

With its unique combination of academia and industry, great science and cool tech, excellent atmosphere, and outstanding community spirit, RecSys is one of our favorite conferences. We look forward to seeing the crowd again in Singapore in 2023!

---

Want be at the boot with us next year? Apply to our open positions:

Careers at Criteo | Criteo jobs

Find opportunities everywhere. ​Choose your next challenge.

careers.criteo.com