Follow us to Baltimore….

The 39th International Conference on Machine Learning (ICML) took place in Baltimore, the USA, from 17th to 23rd July. ICML is one of the most prestigious conferences in machine learning, and covers a wide range of topics in machine learning from both practical and theoretical viewpoints.

As one of the largest industrial AI Labs in Europe ICML is an extremely important event for us as it allows us to present our latest research to the community and also to learn about the latest and greatest ideas which might impact our ML powered production systems.

This year it brought together 4000 researchers and practitioners from machine learning for :

• 15 tutorials
• more than 200 sessions of talk/poster presentations
• 1183 papers
• 31 workshops

Our Contributions

Criteo was especially proud to have contributed to 6 full papers accepted at ICML 2022 :

• “Generalizing to New Physical Systems via Context-Informed Dynamics Model” — Matthieu Kirchmeyer, Yuan Yin, Jérémie Dona, Nicolas Baskiotis, Alain Rakotomamonjy, and Patrick Gallinari
• “A Neural Tangent Kernel Perspective of GANs” — Jean-Yves Franceschi, Emmanuel de Bézenac, Ibrahim Ayed, Mickael Chen, Sylvain Lamprier and Patrick Gallinari
• “Nested Bandits” — Matthieu Martin, Panayotis Mertikopoulos, Thibaud Rahier, and Houssam Zenati
• “UnderGrad: A Universal First-Order Optimization Method with Almost Dimension-Free Convergence Rate Guarantees” — Kimon Antonakopoulos, Dong Quan Vu, Volkan Cevher, Kfir Levy, and Panayotis Mertikopoulos
• “AdaGrad Avoids Saddle Points” — Kimon Antonakopoulos, Panayotis Mertikopoulos, Georgios Piliouras, and Xiao Wang
• “Scalable MCMC Sampling for Nonsymmetric Determinantal Point Processes.” — Mike Gartrell, Insu Han, Elvis Dohmatob, Amin Karbasi

You can find all our papers here: Publications — Criteo AI Lab.

We were also proud to have a paper accepted at the “Disinformation Countermeasures and Machine Learning (DisCoML)” workshop. We were particularly proud of this work as it uses machine learning for an applied problem that is very close to our heart.

We had 2 papers at the “Principles of Distribution Shift workshop (PODS)”. “A bias-variance analysis of weight averaging for OOD generalization“, where we provide a new theoretical analysis of weight averaging which inspired our new model DiWA which achieved state-of-the-art results on the reference OOD generalization benchmark. Additionally, we had a paper Towards OOD Detection in Graph Classification from Uncertainty Estimation Perspective, where we discuss properties of OOD detection on graph-level classification tasks using uncertainty estimation approaches.

Finally, we got an accepted paper High Performance of Gradient Boosting in Binding Affinity Prediction at the CompBio workshop, where we showed the power of gradient boosting decision trees in graph-structured predictions.

Vianney Perchet was also an invited speaker at the “Complex Feedback in Online Learning” workshop, where he presented his paper Decentralized Learning in Online Queuing Systems (with F. Sentenac and E. Boursier)

Sponsorship Booth

In 2022 Criteo was a proud sponsor of ICML which gave us the opportunity to share just how much cutting edge AI can be integrated into real world production systems. This networking dimension and the quality of the exchanges between industry and academia are a real plus of ICML.

Highlights for our team

ICML is not only an opportunity to showcase our work, but also to learn and draw inspiration from the very vibrant global machine learning community. Some of the highlights from the 14 Criteos who attended are:

“DEI values are very strong at Criteo, and I found the idea of ‘Affinity events’ exciting: ICML 2022 Affinity Events.”

“It was a rich experience to meet different research communities, attend conferences and workshops, and have the opportunity to present our work “Nested bandits” to top researchers (well-known ones from the bandit and RL communities) from competitive universities and industrial groups. I think this shows the relevance of our research group and it attracted many talented researchers to see what solutions we propose in our products as well.”

“After two years of online conferences, it was nice to see how “ Stable Conformal Prediction Sets “received an outstanding paper award. The authors proposed a way to offer data privacy “for free” by resorting to a recent tool called dataset condensation. Conformal Prediction (CP) is a methodology that allows to estimate a confidence set for the outcome of a method given its input. The nice point about this paper is that it addresses an important problem: critical applications of ML (such as thus we deploy for Criteo’s application) need more than just a pointwise prediction. In practice, the work proposes a simple, efficient and theoretically grounded approach for solving it using stability properties of the algorithm at hand.

On a funny note, my preferred session was the one on Thursday morning named MISC/Deep Learning, in which most papers were about kernel methods. So, kernels are not dead, they have just been rebranded.”

“Working in generative and temporal modeling, ICML 2022 was a great time for me to learn and discuss this research direction; in particular, the following sessions were particularly interesting. Looking forward to the next in-person conference!

• The tutorial Sampling as First-Order Optimization over a space of probability measures by Anna Korba and Adil Salim is a clear introduction to gradient flows, which have been rising as a crucial tool to improve our understanding of neural network optimization and generative modeling.
• The main conference session DL: Generative Models and Autoencoders with many impressive papers, from audio generation via state-space models to controllable image manipulation via variational keypoint learning.
• The workshop Continuous Time Perspectives in Machine Learning, covering various aspects of deep learning with promising applications in optimization, generative modeling, gradient flows and sequence modeling.”

“I discovered many research topics (spurious correlations, bandits, privacy, etc.) and the world of research where some are seen as super-stars…. two papers that I appreciated:

• “Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations”: they proposed a two-step approach by mostly using contrastive learning to improve the robustness of the model to spurious correlations.
• “Dialog Inpainting: Turning Documents into Dialogs”: they proposed to create a dataset of dialogues — where there is often little data — by leveraging data from Wikipedia and generating the associated questions through a trained model. Especially because they were long talks but also thanks to the exchange, we had during the poster session”

“Overall a positive experience of attending my first physical conference with accepted papers !
Great experience of presenting our papers and discussing with participants (both industrial and academic), in particular in the field of OOD generalization for which Criteo is organising a challenge at ECML later this year. On this topic, I particularly enjoyed the Principles of Distribution Shift workshop (PODS) where I presented our latest work on weight averaging and enjoyed discovering new work in the main conference e.g. “Rich Feature Construction for the Optimization-Generalization Dilemma”, which presents a new method for better OOD generalization”

“Working on privacy-preserving machine learning, including federated learning, many exciting works and presentations caught my attention:

• The tutorial “Quantitative Reasoning About Data Privacy in Machine Learning “Gives a broad overview of why and how to audit data privacy in machine learning applications. In particular, data reconstruction and membership inference attacks have been covered along with two techniques based on Renyi-Differential Privacy and Fisher Information Loss to bound privacy leakage.
• The paper “Privacy for Free: How does Dataset Condensation Help Privacy? “Received an outstanding paper award. The authors proposed to use data condensation techniques to increase the training efficiency of synthetic dataset generation while offering data privacy for free. “

Conclusion

To close, we want to extend a BIG THANK YOU to the organizers. Everything was very well organized, the room easy to find, and the event setup was very smooth!

Overall, this event has been very stimulating. We are happy to return to our team with all those new ideas, new opportunities to try, and new research developments to explore further.

Thank you, ICML Team, for this great conference!

---

Do you want to join us next year? Check out our open positions!

Careers at Criteo | Criteo jobs

Find opportunities everywhere. ​Choose your next challenge.

careers.criteo.com