The Conference and Workshop on Neural Information Processing Systems (abbreviated as NeurIPS) is a machine learning and computational neuroscience conference held every December. The Thirty-seventh edition was held again in New Orleans, from Sunday, Dec 10, through Saturday, Dec 16.

Criteo had a significant presence with five (+1) NeurIPS papers from the AI Lab (+ FAIRPLAY team) and one workshop paper. Stay tuned as we delve deeper into our experience at NeurIPS 2023, highlighting the key takeaways, innovations, and knowledge-sharing moments that made this event a remarkable milestone in our journey toward shaping the future of technology.

Our Participation

Main conference

• Jean-Yves Franceschi (Criteo), Mike Gartrell (Criteo), Ludovic Dos Santos (Criteo), Thibaut Issenhuth (Criteo & Ecole des Ponts), Emmanuel de Bézenac (ETH Zürich), Mickaël Chen (Valeo.ai ), Alain Rakotomamonjy (Criteo). Unifying GANs and Score-Based Diffusion as Generative Particle Models. Preprint and code.
• Skander Karkar (Criteo & Sorbonne University), Ibrahim Ayed (Sorbonne University), Emmanuel de Bezenac (ETH Zurich), patrick Gallinari (Criteo & Sorbonne University), Module-wise Training of Neural Networks via the Minimizing Movement Scheme. Preprint.
• Louis Serrano (Sorbonne University), Lise Le Boudec (Sorbonne University), Armand Kassaï Koupaï (Sorbonne University), Yuan Yin (Sorbonne University), Thomas X Wang (Sorbonne University), Jean-Noël Vittaut (Sorbonne University), Patrick Gallinari (Criteo & Sorbonne University), Operator Learning with Neural Fields: Tackling PDEs on General Geometries. Preprint.
• Ziyad Benomar (ENSAE & FAIRPLAY) and Vianney Perchet (ENSAE, FAIRPLAY, Criteo). Advise querying under budget constraint for online algorithms. Preprint.
• Dorian Baudry (ENSAE & FAIRPLAY), Fabien Pesquerel (INRIA, Univ. Lille, Centrale Lille), Rémy Degenne (INRIA, Univ. Lille, Centrale Lille), Odalric-Ambrym Maillard (INRIA, Univ. Lille, Centrale Lille). Fast Asymptotically Optimal Algorithms for Non-Parametric Stochastic Bandits. Preprint.
• Mathieu Molina (INRIA, FAIRPLAY), Nicolas Gast (INRIA, Univ. Grenoble), Patrick Loiseau (INRIA, FAIRPLAY), and Vianney Perchet (ENSAE, FAIRPLAY, Criteo). Trading-off price for data quality to achieve fair online allocation. Preprint.

Workshop paper

• Imad Aouali. Linear diffusion models meet contextual bandits with large action spaces. NeurIPS 2023 Foundation Models for Decision Making Workshop. Preprint.

Invited Talk at Workshop

• Vianney Perchet. Active and Online Learning with Large (and Combinatorial) Models. NeurIPS 2023 I Can’t Believe It’s Not Better (ICBINB): Failure Modes in the Age of Foundation Models Workshop.

Criteos Takeaways

Imad’s highlights

As a first-year Ph.D. student, NeurIPS 2023 marked my fourth conference, and it quickly became the most awe-inspiring experience yet. I was amazed by the sheer size of the event. The conference venue was packed with thousands of AI researchers from all over the world. The conference’s large size reflected the rapid growth of the AI field, showing the global impact of AI.

Personally, it was a hotspot for swapping ideas and teaming up with other brilliant folks in the field. The coolest part? Finally, I am meeting face-to-face with friends I have collaborated with virtually for a long time but never had the chance to meet in person. And let me tell you, trying to guess someone’s height from virtual meetings? Hilariously tricky!

I also had the chance to present my work, Linear Diffusion Models Meet Contextual Bandits with Large Action Spaces, at the Foundation Models for Decision Making Workshop, and it was pretty cool. Apart from presenting my work, I enjoyed checking out the other presentations and posters, too!

During my first year of the Ph.D., I dove deep into offline reinforcement learning and bandits. However, heading to NeurIPS, my goal was to broaden my horizons, especially by delving into generative AI. So, my game plan was to soak up as much knowledge as possible, steering away from my usual turf. Luckily, the conference program was diverse and rich, with invited talks, oral presentations, poster sessions, tutorials, workshops, etc.

The presentations covered a wide range of topics, including reinforcement learning, theory, LLMs, and vision. To name a few, I liked the Do You Prefer Learning with Preferences? tutorial highlighting recent advancements in machine learning with preference-based feedback. I also enjoyed the Foundation Models for Decision Making workshop, delving into applying neural networks trained on broad data (foundation models) to enhance decision-making agents. The Diffusion Models workshop was equally impressive, tackling recent advances and promoting collaborations to push the boundaries of research in diffusion models.

Overall, NeurIPS 2023 was a blast for me. Getting to hang out with my friends and meet new like-minded people every day was awesome. Plus, the sheer size of NeurIPS 2023 opened up doors for me to connect with loads of researchers. It was a fantastic experience all around!

Jean-Yves’ notes

Like previous years, NeurIPS remains the most important scientific conference in our domain due to its sheer size and the diversity of tackled topics. It is the ideal event to observe the evolution of our field and be introduced to future areas of research. While research on creating, using, and testing generative foundation models for text, image, and video has expectedly exploded, it is interesting to see new lines of work emerge to make sure these models can be used in a responsible manner.

We had the chance to present a poster for our recent paper, Unifying GANs, and Score-Based Diffusion as Generative Particle Models, at the main conference event with Emmanuel de Bézenac, Mickaël Chen, ​Ludovic Dos Santos, and Mike Gartrell. We were thrilled to see and talk to so many people interested in our work, so much so that all coauthors did some overtime after the official session ended!

After some rest, we could fully enjoy the conference, with countless valuable presentations. Among them, I would like to highlight some particularly interesting events I could attend.

• The tutorial Governance and Accountability for Machine Learning: Existing Tools, Ongoing Efforts and Future Directions (replay) was a refreshing event. It consisted of a non-technical overview of AI regulation initiatives and how they are tackled by current research. Overall, it is an excellent introduction to the matter and accessible to a large audience.
• I found the paper The Tunnel Effect: Building Data Representations in Deep Neural Networks thought-provoking, as it identifies a surprising effect of depth when training neural networks through an empirical study.
• The Workshop on Diffusion Models featured impressively technical and original poster presentations on these popular generative models. It included CommonCanvas, showing that training on a restricted set of open-licensed images is sufficient to obtain a powerful model.
• The Machine Unlearning Competition (replay) was the opportunity to discover a nascent area of research to respect privacy regulations like the EU’s GDPR when training and deploying AI models, with a nice introduction and an interesting overview of solutions.

Personally, I was pleased that the good organization of the conference, despite its large scale, allowed me to meet colleagues, friends, and fellow researchers with whom I had exciting conversations.

While I am still wondering whether the current scientific organization is scalable given the explosion of our field, I had a wonderful time at NeurIPS 2023 and look forward to participating in future conferences. Hopefully, in Europe!

Ludovic’s key points

I agree with Jean-Yves about the conference organization and how great it was to present our work to NeurIPS. I would like to add some papers and tutorial I liked:

• The tutorial on latent diffusion models, a very informative tutorial that introduces researchers to Latent Diffusion Models (LDMs), emphasizing their flexibility, efficient trade-off between performance and compute demands.
• Deep structured state space models for are getting their ways to world models in Model Based Reinforcement Learning and their effectiveness with regard to RNNs and Transformers were showcased in the paper Facing Off World Model Backbones: RNNs, Transformers, and S4.
• As in CV and NLP, unsupervised pre-training is getting more and more attention from the RL community aiming at learning foundational agent models. In this context learning agent with a faraway goal can be challenging. To ease the process, Offline Goal-Conditioned RL with Latent States as Actions proposes to learn a hierarchical policy that focus sequentially on sub-goals.
• A more Criteo’s business related paper on using transformers to predict the next item a user will interact with: Recommender Systems with Generative Retrieval. Are Transformers getting their way to the AdTech industry?

---

In conclusion, NeurIPS 2023 was an incredible experience for our team, and we’re excited to share the knowledge and insights we gained from the conferences and networking opportunities with our colleagues. We look forward to attending future editions and hope to continue supporting and contributing to the technology industry’s growth and success.

Engineering

We are creators! From designing ground-breaking products to finding unique ways to solve technical challenges at an…

careers.criteo.com