The 26th International Conference on Artificial Intelligence and Statistics (AISTATS) was held on April 25 — April 27 at Palau de Congressos in Valencia (Spain). AISTATS is an interdisciplinary gathering of researchers at the intersection of computer science, artificial intelligence, machine learning, statistics, and related areas. At Criteo, we like how AISTATS has broadened research in these fields by promoting the exchange of ideas among them. That is why we were happy and proud to be a platinum sponsor of this conference and Criteo AI Lab sent three researchers with papers accepted (authors or co-authors).
Our team had the opportunity to meet experts on diverse topics, share knowledge, and learn from other researchers. Throughout the three-day event, invited talks, orals presentations, posters, mentoring, and affinity groups were organized on topics ranging from bandits, deep learning, game theory, and much more.
The venue, with a main auditorium and a big hall, was perfect for hosting all the attendance and facilitating the interactions and conversations. We enjoyed our time in the booth talking about Criteo AI Lab, connecting attendances to our research team and challenging people with our quizzes. We were happy to find so many people interested in our Criteos’ work and ran out of goodies. 🙌
Takeaways from Criteos
In what follows, our Criteos share their main takeaways from the conference.
Morgane Goibert’s highlights
I attended AISTATS 2023 not only to see the articles that were published at the conference but also to present my own, which has been selected to be a Notable paper: Origins of Low-Dimensional Adversarial Perturbations with Elvis Dohmatob and Chuan Guo.
My main interest is, as can be deduced from the paper’s title, Adversarial Robustness. On this topic, I was pleased to discover several very interesting works, among which I have particularly liked Nash Equilibria and Pitfalls of Adversarial Training in Adversarial Robustness Games from Balcan et al. The paper studies the classical robustification procedure known as adversarial training through the lens of game theory and shows its limits, namely that contrary to Nash equilibrium, it does not converge even in simple scenarios. I also enjoyed other papers on this topic, like Improving Adversarial Robustness via Joint Classification and Multiple Explicit Detection Classes by Baharlouei et al., Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks by Zhang et al or Adversarial Robustness of VAEs through the lens of local geometry by Khan and Storkey.
Something I also really enjoy at conferences like AISTATS is the possibility to stumble across very interesting papers that do not lie exactly in your own research topic, mainly because AISTATS is a conference of a perfect size: not too small to get to know different papers, but not too large not to be lost! As serendipity is very important for researchers, this is important for me to highlight. Among the papers that are not related to robustness that I liked, I can list the following:
- Using Sliced Mutual Information to Study Memorization and Generalization in Deep Neural Networks by Wongso et al.: they use this Sliced Mutual Information tool to study neural networks and show how useful it is to understand better how networks memorize and/or generalize in different contexts.
- Who Should Predict? Exact Algorithms For Learning to Defer to Humans by Mozannar et al: they provide conditions and an algorithm to train neural networks not only to solve a task but also to understand when it’s better to let a human do the job.
- Convolutional Persistence as a Remedy to Neural Model Analysis by Khramtsova et al.: the paper uses a topological tool to understand better the behavior of neural networks, which can help select the best model for the task and better monitor the model’s behavior during training.
Apart from the nice papers mentioned above, AISTATS is a great conference to attend. Its size makes it easy to connect with different researchers and to follow the talks and presentations. Plus the organizers managed to infuse a very warm and welcoming atmosphere to the event!
Song Duong’s favorite papers
Attending AISTATS was truly a one-of-a-kind experience for me especially since this is my first accepted paper Learning from Multiple Sources for Data-to-text and Text-to-data at a conference. The poster session is an excellent platform for exchanging ideas, receiving feedback, and networking with other researchers. The size of the conference made it easy not to lose track and helped to connect with others. Compared to other conferences like ICML or NeurIPS where there are multiple tracks happening in parallel, AISTATS is single-track which really fosters an environment where we are exposed to all kinds of research and allows fruitful cross-team discussions. Even though NLP was not the primary topic at the conference, people were really interested in learning about things that are out of their domain of expertise.
As my work is on large language models (LLMs), I could not help but take a glimpse at TabLLM: Few-shot Classification of Tabular Data with Large Language Models especially that NLP papers are not common at AISTATS (actually this paper and mine are the only paper on language models). This article explores the use of large language models (LLM) for zero-shot and few-shot classification of tabular data by prompting the model with a serialization of the data and a short description of the classification problem. Zero and few shot means that we do not train these models but use them as they are to answer questions about the serialized data. Leveraging LLMs to solve tasks that involve structured data such as tables are problems that appeal to me the most and getting to discuss with researchers working on that was enriching.
Another paper that piqued my interest was Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity. This paper addresses the problem of Byzantine faults in distributed ML systems. Byzantine faults refer to potential malicious behaviors of some agents in a distributed system that for instance send incorrect or manipulated data to the others. The authors propose a theoretically grounded recipe that enables learning even in such challenging environments. This could be a real scenario for Criteo since we are working with several businesses that may or may not compete with each other.
Overall, attending AISTATS was a fantastic experience for me. If I was to pick one memorable moment, It would be seeing the Criteo booth every morning and all these people interested in what we are doing.
Imad Aouali’s takeaways
As a Ph.D. student in theoretical machine learning with practical interest, attending AISTATS 2023 in Valencia, Spain was a valuable experience. The conference provided an excellent opportunity for exchanging ideas, discussing, and collaborating with other researchers and practitioners in the field. I also had the chance to present my work, Mixed-Effect Thompson Sampling with Branislav Kveton (AWS AI Lab) and Sumeet Katariya (Amazon).
The conference program was diverse and rich, with keynote and oral presentations, as well as poster sessions. The presentations covered a wide range of topics, including optimal transport, trustworthy ML, and supervised learning, among others. I found it particularly interesting that some works on bandits proposed ways to efficiently learn to act online by leveraging some latent structure of the problem, which is important for scalability. These include Optimal Algorithms for Latent Bandits with Cluster Structure and Multi-task Representation Learning with Stochastic Linear Bandits. Another paper that caught my attention was Direct Inference of Effect of Treatment (DIET) for a Cookieless World as it is related to Criteo’s business and goal of supporting the open internet and making it better for marketers and consumers alike. Roughly speaking, this work considers treatment effect estimation without relying on cookies (or when the users are using multiple devices).
In general, I had a fantastic experience attending AISTATS 2023. Meeting my colleagues and friends every day and the interest shown by attendees in our poster session meant a lot to me. Additionally, AISTATS 2023 format made it very natural to connect with various researchers and keep up with the talks and presentations.
In conclusion, AISTATS 2023 was an incredible experience for our team, and we’re grateful for the opportunity to have sponsored this event. Congrats to everyone who got a paper accepted and thank you for sharing your research with us. ❤️
