Optimization Matters More Than Estimation
This Research Card shows that in large-scale recommendation systems, like those used at Criteo, focusing on how we optimize algorithms is more impactful than trying to perfectly estimate outcomes. By using simpler, more stable training objectives, we can achieve better click-through rates, faster experimentation, and more scalable performance across millions of ads or products.
- Title: Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation
- Authors: Imad Aouali, Otmane Sakhi
- Team: R&D — CAIL — AI For Performance — Performance Science
- Revue: Recsys ’25, CONSEQUENCES
- Status: Accepted
- Category: Causality
Why did we work on this topic?
Criteo’s systems must decide which ad or product to show to each user from a large catalog of choices. We’d like to improve these recommendations using past interaction data (clicks, views, purchases) without running costly A/B tests. Most existing methods try to build more accurate estimators of how well a new strategy might perform. But as the number of possible ads grows, these methods become unstable and difficult to optimize. We wanted to understand why learning breaks down at scale and how to make it more reliable when dealing with hundreds of thousands or millions of ads.
What did we achieve?
We found that the real challenge is not estimating how many clicks or purchases a strategy would get, but optimizing the algorithm so it can actually find that strategy. Our main results show that:
- Traditional methods get stuck or perform inconsistently when the number of choices is large.
- Simpler objectives are easy to train, stable, and lead to higher click-through-rates: they outperform all methods on real-world datasets like MovieLens (60K items), Twitch (200K), and GoodReads (1M).
How did we proceed?
- Analyzed existing methods and showed that their optimization becomes unstable as the number of ads increases.
- Proposed simpler objectives that are mathematically smoother and easier for optimization algorithms to handle.
- Tested both types of methods on large-scale recommendation datasets to compare click-through-rate performance.
- Validated robustness — proposed models worked well across learning rates, batch sizes, and architectures, unlike traditional methods.
What is the originality here?
- New perspective: Instead of improving estimation accuracy (most common approach), we focused on optimization stability, which matters more in large-scale advertising.
- Practical impact: Our methods train faster, require less parameter tuning, and achieve higher CTR or purchase rates.
- Scalable design: The approach works even when dealing with millions of products or ads.
- Simple yet powerful: Despite being conceptually simpler, our method consistently beats complex alternatives.
- Strategic insight: This work shifts the way we think about offline learning: prioritizing trainability over estimation.
Check all our research cards 👇
https://medium.com/criteo-engineering/research-cards/home
