Building a Sustainable Future through Real-Time Data

Main takeaways from the Aerospike Summit Paris 2024

On June 6th, 2024, Criteo Headquarters hosted the Aerospike Summit Paris, an event that brought together database innovators, technologists, developers, engineers, and architects. The summit focused on real-time data processing and sustainability, highlighting how cutting-edge technologies like Aerospike can contribute to a greener and more sustainable future.

This year’s theme, “Building a Sustainable Future through Real-Time Data,” aligned perfectly with Criteo’s commitment to innovation and environmental responsibility. We have been using Aerospike for some years now and can tell how we have been able to use fewer technologies, reduce server footprint, and lower their total cost of ownership.

To learn more about our plan for our next-gen caching infrastructure, check this video from the Aerospike Summit 2022 👇

Keynote Sessions: Innovations and Sustainability

Moving Forward: Aerospike’s Journey through Innovations, Sustainability, and Roadmap

Subbu Iyer, CEO of Aerospike, shared the company’s latest innovations, emphasizing their commitment to sustainability. Subbu highlighted how Aerospike’s core design principles reduce infrastructure needs by up to 80%, leading to significant CO2 reductions. The session covered advancements in multi-model capabilities, support for property graphs, multi-record transactions, and in-memory technologies, all contributing to more efficient and sustainable data systems.

Martin James, VP of EMEA at Aerospike (left) & Subbu Iver, CEO of Aerospike (right)

Main takeaways

Aerospike originated as a NoSQL Key-Value database and has since expanded its support to include documents, graphs, and, most recently, vectors.
In addition to their real-time data services, the core strength lies in the high-performance storage engine, featuring the “Hybrid Memory Architecture” (HMA). With HMA, data is stored on SSD while indexes are kept in DRAM, offering two key advantages: achieving near-memory performance and significant cost savings.
Unlike conventional solutions, Aerospike stands out by helping companies reduce CO2 emissions through server consolidation.
In the realm of AI applications, data plays a pivotal role and is transitioning towards real-time processing. This shift prompts companies to reconsider their existing infrastructure or explore new alternatives.
Aerospike’s primary focus areas revolve around enhancing Cloud capabilities, driving AI innovations with Graph and Vector databases, and streamlining application development on the Aerospike platform.
Aerospike Graph provides a scalable, production-grade transactional database capable of supporting trillions of vertices and edges with minimal query latency.
Aerospike Vector enables the storage of billions of vectors while ensuring high scalability for both data ingestion and query processing.
By adopting a Multi-Model approach encompassing Key-Value, Graph, and Vector data models, users can seamlessly develop real-time data applications using a single storage solution.

Powering Real-Time Decisions With A Reduced Footprint

Maxime Brugidou, Criteo’s R&D Platform Director, delivered a compelling presentation on our journey towards a greener tech industry. He detailed how Criteo has leveraged Aerospike’s technologies to reduce our server footprint without compromising business value. Maxime provided an in-depth look into our architecture, choice of hardware, and the automation and maintenance improvements that enhance the efficiency and sustainability of Criteo’s AdTech platform.

Maxime, our R&D Platform Director, on the stage!

Main takeaways

AdTech relies heavily on real-time responses (less than 100ms) and highly scalable systems to thrive.
The volume of traffic we handle is substantial: 18 million external queries per second, 120,000 ads displayed per second, and a daily peak of 290 million queries per second on key-value storage.
Our operations are supported by our in-house hardware, a strategic choice that has proven to be a significant asset and a competitive edge (spanning 7 data centers across 3 continents).
However, the environmental impact of our hardware is a top concern for us. Over the past few years, we have diligently monitored and successfully reduced electricity consumption.
What about our CO2 emissions? Thanks to several initiatives, our CO2 emissions have dropped to zero over the past two years. Achieving this milestone required extensive collaboration with vendors to procure more efficient hardware. Additionally, efforts were made to optimize data center locations and sizes. For detailed information, refer to the latest Criteo’s Corporate Social Responsibility Report.
How does this tie in with Aerospike? Our strategic roadmap includes a three-year plan to transition to Aerospike as the primary real-time datastore for Criteo.
In 2020, our technology stack featured key-value store solutions like Memcached and Couchbase. However, we encountered challenges with real-time data processing due to large datasets, global scale, high traffic volumes, and low-latency requirements.
The decision to adopt Aerospike was driven by the exceptional performance achieved through the combination of Aerospike and NVMe hardware, delivering impressive sub-millisecond access times. Benchmark results exceeded our expectations.
Following the implementation in production, Aerospike seamlessly integrated with minimal tuning, resulting in zero incidents over the past 18 months. Reliability and operational efficiency are pivotal factors in our endorsement of Aerospike’s solution.
Notable improvements include a 40% reduction in data duplication (equivalent to around 200TB) and a significant decrease in the number of servers by 75% (from 3,200 to 800 servers).
While progress has been made, our commitment extends to achieving SBTI targets by 2030. Explore our sustainability portal to delve deeper into our sustainability initiatives 👇 💚

Sustainability | Criteo

Criteo strives for continuous improvement and positive impact in terms of Diversity, Equity, and Inclusion (DEI)…

www.criteo.com

Technical Deep Dives and Case Studies

How to Reach 1 Trillion Transactions on the BSV Blockchain in a Week

Martin Coxall from BSV Blockchain captivated the audience with a case study on achieving 1 trillion transactions within a week using their blockchain technology. This session highlighted the potential of blockchain in managing vast amounts of data efficiently, aligning with the summit’s sustainability theme by showcasing high-efficiency data handling.

Martin Coxall highlights the benefits of BSV Blockchain

Main takeaways

The BSV Association supports the BSV Blockchain as the original Bitcoin. Reflecting its mission to fulfill the “Satoshi Vision,” BSV was created to restore the original Satoshi protocol, keep it stable, and enable it to scale massively.
Trust is crucial in the realm of blockchain technology. Despite its potential, blockchain has yet to fully win over the trust of the masses. The current iteration of the internet, known as Web2, is largely controlled by corporations that offer services in return for personal data.
So, what sets Web3 apart? Web3 introduces decentralized applications that operate on the blockchain, enabling users to engage without compromising their privacy or data ownership. This shift paves the way for the advancement of blockchain technologies.
A common misconception revolves around the roles of Crypto, Blockchain, and Bitcoin. While Crypto is often viewed as the focal point, the true essence lies in Blockchain, which serves as the primary value proposition.
The advantages of Blockchain are manifold: it facilitates the management of verifiable digital identities, ensures secure transactions, and upholds data integrity.
Trust forms the bedrock of any successful business, necessitating scalability, efficiency, low transaction costs, resilience, stability, and sustainability. Blockchain emerges as a viable solution to address these fundamental challenges.
In safeguarding data integrity, BSV Blockchain introduces a “timestamping” mechanism that establishes a universal source of truth. Each block in the chain references the previous block’s header, culminating in an immutable chain.
The Blockchain Trilemma encapsulates the trade-off among three pivotal elements of blockchain technology: Security, Scalability, and Decentralization.
How does BSV Blockchain tackle this challenge? Through Teranode, a groundbreaking initiative that reimagines the BSV node software in alignment with Satoshi’s White Paper vision.
With a maximum block size of 100GB per Teranode block, the ability to transmit more data within a transaction enhances cost efficiency and reduces C02 emissions per transaction.
Notably, Aerospike’s solution is an integral component of the Teranode’s architecture.
Demo 👇 📊

Teranode

Teranode explorer

teranode.bsvblockchain.org

Panel Discussion: Application Modernization

Moderated by Pierre Berard, Strategic Account Manager at Aerospike, this panel featured industry leaders, including Gilles Monnin, Peter Goron, Thomas Giacomo, and Gilles Philippart. The discussion revolved around application modernization, with a particular focus on how modern data management practices impact customer journeys and drive efficiency, which is essential for achieving sustainability goals.

From left to right: Thomas Giacomo (BSV Blockchain), Peter Goron (Criteo), Gilles Philippart (Confluent) & Gilles Monnin (ITS Services)

Vector Databases: Using Vector Search with Graph & Key-Value Queries

Srini V. Srinivasan, CTO and Founder of Aerospike, and Behrad Babaee, Principal Solutions Architect at Aerospike, presented on the innovative use of vector databases. They demonstrated how embedding vectors in a database with graph and key-value data can enhance search capabilities and solve complex application problems, all while maintaining high efficiency and sustainability standards. This session emphasized the importance of efficient data replication and disaster recovery in supporting sustainable business practices.

Srini V. Srinivasan, CTO and Founder of Aerospike (left) & Behrad Babaee, Principal Solutions Architect at Aerospike (right)

Main takeaways

In order to stay relevant in the industry, it is essential to prioritize innovation, iteration, and adaptation. Aerospike has consistently transformed its platform to deliver cutting-edge solutions that go beyond its initial focus on NoSQL Key-value storage.
The significance of Vectors lies in their ability to enable comprehensive domain searches by capturing the semantic essence of source data through embedding models. These models are specialized algorithms trained to condense information into compact representations within a multi-dimensional space.
A vector database serves two primary functions: storage and search. Its core capability lies in generating embeddings from the domain and executing proximity searches for swift data retrieval.
HNSW, which stands for Hierarchical Navigable Small Worlds, is a graph-based technique for approximate nearest neighbor searches commonly utilized in various vector databases. Aerospike’s implementation encompasses all functionalities of the Aerospike Database, including the storage layer, along with a novel query layer known as Aerospike Vector Search.
One of the primary scalability challenges faced by HNSW is in terms of Writes. Aerospike excels in handling high-throughput write operations from applications while ensuring real-time access to the most up-to-date data.
By leveraging a multi-model database, you can enhance your applications with key-value access, vector search capabilities, and graph search functionalities.

Networking and Reception

The summit concluded with a reception featuring drinks and finger food, providing attendees an opportunity to network and discuss the day’s insights in a relaxed and informal setting. This networking session was invaluable for professionals to connect, share ideas, and explore potential collaborations, all with a focus on sustainable and innovative data practices.

The Aerospike Summit Paris 2024, hosted at Criteo, was a resounding success. It provided valuable knowledge and actionable insights into the future of real-time data processing and sustainability. With a lineup of esteemed speakers and thought-provoking sessions, the event underscored the importance of innovation and efficiency in driving a sustainable future through advanced data technologies. Attendees left with a wealth of information and inspiration to implement in their own projects and organizations.

At Criteo, we are proud to have played a central role in this important event, reinforcing our commitment to sustainable innovation.