Tech • AI • Robotics
Snap’s GPU-Accelerated Secret to Processing 10 Petabytes a Day | NVIDIA AI Podcast Ep. 298
7/10TL;DR
Snap cut data pipeline costs by 76% and significantly reduced compute and memory usage by shifting large-scale processing to GPU-accelerated infrastructure.
KEY POINTS
Massive data scale and strict deadlines
Snap operates at enormous scale, with its experimentation platform alone processing more than 10 petabytes of data daily. Results must be ready each morning to inform product decisions across engineering, data science, and product teams. This requires both high performance and predictable delivery under tight service-level agreements.
Experimentation as a core product driver
The company relies heavily on A/B testing and advanced statistical methods to guide feature development. Techniques such as heterogeneous treatment effect detection, variance reduction, and sample size mismatch correction help ensure that new features deliver value across diverse global user segments without degrading user experience.
Shift from CPUs to GPUs
To handle growing complexity without escalating costs, Snap transitioned from CPU-heavy processing to GPU-accelerated workloads using NVIDIA Spark RAPIDS. The approach enables faster execution and more efficient scaling, particularly for data-intensive operations like joins and repartitioning.
Performance gains across workloads
Benchmarking showed substantial improvements: over 3× speedups for join-heavy jobs, around 2× for union operations, and roughly 1.5× for aggregations. These gains stem from GPUs’ parallel processing capabilities and high-bandwidth memory.
Zero code migration advantage
A key factor in adoption was the ability to deploy GPU acceleration with no changes to existing codebases. This minimized engineering overhead and accelerated rollout, aligning with internal priorities around developer productivity.
Innovative use of idle GPU capacity
Snap identified that large amounts of GPU capacity used for real-time inference sat idle during off-peak hours, particularly between 1 a.m. and 5 a.m. Pacific Time. The company repurposed this unused capacity for batch data processing, significantly improving resource utilization.
New platform built on Kubernetes
To enable this shift, Snap developed a new data platform on Google Kubernetes Engine (GKE). This allowed batch workloads to run on infrastructure originally designed for online services, while introducing mechanisms like preemption to prioritize user-facing workloads when needed.
Resilient multi-layer fallback system
The system includes dynamic fallback paths: from GPUs to CPUs, and from Kubernetes-based execution to traditional Dataproc clusters when necessary. This ensures reliability even during peak usage or resource shortages.
Dramatic efficiency improvements
The migration delivered major operational gains, including a 76% reduction in job costs, 62% fewer CPU cores required, and an 80% drop in memory usage. It also eliminated roughly 120 terabytes of disk and memory spill, a common bottleneck in large-scale data pipelines.
Rapid production rollout
Despite the scale, Snap moved from experimentation to full production in approximately 8 to 9 months. The effort relied on close collaboration across infrastructure and technology partners, as well as extensive benchmarking and staged deployment.
Broader impact on internal AI and data strategy
The new platform now enables multiple teams to leverage shared GPU resources for a variety of workloads. This has influenced Snap’s broader roadmap, encouraging more widespread adoption of GPU acceleration and more flexible scheduling of compute-intensive tasks.
CONCLUSION
Snap’s transition to GPU-accelerated data processing demonstrates how large-scale platforms can dramatically cut costs and improve performance by rethinking infrastructure and maximizing existing resources.
Explain this