data Articles

281 min read

Data Engineering•2026-02-17

Drastically Reducing Out-of-Memory Errors in Apache Spark at Pinterest

This post describes Pinterest's Auto Memory Retries feature for Apache Spark, which automatically retries OOM-failed tasks on larger executors to reduce failures and resource waste.

engineering

data

apache-spark

open-source

data Articles

Related Tags

Viaduct 1.0 and the future of Airbnb’s data mesh

Privacy-first connections: Empowering social experiences at Airbnb

Scaling Global Storytelling: Modernizing Localization Analytics at Netflix

Drastically Reducing Out-of-Memory Errors in Apache Spark at Pinterest

Viaduct, Five Years On: Modernizing the Data-Oriented Service Mesh