Dynamically Splitting Wide Partitions in Cassandra for Time Series Workloads

This post describes Netflix's approach to optimizing Apache Cassandra for large-scale TimeSeries workloads by dynamically adjusting partition strategies to address wide partition performance issues.

• •Wide partitions in Cassandra cause high read tail latencies in seconds and increased thread queueing, contrasting with typical single-digit millisecond reads
• •The TimeSeries Abstraction Layer divides data into discrete time slices and buckets to manage partition sizes without requiring tombstones for data removal
• •Initial provisioning using Monte Carlo simulations may become suboptimal as workloads evolve or when specific TimeSeries IDs exhibit outlier behavior with higher event volumes
• •A background DynamicTimeSliceConfigWorker monitors partition size histograms and automatically adjusts time bucket intervals to maintain optimal density between 2-10 MiB per partition