Optimizing Recommendation Systems with JDK’s Vector API

This post details Netflix's multi-step CPU optimization journey for video serendipity scoring in the Ranker recommendation service using JDK's Vector API.

• •Serendipity scoring consumed 7.5% of total CPU per node, using O(M×N) separate cosine similarity dot products in a nested loop with poor cache locality
• •Step 1 reformulated the work as matrix multiplication (A × B^T), turning M×N dot products into a single batched operation
• •Initial batching caused a 5% performance regression due to non-contiguous double[][] memory layout and increased GC pressure
• •Step 3 resolved this by switching to flat double[] row-major buffers with ThreadLocal reuse, eliminating per-request allocations
• •BLAS was evaluated but rejected due to F2J fallback, JNI transition overhead, and column-major layout mismatch with Java