Accelerating data lakes: Optimizing Apache Iceberg and Spark with gcs-analytics-core

Google Cloud announced gcs-analytics-core, an open-source Java library that optimizes Apache Iceberg and Spark performance on GCS by centralizing analytics optimizations across multiple engines.

• •Vectored I/O and Smart Parquet prefetching reduce I/O latency and network calls for columnar data reads
• •Integrates natively into Apache Iceberg 1.11.0+ as GCSFileIO implementation without requiring custom tuning
• •TPC-DS benchmarks show significant improvements: 71.51% scan time reduction on 1GB datasets, declining to 18.40% on 10TB datasets
• •Maintains compatibility with all Iceberg catalogs including REST catalog and Hive metadata systems
• •Requires enabling optimization flags and Iceberg 1.11.0+ with vectorization enabled for maximum performance