Inside the eighth-generation TPU: An architecture deep dive

Google introduced eighth-generation TPUs (TPU 8t and TPU 8i) optimized for modern AI workloads with improved efficiency and scalability for training and serving.

• •TPU 8t features SparseCore for embedding lookups, native FP4 quantization, and Virgo Network with 4x increased datacenter network bandwidth for large-scale pre-training
• •TPU 8t includes TPUDirect RDMA and Storage enabling direct memory transfers, bypassing host CPU and reducing latency by 10x
• •TPU 8t scales to over 1 million chips per cluster with 1.6 million ExaFlops performance via JAX and Pathways
• •TPU 8i has 3x larger on-chip SRAM and Collectives Acceleration Engine (CAE) reducing collective operation latency by 5x for reasoning workloads
• •TPU 8i uses Boardfly topology connecting up to 1,152 chips with 50% lower all-to-all communication latency versus 3D torus