DiffusionGemma: 4x faster text generation

DiffusionGemma is an experimental open model using text diffusion for faster text generation instead of autoregressive decoding.

• •Generates up to 4x faster on GPUs by processing 256 tokens in parallel with each forward pass
• •26B MoE model activating only 3.8B parameters, fitting within 18GB VRAM on consumer GPUs when quantized
• •Bidirectional attention enables advantages for non-linear tasks like code infilling and mathematical graphs
• •Iteratively refines output with self-correction by evaluating entire text blocks at once
• •Optimized for local inference; offers diminishing returns in high-QPS cloud serving

This summary was automatically generated by AI based on the original article and may not be fully accurate.