Receive daily AI-curated summaries of engineering articles from top tech companies worldwide.
Endigest AI Core Summary
This guide demonstrates parameter-efficient fine-tuning of NVIDIA Cosmos Predict 2.5, a large-scale world model for robot video generation, using LoRA and DoRA techniques with diffusers.
•LoRA and DoRA inject trainable adapter modules into the frozen DiT model, reducing memory and enabling single-GPU training with portable adapter files
•Training uses 92 robot manipulation videos with text prompts describing pick-and-place tasks, evaluated on 50 (prompt, image) pairs
•The model employs rectified flow training: predicting velocity vectors that transport noise toward clean data, conditioned on initial frames and text prompts
•Training uses AdamW optimizer with linear warmup/decay scheduling, yielding ~50M trainable parameters at LoRA rank=32
•
The fine-tuned model generates synthetic robot trajectories for downstream learning, completing in 17 hours on single H100 or 2.5 hours on 8 H100 GPUs
This summary was automatically generated by AI based on the original article and may not be fully accurate.
Hugging Face
Google Cloud