NVIDIA and Hugging Face have announced a collaboration that brings production-grade distributed diffusion training to any Diffusers-format model on the Hugging Face Hub, eliminating the need for checkpoint conversion or model rewrites. The integration uses the NVIDIA NeMo Automodel open-source library, a PyTorch-based training framework that lets developers fine-tune large diffusion models, whether they're working with a single graphics processing unit (GPU) or hundreds of them. The work is fully open source under the Apache 2.0 license and is documented in the Diffusers training guide.

What Problem Does This Partnership Solve?

Training large AI models that generate images and videos has traditionally been a complex, time-consuming process. Developers had to convert model checkpoints into proprietary formats, rewrite code for different hardware setups, and manage separate training pipelines that didn't integrate cleanly with existing tools. This collaboration removes those friction points. Pretrained weights from the Hugging Face Hub now work out of the box, and fine-tuned checkpoints load directly into a DiffusionPipeline for inference or back to the Hub for sharing. This means downstream tools like quantization, compilation, LoRA adapters, and custom samplers all keep working without extra steps.

Which AI Models Can You Fine-Tune Now?

The NeMo Automodel integration ships with ready-to-use fine-tuning recipes for several open diffusion models. These include text-to-image and text-to-video models of varying sizes, each optimized for different hardware constraints and use cases:

• FLUX.1-dev: A text-to-image model with 12 billion parameters that fits on standard enterprise GPUs and includes a LoRA recipe for parameter-efficient fine-tuning.
• FLUX.2-dev: A larger text-to-image model with 32 billion parameters, also with a LoRA recipe for efficient adaptation.
• Wan 2.1: Text-to-video models available in 1.3 billion and 14 billion parameter versions, with the smaller version fitting on a single 40GB A100 GPU.
• Wan 2.2: A 27 billion parameter text-to-video model using a mixture of experts architecture, where only 14 billion parameters are active per step.
• HunyuanVideo 1.5: A 13 billion parameter text-to-video model with a LoRA recipe for efficient fine-tuning.
• Qwen-Image: A 20 billion parameter text-to-image model using multimodal diffusion transformers.

How to Fine-Tune a Diffusion Model Using NeMo Automodel

The typical workflow for fine-tuning any of the supported models involves several straightforward steps that developers can follow to adapt these models for custom tasks:

• Pre-encode the dataset: Stream images directly from Hugging Face and distribute preprocessing across all visible GPUs. The diffusion recipe consumes cached VAE latents and text embeddings instead of encoding source images during every training step, which significantly accelerates throughput.
• Launch training with existing YAML configuration: Use pre-built YAML files that already select the model, fine-tuning approach, batch size, and parallelism strategy. Supply dataset-specific paths and settings as command-line overrides without modifying the core configuration.
• Generate from the fine-tuned checkpoint: Use the existing generation YAML and point the model checkpoint to your completed training checkpoint. The generation command can include trigger tokens to invoke learned styles or effects.
• Monitor performance metrics: Track training efficiency and generation quality across your chosen hardware setup, whether that is a single node or a distributed cluster.

What Makes This Collaboration Different?

The key innovation is that NeMo Automodel is built around two design principles that matter for the Diffusers ecosystem. First, it is Hugging Face native, meaning users can point the pretrained_model_name_or_path parameter at any Diffusers model ID on the Hub and start training immediately. NeMo Automodel uses Diffusers model classes and pipelines for loading and generation, and checkpoints round-trip cleanly back into the Diffusers ecosystem.

Second, it offers one program that works at any scale. The recipes and training scripts can be easily modified to suit training at any scale, from a single GPU to hundreds. Parallelism is a configuration choice rather than a code rewrite. Users can switch between FSDP2, tensor parallel, expert parallel, context parallel, and pipeline parallel by declaring configurations instead of rewriting models. This flexibility means the same training recipe can scale from a researcher's laptop to a large enterprise cluster without fundamental changes to the code.

The integration also supports both full fine-tuning and parameter-efficient fine-tuning. Users can choose between maximum quality with full fine-tuning on a large cluster or maximum efficiency with LoRA-style parameter-efficient fine-tuning (PEFT) on a single node. The same recipe structure handles both approaches, giving developers flexibility based on their computational budget and quality requirements.

Why Should Developers Care About This?

For teams building custom image and video generation applications, this collaboration dramatically reduces the barrier to entry. There is no checkpoint conversion, no separate training format to convert to and then convert back, and no need to rewrite models when switching between different hardware setups. When a new diffusion model lands in Diffusers, enabling it in NeMo Automodel takes a small, contained code addition that includes a data preprocessing handler and a model adapter rather than a full custom training script.

The practical gains also extend to scalability. NeMo Automodel adds sharding schemes such as FSDP2, tensor, context, and pipeline parallelisms, as well as multi-node orchestration with SLURM today and Kubernetes coming. It also includes multiresolution bucketing, which accelerates training throughput by intelligently grouping images of similar aspect ratios. These capabilities make training larger models like FLUX.1-dev at 12 billion parameters and HunyuanVideo at 13 billion parameters possible on enterprise hardware.

The collaboration is fully open source under the Apache 2.0 license, meaning developers can inspect the code, contribute improvements, and deploy it without licensing restrictions. Installation options include a Docker container at nvcr.io/nvidia/nemo-automodel:26.06, pip installation, or building from source on GitHub. This openness aligns with the broader trend of making advanced AI training tools more accessible to the developer community.