DiffSynth-Studio

切换到中文版

Introduction

Welcome to the magical world of Diffusion models! DiffSynth-Studio is an open-source Diffusion model engine developed and maintained by the ModelScope Community team. We hope to foster technological innovation through framework construction, aggregate the power of the open-source community, and explore the rich capabilities of generative model technology!

Framework features:

• Model Support: The framework integrates mainstream open-source Diffusion models, covering image generation, video generation, audio generation, and image quality metrics models.
• VRAM Management: Dynamically schedules model parameters across disk, memory, and VRAM, allowing consumer-grade GPUs with low VRAM to run inference with large models.
• Parameter Quantization: Converts model parameters to quantized precisions such as NF4 and INT8, significantly reducing the VRAM requirements of model inference and LoRA training.
• Arbitrary Training: Almost every model that supports inference also supports training, whether base models, LoRAs, or any Adapter models with additional inputs.
• Split Training: Uses a computational graph inference engine to track every variable in the Pipeline, splitting the training process into two stages for efficient training.

References:

• Developer documentation (for humans): 中文版、English version
• Agent Skills (for AI): DiffSynth-Studio Model Integration Skills

See also:

• DiffSynth-ComfyUI: A ComfyUI node package built on DiffSynth-Studio, enabling you to build workflows in ComfyUI to run model inference.
• DiffSynth-WebUI: A lightweight LoRA training tool built on DiffSynth-Studio, enabling LoRA training of models on consumer-grade GPUs.
• ModelScope AIGC Zone (for Chinese users): Productized features powered by DiffSynth-Studio as the core inference and training engine; experience the rich potential of the open-source model ecosystem.
• ModelScope Civision (for global users): Unlock the vast potential of the open-source model ecosystem through productized capabilities powered by DiffSynth-Studio.

Update History

DiffSynth-Studio has undergone major version updates, and some old features are no longer maintained. If you need to use old features, please switch to the last historical version before the major version update.

Currently, the development personnel of this project are limited, with most of the work handled by Artiprocher and mi804. Therefore, the progress of new feature development will be relatively slow, and the speed of responding to and resolving issues is limited. We apologize for this and ask developers to understand.

• October 12, 2026 We now support distillation accelerated training based on OPD (On-Policy Distillation), which enables distillation acceleration of models at an extremely low cost. For details, please refer to Distillation Accelerated Training.

• September 29, 2026 We are excited to open-source EntroPack, a general-purpose tensor compression library for PyTorch that provides lossless compression with exact recovery and rate-controlled lossy compression, with GPU encoding and decoding. It is integrated into DiffSynth-Studio as a quantization backend, compressing model weights at arbitrary target bitrates in both inference and LoRA training. For more information, please refer to:

  • Project Page: https://github.com/modelscope/entropack
  • Technical Report: https://arxiv.org/abs/2609.34185
  • Pre-quantized Models: https://www.modelscope.cn/models/DiffSynth-Studio/EntroPackPreQuants
  • Example Code: Z-Image-Turbo, Qwen-Image-2.1, MiniMax-H3, LoRA Training Example

• September 20, 2026 Qwen-Image-2.1 was open-sourced. The model supports text-to-image generation, image editing, and RGBA output with a transparency channel. DiffSynth-Studio now provides full support for it. For details, please refer to the documentation and example code.

• September 14, 2026 We trained and open-sourced DiffSynth-Music, a controllable music generation model based on ACE-Step, which supports five control modes: Beats, Vocals, Accompany, Prosody, and Reference. For more information, please refer to:

  • Model: https://modelscope.cn/models/DiffSynth-Studio/DiffSynth-Music
  • Technical Report: https://arxiv.org/abs/2609.12774
  • Code: ./examples/diffsynth_music/

• September 11, 2026 We have added support for YuE2, a new music generation model. For details, please refer to the documentation and example code.

• September 10, 2026 We have integrated LTX-2.5, the latest audio-video generation model from Lightricks. The features include text-to-audio/video with automatic duration prediction, image-to-audio/video with keyframe interpolation, audio-to-video, audio-video retake, IC-LoRA pixel spatial upscaling, text-to-audio, INT8 quantized inference, low VRAM inference, and training. For details, please refer to the documentation and code.

• September 10, 2026 We have open-sourced DiffSynth-ComfyUI. Now you can run model inference with DiffSynth-Studio in ComfyUI.

• September 8, 2026 We trained a MiniMax-H3 Training Adapter for the CFG-distilled MiniMax-H3 base, a DeCFG LoRA training adapter offered in FL2VA and Ref2VA versions. The training data comes from the self-generated MiniMax-H3-Self-Generated-Dataset. Based on this adapter, we trained two toy models, MiniMax-H3-Songyu-LoRA (character identity, FL2VA) and MiniMax-H3-Ref2VA-FirstFrame-Lineart (lineart first-frame control, Ref2VA), as references for inference and fine-tuning.

• September 1, 2026 We have integrated SenseNova-U1.5, SenseTime’s unified multimodal model, for which we provide text-to-image generation, image editing, low VRAM inference, and training support. For details, please refer to the documentation and example code.

• August 31, 2026 We have integrated Qwen-Video-Edit, a video editing model developed by open-source community contributor yunpeng1998 based on the image editing model Qwen-Image-Edit. This serves as an excellent example of exploring and expanding model capabilities.

• August 25, 2026 We have open-sourced DiffSynth-WebUI, enabling one-click private deployment of LoRA training services. Combined with the model quantization feature, you can train large models even with consumer-grade GPUs.

• August 19, 2026 We have released the model quantization feature. It provides a unified QuantizeConfig entry point supporting multiple quantization backends including bitsandbytes, torchao, and comfy-kitchen, with capabilities such as online quantization, loading pre-quantized weights, mixed quantization, saving quantized models, and quantization + LoRA training. For details, please refer to the documentation.

Installation

Install from source (recommended):

git clone https://github.com/modelscope/DiffSynth-Studio.git  
cd DiffSynth-Studio
pip install -e .

For more installation methods and instructions for non-NVIDIA GPUs, please refer to the Installation Guide.

Basic Framework

DiffSynth-Studio redesigns the inference and training pipelines for mainstream Diffusion models (including FLUX, Wan, etc.), enabling efficient memory management and flexible model training.

Quick start: experience popular and the latest models:

Model overview:

• Image generation
  • Qwen-Image-2.1: Documentation, Example code
  • SenseNova-U1: Documentation, Example code
  • Boogu-Image: Documentation, Example code
  • Krea-2: Documentation, Example code
  • Ideogram 4: Documentation, Example code
  • HiDream-O1-Image: Documentation, Example code
  • JoyAI-Image: Documentation, Example code
  • ERNIE-Image: Documentation, Example code
  • FLUX.2: Documentation, Example code
  • Z-Image: Documentation, Example code
  • Anima: Documentation, Example code
  • Qwen-Image: Documentation, Example code
  • FLUX.1: Documentation, Example code
  • Stable Diffusion XL: Documentation, Example code
  • Stable Diffusion: Documentation, Example code
• Video generation
  • MiniMax-H3: Documentation, Example code
  • LingBot-Video: Documentation, Example code
  • LTX-2: Documentation, Example code
  • Wan: Documentation, Example code
• Audio generation
  • DiffSynth-Music: Documentation, Example code
  • MiniMax-Music3: Documentation, Example code
  • YuE2: Documentation, Example code
  • ACE-Step: Documentation, Example code
• Evaluation models: Documentation, Example code

View all supported models

Innovative Achievements

We believe that a well-developed open-source code framework can lower the threshold for technical exploration. We have achieved many interesting technologies based on this codebase. Perhaps you also have many wild ideas, and with DiffSynth-Studio, you can quickly realize these ideas.

Contact Us

All Supported Models