mirror of
https://github.com/huggingface/diffusers.git
synced 2026-01-29 07:22:12 +03:00
80 lines
3.6 KiB
Markdown
80 lines
3.6 KiB
Markdown
<!--Copyright 2025 The HuggingFace Team. All rights reserved.
|
||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
||
the License. You may obtain a copy of the License at
|
||
|
||
http://www.apache.org/licenses/LICENSE-2.0
|
||
|
||
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
||
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||
specific language governing permissions and limitations under the License.
|
||
-->
|
||
|
||
# DDIMScheduler
|
||
|
||
[Denoising Diffusion Implicit Models](https://huggingface.co/papers/2010.02502) (DDIM) by Jiaming Song, Chenlin Meng and Stefano Ermon.
|
||
|
||
The abstract from the paper is:
|
||
|
||
*Denoising diffusion probabilistic models (DDPMs) have achieved high quality image generation without adversarial training, yet they require simulating a Markov chain for many steps to produce a sample.
|
||
To accelerate sampling, we present denoising diffusion implicit models (DDIMs), a more efficient class of iterative implicit probabilistic models
|
||
with the same training procedure as DDPMs. In DDPMs, the generative process is defined as the reverse of a Markovian diffusion process.
|
||
We construct a class of non-Markovian diffusion processes that lead to the same training objective, but whose reverse process can be much faster to sample from.
|
||
We empirically demonstrate that DDIMs can produce high quality samples 10× to 50× faster in terms of wall-clock time compared to DDPMs, allow us to trade off computation for sample quality, and can perform semantically meaningful image interpolation directly in the latent space.*
|
||
|
||
The original codebase of this paper can be found at [ermongroup/ddim](https://github.com/ermongroup/ddim), and you can contact the author on [tsong.me](https://tsong.me/).
|
||
|
||
## Tips
|
||
|
||
The paper [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) claims that a mismatch between the training and inference settings leads to suboptimal inference generation results for Stable Diffusion. To fix this, the authors propose:
|
||
|
||
> [!WARNING]
|
||
> 🧪 This is an experimental feature!
|
||
|
||
1. rescale the noise schedule to enforce zero terminal signal-to-noise ratio (SNR)
|
||
|
||
```py
|
||
pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, rescale_betas_zero_snr=True)
|
||
```
|
||
|
||
2. train a model with `v_prediction` (add the following argument to the [train_text_to_image.py](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py) or [train_text_to_image_lora.py](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py) scripts)
|
||
|
||
```bash
|
||
--prediction_type="v_prediction"
|
||
```
|
||
|
||
3. change the sampler to always start from the last timestep
|
||
|
||
```py
|
||
pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
|
||
```
|
||
|
||
4. rescale classifier-free guidance to prevent over-exposure
|
||
|
||
```py
|
||
image = pipe(prompt, guidance_rescale=0.7).images[0]
|
||
```
|
||
|
||
For example:
|
||
|
||
```py
|
||
from diffusers import DiffusionPipeline, DDIMScheduler
|
||
import torch
|
||
|
||
pipe = DiffusionPipeline.from_pretrained("ptx0/pseudo-journey-v2", torch_dtype=torch.float16)
|
||
pipe.scheduler = DDIMScheduler.from_config(
|
||
pipe.scheduler.config, rescale_betas_zero_snr=True, timestep_spacing="trailing"
|
||
)
|
||
pipe.to("cuda")
|
||
|
||
prompt = "A lion in galaxies, spirals, nebulae, stars, smoke, iridescent, intricate detail, octane render, 8k"
|
||
image = pipe(prompt, guidance_rescale=0.7).images[0]
|
||
image
|
||
```
|
||
|
||
## DDIMScheduler
|
||
[[autodoc]] DDIMScheduler
|
||
|
||
## DDIMSchedulerOutput
|
||
[[autodoc]] schedulers.scheduling_ddim.DDIMSchedulerOutput
|