mirror of
https://github.com/huggingface/diffusers.git
synced 2026-01-29 07:22:12 +03:00
74 lines
7.5 KiB
Markdown
74 lines
7.5 KiB
Markdown
<!--Copyright 2025 The HuggingFace Team. All rights reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations under the License.
|
|
-->
|
|
|
|
# Schedulers
|
|
|
|
🤗 Diffusers provides many scheduler functions for the diffusion process. A scheduler takes a model's output (the sample which the diffusion process is iterating on) and a timestep to return a denoised sample. The timestep is important because it dictates where in the diffusion process the step is; data is generated by iterating forward *n* timesteps and inference occurs by propagating backward through the timesteps. Based on the timestep, a scheduler may be *discrete* in which case the timestep is an `int` or *continuous* in which case the timestep is a `float`.
|
|
|
|
Depending on the context, a scheduler defines how to iteratively add noise to an image or how to update a sample based on a model's output:
|
|
|
|
- during *training*, a scheduler adds noise (there are different algorithms for how to add noise) to a sample to train a diffusion model
|
|
- during *inference*, a scheduler defines how to update a sample based on a pretrained model's output
|
|
|
|
Many schedulers are implemented from the [k-diffusion](https://github.com/crowsonkb/k-diffusion) library by [Katherine Crowson](https://github.com/crowsonkb/), and they're also widely used in A1111. To help you map the schedulers from k-diffusion and A1111 to the schedulers in 🤗 Diffusers, take a look at the table below:
|
|
|
|
| A1111/k-diffusion | 🤗 Diffusers | Usage |
|
|
|---------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------|
|
|
| DPM++ 2M | [`DPMSolverMultistepScheduler`] | |
|
|
| DPM++ 2M Karras | [`DPMSolverMultistepScheduler`] | init with `use_karras_sigmas=True` |
|
|
| DPM++ 2M SDE | [`DPMSolverMultistepScheduler`] | init with `algorithm_type="sde-dpmsolver++"` |
|
|
| DPM++ 2M SDE Karras | [`DPMSolverMultistepScheduler`] | init with `use_karras_sigmas=True` and `algorithm_type="sde-dpmsolver++"` |
|
|
| DPM++ 2S a | N/A | very similar to `DPMSolverSinglestepScheduler` |
|
|
| DPM++ 2S a Karras | N/A | very similar to `DPMSolverSinglestepScheduler(use_karras_sigmas=True, ...)` |
|
|
| DPM++ SDE | [`DPMSolverSinglestepScheduler`] | |
|
|
| DPM++ SDE Karras | [`DPMSolverSinglestepScheduler`] | init with `use_karras_sigmas=True` |
|
|
| DPM2 | [`KDPM2DiscreteScheduler`] | |
|
|
| DPM2 Karras | [`KDPM2DiscreteScheduler`] | init with `use_karras_sigmas=True` |
|
|
| DPM2 a | [`KDPM2AncestralDiscreteScheduler`] | |
|
|
| DPM2 a Karras | [`KDPM2AncestralDiscreteScheduler`] | init with `use_karras_sigmas=True` |
|
|
| DPM adaptive | N/A | |
|
|
| DPM fast | N/A | |
|
|
| Euler | [`EulerDiscreteScheduler`] | |
|
|
| Euler a | [`EulerAncestralDiscreteScheduler`] | |
|
|
| Heun | [`HeunDiscreteScheduler`] | |
|
|
| LMS | [`LMSDiscreteScheduler`] | |
|
|
| LMS Karras | [`LMSDiscreteScheduler`] | init with `use_karras_sigmas=True` |
|
|
| N/A | [`DEISMultistepScheduler`] | |
|
|
| N/A | [`UniPCMultistepScheduler`] | |
|
|
|
|
## Noise schedules and schedule types
|
|
| A1111/k-diffusion | 🤗 Diffusers |
|
|
|--------------------------|----------------------------------------------------------------------------|
|
|
| Karras | init with `use_karras_sigmas=True` |
|
|
| sgm_uniform | init with `timestep_spacing="trailing"` |
|
|
| simple | init with `timestep_spacing="trailing"` |
|
|
| exponential | init with `timestep_spacing="linspace"`, `use_exponential_sigmas=True` |
|
|
| beta | init with `timestep_spacing="linspace"`, `use_beta_sigmas=True` |
|
|
|
|
All schedulers are built from the base [`SchedulerMixin`] class which implements low level utilities shared by all schedulers.
|
|
|
|
## SchedulerMixin
|
|
[[autodoc]] SchedulerMixin
|
|
|
|
## SchedulerOutput
|
|
[[autodoc]] schedulers.scheduling_utils.SchedulerOutput
|
|
|
|
## KarrasDiffusionSchedulers
|
|
|
|
[`KarrasDiffusionSchedulers`] are a broad generalization of schedulers in 🤗 Diffusers. The schedulers in this class are distinguished at a high level by their noise sampling strategy, the type of network and scaling, the training strategy, and how the loss is weighed.
|
|
|
|
The different schedulers in this class, depending on the ordinary differential equations (ODE) solver type, fall into the above taxonomy and provide a good abstraction for the design of the main schedulers implemented in 🤗 Diffusers. The schedulers in this class are given [here](https://github.com/huggingface/diffusers/blob/a69754bb879ed55b9b6dc9dd0b3cf4fa4124c765/src/diffusers/schedulers/scheduling_utils.py#L32).
|
|
|
|
## PushToHubMixin
|
|
|
|
[[autodoc]] utils.PushToHubMixin
|