From 40de88af8c8ef6ecd69f99dabeeb07f8362fcf87 Mon Sep 17 00:00:00 2001 From: Steven Liu <59462357+stevhliu@users.noreply.github.com> Date: Thu, 13 Nov 2025 08:43:24 -0800 Subject: [PATCH] [docs] AutoModel (#12644) * automodel * fix --- docs/source/en/_toctree.yml | 2 + docs/source/en/api/models/auto_model.md | 10 +---- docs/source/en/using-diffusers/automodel.md | 46 +++++++++++++++++++++ 3 files changed, 49 insertions(+), 9 deletions(-) create mode 100644 docs/source/en/using-diffusers/automodel.md diff --git a/docs/source/en/_toctree.yml b/docs/source/en/_toctree.yml index 5e9299aece..e3b9f99927 100644 --- a/docs/source/en/_toctree.yml +++ b/docs/source/en/_toctree.yml @@ -22,6 +22,8 @@ title: Reproducibility - local: using-diffusers/schedulers title: Schedulers + - local: using-diffusers/automodel + title: AutoModel - local: using-diffusers/other-formats title: Model formats - local: using-diffusers/push_to_hub diff --git a/docs/source/en/api/models/auto_model.md b/docs/source/en/api/models/auto_model.md index 376dd12d12..aee9b5dbe5 100644 --- a/docs/source/en/api/models/auto_model.md +++ b/docs/source/en/api/models/auto_model.md @@ -12,15 +12,7 @@ specific language governing permissions and limitations under the License. # AutoModel -The `AutoModel` is designed to make it easy to load a checkpoint without needing to know the specific model class. `AutoModel` automatically retrieves the correct model class from the checkpoint `config.json` file. - -```python -from diffusers import AutoModel, AutoPipelineForText2Image - -unet = AutoModel.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", subfolder="unet") -pipe = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", unet=unet) -``` - +[`AutoModel`] automatically retrieves the correct model class from the checkpoint `config.json` file. ## AutoModel diff --git a/docs/source/en/using-diffusers/automodel.md b/docs/source/en/using-diffusers/automodel.md new file mode 100644 index 0000000000..957cbd17e3 --- /dev/null +++ b/docs/source/en/using-diffusers/automodel.md @@ -0,0 +1,46 @@ + + +# AutoModel + +The [`AutoModel`] class automatically detects and loads the correct model class (UNet, transformer, VAE) from a `config.json` file. You don't need to know the specific model class name ahead of time. It supports data types and device placement, and works across model types and libraries. + +The example below loads a transformer from Diffusers and a text encoder from Transformers. Use the `subfolder` parameter to specify where to load the `config.json` file from. + +```py +import torch +from diffusers import AutoModel, DiffusionPipeline + +transformer = AutoModel.from_pretrained( + "Qwen/Qwen-Image", subfolder="transformer", torch_dtype=torch.bfloat16, device_map="cuda" +) + +text_encoder = AutoModel.from_pretrained( + "Qwen/Qwen-Image", subfolder="text_encoder", torch_dtype=torch.bfloat16, device_map="cuda" +) +``` + +[`AutoModel`] also loads models from the [Hub](https://huggingface.co/models) that aren't included in Diffusers. Set `trust_remote_code=True` in [`AutoModel.from_pretrained`] to load custom models. + +```py +import torch +from diffusers import AutoModel + +transformer = AutoModel.from_pretrained( + "custom/custom-transformer-model", trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="cuda" +) +``` + +If the custom model inherits from the [`ModelMixin`] class, it gets access to the same features as Diffusers model classes, like [regional compilation](../optimization/fp16#regional-compilation) and [group offloading](../optimization/memory#group-offloading). + +> [!NOTE] +> Learn more about implementing custom models in the [Community components](../using-diffusers/custom_pipeline_overview#community-components) guide. \ No newline at end of file