mirror of
https://github.com/huggingface/diffusers.git
synced 2026-01-27 17:22:53 +03:00
ef2ea33c3b
* Changes for VQ-diffusion VQVAE Add specify dimension of embeddings to VQModel: `VQModel` will by default set the dimension of embeddings to the number of latent channels. The VQ-diffusion VQVAE has a smaller embedding dimension, 128, than number of latent channels, 256. Add AttnDownEncoderBlock2D and AttnUpDecoderBlock2D to the up and down unet block helpers. VQ-diffusion's VQVAE uses those two block types. * Changes for VQ-diffusion transformer Modify attention.py so SpatialTransformer can be used for VQ-diffusion's transformer. SpatialTransformer: - Can now operate over discrete inputs (classes of vector embeddings) as well as continuous. - `in_channels` was made optional in the constructor so two locations where it was passed as a positional arg were moved to kwargs - modified forward pass to take optional timestep embeddings ImagePositionalEmbeddings: - added to provide positional embeddings to discrete inputs for latent pixels BasicTransformerBlock: - norm layers were made configurable so that the VQ-diffusion could use AdaLayerNorm with timestep embeddings - modified forward pass to take optional timestep embeddings CrossAttention: - now may optionally take a bias parameter for its query, key, and value linear layers FeedForward: - Internal layers are now configurable ApproximateGELU: - Activation function in VQ-diffusion's feedforward layer AdaLayerNorm: - Norm layer modified to incorporate timestep embeddings * Add VQ-diffusion scheduler * Add VQ-diffusion pipeline * Add VQ-diffusion convert script to diffusers * Add VQ-diffusion dummy objects * Add VQ-diffusion markdown docs * Add VQ-diffusion tests * some renaming * some fixes * more renaming * correct * fix typo * correct weights * finalize * fix tests * Apply suggestions from code review Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com> * Apply suggestions from code review Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * finish * finish * up Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com> Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
105 lines
3.2 KiB
YAML
105 lines
3.2 KiB
YAML
- sections:
|
|
- local: index
|
|
title: "🧨 Diffusers"
|
|
- local: quicktour
|
|
title: "Quicktour"
|
|
- local: installation
|
|
title: "Installation"
|
|
title: "Get started"
|
|
- sections:
|
|
- sections:
|
|
- local: using-diffusers/loading
|
|
title: "Loading Pipelines, Models, and Schedulers"
|
|
- local: using-diffusers/configuration
|
|
title: "Configuring Pipelines, Models, and Schedulers"
|
|
- local: using-diffusers/custom_pipeline_overview
|
|
title: "Loading and Adding Custom Pipelines"
|
|
title: "Loading & Hub"
|
|
- sections:
|
|
- local: using-diffusers/unconditional_image_generation
|
|
title: "Unconditional Image Generation"
|
|
- local: using-diffusers/conditional_image_generation
|
|
title: "Text-to-Image Generation"
|
|
- local: using-diffusers/img2img
|
|
title: "Text-Guided Image-to-Image"
|
|
- local: using-diffusers/inpaint
|
|
title: "Text-Guided Image-Inpainting"
|
|
- local: using-diffusers/custom_pipeline_examples
|
|
title: "Community Pipelines"
|
|
- local: using-diffusers/contribute_pipeline
|
|
title: "How to contribute a Pipeline"
|
|
title: "Pipelines for Inference"
|
|
title: "Using Diffusers"
|
|
- sections:
|
|
- local: optimization/fp16
|
|
title: "Memory and Speed"
|
|
- local: optimization/onnx
|
|
title: "ONNX"
|
|
- local: optimization/open_vino
|
|
title: "OpenVINO"
|
|
- local: optimization/mps
|
|
title: "MPS"
|
|
title: "Optimization/Special Hardware"
|
|
- sections:
|
|
- local: training/overview
|
|
title: "Overview"
|
|
- local: training/unconditional_training
|
|
title: "Unconditional Image Generation"
|
|
- local: training/text_inversion
|
|
title: "Textual Inversion"
|
|
- local: training/dreambooth
|
|
title: "Dreambooth"
|
|
- local: training/text2image
|
|
title: "Text-to-image fine-tuning"
|
|
title: "Training"
|
|
- sections:
|
|
- local: conceptual/stable_diffusion
|
|
title: "Stable Diffusion"
|
|
- local: conceptual/philosophy
|
|
title: "Philosophy"
|
|
- local: conceptual/contribution
|
|
title: "How to contribute?"
|
|
title: "Conceptual Guides"
|
|
- sections:
|
|
- sections:
|
|
- local: api/models
|
|
title: "Models"
|
|
- local: api/schedulers
|
|
title: "Schedulers"
|
|
- local: api/diffusion_pipeline
|
|
title: "Diffusion Pipeline"
|
|
- local: api/logging
|
|
title: "Logging"
|
|
- local: api/configuration
|
|
title: "Configuration"
|
|
- local: api/outputs
|
|
title: "Outputs"
|
|
title: "Main Classes"
|
|
- sections:
|
|
- local: api/pipelines/overview
|
|
title: "Overview"
|
|
- local: api/pipelines/ddim
|
|
title: "DDIM"
|
|
- local: api/pipelines/ddpm
|
|
title: "DDPM"
|
|
- local: api/pipelines/latent_diffusion
|
|
title: "Latent Diffusion"
|
|
- local: api/pipelines/latent_diffusion_uncond
|
|
title: "Unconditional Latent Diffusion"
|
|
- local: api/pipelines/pndm
|
|
title: "PNDM"
|
|
- local: api/pipelines/score_sde_ve
|
|
title: "Score SDE VE"
|
|
- local: api/pipelines/stable_diffusion
|
|
title: "Stable Diffusion"
|
|
- local: api/pipelines/stochastic_karras_ve
|
|
title: "Stochastic Karras VE"
|
|
- local: api/pipelines/dance_diffusion
|
|
title: "Dance Diffusion"
|
|
- local: api/pipelines/vq_diffusion
|
|
title: "VQ Diffusion"
|
|
- local: api/pipelines/repaint
|
|
title: "RePaint"
|
|
title: "Pipelines"
|
|
title: "API"
|