1
0
mirror of https://github.com/huggingface/diffusers.git synced 2026-01-29 07:22:12 +03:00
Commit Graph

81 Commits

Author SHA1 Message Date
Daniel Gu
c161e299d8 Fix examples to load model in float16. 2023-05-16 13:33:51 -07:00
Daniel Gu
e56fab2def Improve UniDiffuser docs, particularly the usage examples, and improve slow tests with new expected outputs. 2023-05-16 12:58:11 -07:00
Daniel Gu
a54d6318df Fix some typos in the UniDiffuser docs. 2023-05-11 00:06:14 -07:00
Daniel Gu
ae7d549e0b Add documentation for UniDiffuser and fix some typos/formatting in docstrings. 2023-05-10 20:13:28 -07:00
clarencechen
029a28f06c Diffedit Zero-Shot Inpainting Pipeline (#2837)
* Update Pix2PixZero Auto-correlation Loss

* Add Stable Diffusion DiffEdit pipeline

* Add draft documentation and import code

* Bugfixes and refactoring

* Add option to not decode latents in the inversion process

* Harmonize preprocessing

* Revert "Update Pix2PixZero Auto-correlation Loss"

This reverts commit b218062fed.

* Update annotations

* rename `compute_mask` to `generate_mask`

* Update documentation

* Update docs

* Update Docs

* Fix copy

* Change shape of output latents to batch first

* Update docs

* Add first draft for tests

* Bugfix and update tests

* Add `cross_attention_kwargs` support for all pipeline methods

* Fix Copies

* Add support for PIL image latents

Add support for mask broadcasting

Update docs and tests

Align `mask` argument to `mask_image`

Remove height and width arguments

* Enable MPS Tests

* Move example docstrings

* Fix test

* Fix test

* fix pipeline inheritance

* Harmonize `prepare_image_latents` with StableDiffusionPix2PixZeroPipeline

* Register modules set to `None` in config for `test_save_load_optional_components`

* Move fixed logic to specific test class

* Clean changes to other pipelines

* Update new tests to coordinate with #2953

* Update slow tests for better results

* Safety to avoid potential problems with torch.inference_mode

* Add reference in SD Pipeline Overview

* Fix tests again

* Enforce determinism in noise for generate_mask

* Fix copies

* Widen test tolerance for fp16 based on `test_stable_diffusion_upscale_pipeline_fp16`

* Add LoraLoaderMixin and update `prepare_image_latents`

* clean up repeat and reg

* bugfix

* Remove invalid args from docs

Suppress spurious warning by repeating image before latent to mask gen
2023-05-05 07:23:51 -07:00
M. Tolga Cangöz
5151f210d8 Update logging.mdx (#2863)
Fix typos
2023-05-05 07:23:51 -07:00
apolinário
1147c76eca Update IF name to XL (#3262)
Co-authored-by: multimodalart <joaopaulo.passos+multimodal@gmail.com>
2023-05-05 07:23:50 -07:00
Ernie Chu
76e5941cb2 [docs] Update interface in repaint.mdx (#3119)
Update repaint.mdx

accomodate to #1701
2023-05-05 07:23:50 -07:00
Nipun Jindal
7880ed77fb [2064]: Add stochastic sampler (sample_dpmpp_sde) (#3020)
* [2064]: Add stochastic sampler

* [2064]: Add stochastic sampler

* [2064]: Add stochastic sampler

* [2064]: Add stochastic sampler

* [2064]: Add stochastic sampler

* [2064]: Add stochastic sampler

* [2064]: Add stochastic sampler

* Review comments

* [Review comment]: Add is_torchsde_available()

* [Review comment]: Test and docs

* [Review comment]

* [Review comment]

* [Review comment]

* [Review comment]

* [Review comment]

---------

Co-authored-by: njindal <njindal@adobe.com>
2023-05-05 07:23:50 -07:00
Pedro Cuenca
59986b6c56 [docs] only mention one stage (#3246)
* [docs] only mention one stage

* add blurb on auto accepting

---------

Co-authored-by: William Berman <WLBberman@gmail.com>
2023-05-05 07:23:50 -07:00
Sanchit Gandhi
f83fbbdc56 [AudioLDM] Update docs to use updated ckpt (#3240)
* [AudioLDM] Update docs to use updated ckpt

* make style
2023-05-05 07:23:50 -07:00
Patrick von Platen
416f31adf8 add model (#3230)
* add

* clean

* up

* clean up more

* fix more tests

* Improve docs further

* improve

* more fixes docs

* Improve docs more

* Update src/diffusers/models/unet_2d_condition.py

* fix

* up

* update doc links

* make fix-copies

* add safety checker and watermarker to stage 3 doc page code snippets

* speed optimizations docs

* memory optimization docs

* make style

* add watermarking snippets to doc string examples

* make style

* use pt_to_pil helper functions in doc strings

* skip mps tests

* Improve safety

* make style

* new logic

* fix

* fix bad onnx design

* make new stable diffusion upscale pipeline model arguments optional

* define has_nsfw_concept when non-pil output type

* lowercase linked to notebook name

---------

Co-authored-by: William Berman <WLBberman@gmail.com>
2023-05-05 07:23:50 -07:00
Patrick von Platen
0431637f11 Add ControlNet v1.1 docs (#3226)
Add v1.1 docs
2023-05-05 07:22:14 -07:00
1lint
f3300a869a add from_ckpt method as Mixin (#2318)
* add mixin class for pipeline from original sd ckpt

* Improve

* make style

* merge main into

* Improve more

* fix more

* up

* Apply suggestions from code review

* finish docs

* rename

* make style

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2023-05-05 07:22:13 -07:00
Takuma Mori
7f3cb6d29f Add to support Guess Mode for StableDiffusionControlnetPipleline (#2998)
* add guess mode (WIP)

* fix uncond/cond order

* support guidance_scale=1.0 and batch != 1

* remove magic coeff

* add docstring

* add intergration test

* add document to controlnet.mdx

* made the comments a bit more explanatory

* fix table
2023-05-05 07:22:13 -07:00
Sayak Paul
870169e08f [Docs] refactor text-to-video zero (#3049)
* fix: norm group test for UNet3D.

* refactor text-to-video zero docs.
2023-05-05 07:22:13 -07:00
Susung Hong
184dab3b31 [Docs] update Self-Attention Guidance docs (#2952)
* Update index.mdx

* Edit docs & add HF space link

* Only change equation numbers in comments
2023-05-05 07:22:13 -07:00
Sayak Paul
2d6b410525 [LoRA] Enabling limited LoRA support for text encoder (#2918)
* add: first draft for a better LoRA enabler.

* make fix-copies.

* feat: backward compatibility.

* add: entry to the docs.

* add: tests.

* fix: docs.

* fix: norm group test for UNet3D.

* feat: add support for flat dicts.

* add depcrcation message instead of warning.
2023-05-05 07:22:13 -07:00
Andranik Movsisyan
aa028386f3 [Pipeline] Add TextToVideoZeroPipeline (#2954)
* add TextToVideoZeroPipeline and CrossFrameAttnProcessor

* add docs for text-to-video zero

* add teaser image for text-to-video zero docs

* Fix review changes. Add Documentation. Add test

* clean up the codes in pipeline_text_to_video.py. Add descriptive comments and docstrings

* make style && make quality

* make fix-copies

* make requested changes to docs. use huggingface server links for resources, delete res folder

* make style && make quality && make fix-copies

* make style && make quality

* Apply suggestions from code review

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2023-05-05 07:22:12 -07:00
Guspan Tanadi
229f9a22c9 docs: Link Navigation Path API Pipelines (#2976)
* docs: link navigation Safe Stable Diffusion

Link navigation API pipelines text2img and using diffusers Conditional Image Generation.

* docs: link navigation Versatile Diffusion

Removing exceeding path Stable Diffusion Overview.

* docs: Python extension Spectrogram Diffusion

Link navigation Spectrogram Diffusion Pipeline source code

* docs: Link navigation AltDiffusion Pipelines

Stable Diffusion Overview and Using Diffusers path.
2023-05-05 07:22:12 -07:00
Guspan Tanadi
11bcc6e895 Removing explicit markdown extension (#2944)
Trigger from previous PR. Build the page once again.
2023-05-05 07:22:11 -07:00
M. Tolga Cangöz
9ddd0ebaed Update ddpm.mdx (#2929) 2023-05-05 07:22:11 -07:00
M. Tolga Cangöz
353e6b5fb3 Update ddim.mdx (#2926) 2023-05-05 07:22:11 -07:00
M. Tolga Cangöz
32039273b7 Update score_sde_vp.mdx (#2938) 2023-05-05 07:22:11 -07:00
M. Tolga Cangöz
909a0d86a4 Update score_sde_ve.mdx (#2937) 2023-05-05 07:22:11 -07:00
M. Tolga Cangöz
8ab78d3a23 Update unipc.mdx (#2936) 2023-05-05 07:22:11 -07:00
M. Tolga Cangöz
98e9d4d337 Update euler_ancestral.mdx (#2932) 2023-05-05 07:22:11 -07:00
M. Tolga Cangöz
c43356267b Update controlnet.mdx (#2912)
.
2023-03-31 14:32:36 +01:00
M. Tolga Cangöz
89b23d9869 Update image_variation.mdx (#2911)
.
2023-03-31 14:31:43 +01:00
Guspan Tanadi
419660c99b Have fix current pipeline link (#2910)
Also capitalization notebook provider name
2023-03-31 14:31:14 +01:00
Sayak Paul
b2021273eb [Docs] add an example use for StableUnCLIPPipeline in the pipeline docs (#2897)
* improve stable unclip doc.

* add: entry of StableUnCLIPPipeline to the docs

* Apply suggestions from code review

Co-authored-by: apolinario <joaopaulo.passos@gmail.com>

---------

Co-authored-by: apolinario <joaopaulo.passos@gmail.com>
2023-03-30 17:14:04 +05:30
M. Tolga Cangöz
628fefb232 Update stable_diffusion_safe.mdx (#2870)
Fix typos
2023-03-28 17:23:54 +01:00
M. Tolga Cangöz
03fe36f183 Update paint_by_example.mdx (#2869)
.
2023-03-28 17:23:39 +01:00
M. Tolga Cangöz
ef4c2fa4f1 Update alt_diffusion.mdx (#2865)
Fix typos
2023-03-28 17:17:53 +01:00
M. Tolga Cangöz
3980858ad4 Update overview.mdx (#2864)
Fix typos
2023-03-28 17:17:33 +01:00
Sayak Paul
fab4f3d6e4 improve stable unclip doc. (#2823) 2023-03-28 08:18:29 +05:30
Sayak Paul
5883d8d4d1 [Docs] update docs (Stable unCLIP) to reflect the updated ckpts. (#2815)
* update docs to reflect the updated ckpts.

* update: point about prompt.

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* emove image resizing.

* Apply suggestions from code review

* Apply suggestions from code review

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2023-03-24 17:24:19 +01:00
Bahjat Kawar
37a44bb283 Add ModelEditing pipeline (#2721)
* TIME first commit

* styling.

* styling 2.

* fixes; tests

* apply styling and doc fix.

* remove sups.

* fixes

* remove temp file

* move augmentations to const

* added doc entry

* code quality

* customize augmentations

* quality

* quality

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2023-03-24 13:01:39 +05:30
Sanchit Gandhi
b94880e536 Add AudioLDM (#2232)
* Add AudioLDM

* up

* add vocoder

* start unet

* unconditional unet

* clap, vocoder and vae

* clean-up: conversion scripts

* fix: conversion script token_type_ids

* clean-up: pipeline docstring

* tests: from SD

* clean-up: cpu offload vocoder instead of safety checker

* feat: adapt tests to audioldm

* feat: add docs

* clean-up: amend pipeline docstrings

* clean-up: make style

* clean-up: make fix-copies

* fix: add doc path to toctree

* clean-up: args for conversion script

* clean-up: paths to checkpoints

* fix: use conditional unet

* clean-up: make style

* fix: type hints for UNet

* clean-up: docstring for UNet

* clean-up: make style

* clean-up: remove duplicate in docstring

* clean-up: make style

* clean-up: make fix-copies

* clean-up: move imports to start in code snippet

* fix: pass cross_attention_dim as a list/tuple to unet

* clean-up: make fix-copies

* fix: update checkpoint path

* fix: unet cross_attention_dim in tests

* film embeddings -> class embeddings

* Apply suggestions from code review

Co-authored-by: Will Berman <wlbberman@gmail.com>

* fix: unet film embed to use existing args

* fix: unet tests to use existing args

* fix: make style

* fix: transformers import and version in init

* clean-up: make style

* Revert "clean-up: make style"

This reverts commit 5d6d1f8b32.

* clean-up: make style

* clean-up: use pipeline tester mixin tests where poss

* clean-up: skip attn slicing test

* fix: add torch dtype to docs

* fix: remove conversion script out of src

* fix: remove .detach from 1d waveform

* fix: reduce default num inf steps

* fix: swap height/width -> audio_length_in_s

* clean-up: make style

* fix: remove nightly tests

* fix: imports in conversion script

* clean-up: slim-down to two slow tests

* clean-up: slim-down fast tests

* fix: batch consistent tests

* clean-up: make style

* clean-up: remove vae slicing fast test

* clean-up: propagate changes to doc

* fix: increase test tol to 1e-2

* clean-up: finish docs

* clean-up: make style

* feat: vocoder / VAE compatibility check

* feat: possibly expand / cut audio waveform

* fix: pipeline call signature test

* fix: slow tests output len

* clean-up: make style

* make style

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: William Berman <WLBberman@gmail.com>
2023-03-23 19:00:21 +01:00
YiYi Xu
df91c44712 Flax controlnet (#2727)
* add contronet flax

---------

Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-03-23 05:46:23 -10:00
Sayak Paul
0d7aac3e8d [Docs] small fixes to the text to video doc. (#2787)
* small fixes to the text to video doc.

* add: Spaces link.

* add: warning on research-only model.
2023-03-23 18:57:02 +05:30
Kashif Rasul
2ef9bdd76f Music Spectrogram diffusion pipeline (#1044)
* initial TokenEncoder and ContinuousEncoder

* initial modules

* added ContinuousContextTransformer

* fix copy paste error

* use numpy for get_sequence_length

* initial terminal relative positional encodings

* fix weights keys

* fix assert

* cross attend style: concat encodings

* make style

* concat once

* fix formatting

* Initial SpectrogramPipeline

* fix input_tokens

* make style

* added mel output

* ignore weights for config

* move mel to numpy

* import pipeline

* fix class names and import

* moved models to models folder

* import ContinuousContextTransformer and SpectrogramDiffusionPipeline

* initial spec diffusion converstion script

* renamed config to t5config

* added weight loading

* use arguments instead of t5config

* broadcast noise time to batch dim

* fix call

* added scale_to_features

* fix weights

* transpose laynorm weight

* scale is a vector

* scale the query outputs

* added comment

* undo scaling

* undo depth_scaling

* inital get_extended_attention_mask

* attention_mask is none in self-attention

* cleanup

* manually invert attention

* nn.linear need bias=False

* added T5LayerFFCond

* remove to fix conflict

* make style and dummy

* remove unsed variables

* remove predict_epsilon

* Move accelerate to a soft-dependency (#1134)

* finish

* finish

* Update src/diffusers/modeling_utils.py

* Update src/diffusers/pipeline_utils.py

Co-authored-by: Anton Lozhkov <anton@huggingface.co>

* more fixes

* fix

Co-authored-by: Anton Lozhkov <anton@huggingface.co>

* fix order

* added initial midi to note token data pipeline

* added int to int tokenizer

* remove duplicate

* added logic for segments

* add melgan to pipeline

* move autoregressive gen into pipeline

* added note_representation_processor_chain

* fix dtypes

* remove immutabledict req

* initial doc

* use np.where

* require note_seq

* fix typo

* update dependency

* added note-seq to test

* added is_note_seq_available

* fix import

* added toc

* added example usage

* undo for now

* moved docs

* fix merge

* fix imports

* predict first segment

* avoid un-needed copy to and from cpu

* make style

* Copyright

* fix style

* add test and fix inference steps

* remove bogus files

* reorder models

* up

* remove transformers dependency

* make work with diffusers cross attention

* clean more

* remove @

* improve further

* up

* uP

* Apply suggestions from code review

* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py

* loop over all tokens

* make style

* Added a section on the model

* fix formatting

* grammer

* formatting

* make fix-copies

* Update src/diffusers/pipelines/__init__.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/pipelines/spectrogram_diffusion/pipeline_spectrogram_diffusion.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* added callback ad optional ionnx

* do not squeeze batch dim

* clean up more

* upload

* convert jax to nnumpy

* make style

* fix warning

* make fix-copies

* fix warning

* add initial fast tests

* add initial pipeline_params

* eval mode due to dropout

* skip batch tests as pipeline runs on a single file

* make style

* fix relative path

* fix doc tests

* Update src/diffusers/models/t5_film_transformer.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/models/t5_film_transformer.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update docs/source/en/api/pipelines/spectrogram_diffusion.mdx

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* add MidiProcessor

* format

* fix org

* Apply suggestions from code review

* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py

* make style

* pin protobuf to <4

* fix formatting

* white space

* tensorboard needs protobuf

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Anton Lozhkov <anton@huggingface.co>
2023-03-23 14:06:17 +01:00
Naoki Ainoya
14e3a28c12 Rename 'CLIPFeatureExtractor' class to 'CLIPImageProcessor' (#2732)
The 'CLIPFeatureExtractor' class name has been renamed to 'CLIPImageProcessor' in order to comply with future deprecation. This commit includes the necessary changes to the affected files.
2023-03-23 13:49:22 +01:00
Sayak Paul
c681ad1af2 add: section on multiple controlnets. (#2762)
* add: section on multiple controlnets.

Co-authored-by: William Berman <WLBberman@gmail.com>

* fix: docs.

* fix: docs.

---------

Co-authored-by: William Berman <WLBberman@gmail.com>
2023-03-23 09:55:25 +05:30
Patrick von Platen
ca1a22296d [MS Text To Video] Add first text to video (#2738)
* [MS Text To Video} Add first text to video

* upload

* make first model example

* match unet3d params

* make sure weights are correcctly converted

* improve

* forward pass works, but diff result

* make forward work

* fix more

* finish

* refactor video output class.

* feat: add support for a video export utility.

* fix: opencv availability check.

* run make fix-copies.

* add: docs for the model components.

* add: standalone pipeline doc.

* edit docstring of the pipeline.

* add: right path to TransformerTempModel

* add: first set of tests.

* complete fast tests for text to video.

* fix bug

* up

* three fast tests failing.

* add: note on slow tests

* make work with all schedulers

* apply styling.

* add slow tests

* change file name

* update

* more correction

* more fixes

* finish

* up

* Apply suggestions from code review

* up

* finish

* make copies

* fix pipeline tests

* fix more tests

* Apply suggestions from code review

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

* apply suggestions

* up

* revert

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
2023-03-22 18:39:33 +01:00
Steven Liu
588e50bc57 [docs] Reorganize table of contents (#2671)
* reorg toc

* reorg toc some more

* remove duplicate config
2023-03-15 16:28:18 -07:00
YiYi Xu
a062e47ec3 add flax pipelines to api doc + doc string examples (#2600)
* add api doc for flax pipeline + doc string examples

* make style

---------

Co-authored-by: yiyixuxu <yixu@yis-macbook-pro.lan>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2023-03-09 13:00:29 +01:00
Vico Chu
b36cbd4fba Fix: controlnet docs format (#2559) 2023-03-06 09:25:21 +01:00
Patrick von Platen
7f0f7e1e91 Correct section docs (#2540) 2023-03-03 18:34:34 +01:00
Patrick von Platen
1021929313 Small fixes for controlnet (#2542)
* Small fixes for controlnet

* finish links
2023-03-03 14:20:43 +01:00