YiYi Xu
5c404f20f4
[WIP] masked_latent_inputs for inpainting pipeline ( #4819 )
...
* add
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-01 06:55:31 -10:00
YiYi Xu
d8b6f5d09e
support AutoPipeline.from_pipe between a pipeline and its ControlNet pipeline counterpart ( #4861 )
...
add
2023-09-01 06:53:03 -10:00
Dhruv Nair
189e9f01b3
Test Cleanup Precision issues ( #4812 )
...
* proposal for flaky tests
* more precision fixes
* move more tests to use cosine distance
* more test fixes
* clean up
* use default attn
* clean up
* update expected value
* make style
* make style
* Apply suggestions from code review
* Update src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
* make style
* fix failing tests
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-01 17:58:37 +05:30
NguyΓͺΜn CΓ΄ng TuΜ Anh
38466c369f
Add GLIGEN Text Image implementation ( #4777 )
...
* Add GLIGEN Text Image implementation
* add style transfer from image
* fix check_repository_consistency
* add convert script GLIGEN model to Diffusers
* rename attention type
* fix style code
* remove PositionNetTextImage
* Revert "fix check_repository_consistency"
This reverts commit 15f098c96e
2023-09-01 15:48:01 +05:30
YiYi Xu
75f81c25d1
fix sdxl-inpaint fast test ( #4859 )
...
fix inpaint test
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-08-31 15:42:58 -10:00
Patrick von Platen
9f1936d2fc
Fix Unfuse Lora ( #4833 )
...
* Fix Unfuse Lora
* add tests
* Fix more
* Fix more
* Fix all
* make style
* make style
2023-08-30 09:32:25 +05:30
Sayak Paul
3768d4d77c
[Core] refactor encode_prompt ( #4617 )
...
* refactoring of encode_prompt()
* better handling of device.
* fix: device determination
* fix: device determination 2
* handle num_images_per_prompt
* revert changes in loaders.py and give birth to encode_prompt().
* minor refactoring for encode_prompt()/
* make backward compatible.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix: concatenation of the neg and pos embeddings.
* incorporate encode_prompt() in test_stable_diffusion.py
* turn it into big PR.
* make it bigger
* gligen fixes.
* more fixes to fligen
* _encode_prompt -> encode_prompt in tests
* first batch
* second batch
* fix blasphemous mistake
* fix
* fix: hopefully for the final time.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-30 08:57:26 +05:30
Nikhil Gajendrakumar
8ccb619416
VaeImageProcessor: Allow image resizing also for torch and numpy inputs ( #4832 )
...
Co-authored-by: Nikhil Gajendrakumar <nikhilkatte@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-29 22:45:05 +02:00
VitjanZ
7200daa412
Support saving multiple t2i adapter models under one checkpoint ( #4798 )
...
* adding save and load for MultiAdapter, adding test
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Adding changes from review test_stable_diffusion_adapter
* import sorting fix
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-29 09:24:40 +02:00
Patrick von Platen
c583f3b452
Fuse loras ( #4473 )
...
* Fuse loras
* initial implementation.
* add slow test one.
* styling
* add: test for checking efficiency
* print
* position
* place model offload correctly
* style
* style.
* unfuse test.
* final checks
* remove warning test
* remove warnings altogether
* debugging
* tighten up tests.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* denugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debuging
* debugging
* debugging
* debugging
* suit up the generator initialization a bit.
* remove print
* update assertion.
* debugging
* remove print.
* fix: assertions.
* style
* can generator be a problem?
* generator
* correct tests.
* support text encoder lora fusion.
* tighten up tests.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-29 09:14:24 +02:00
Chong Mou
12358b986f
add models for T2I-Adapter-XL ( #4696 )
...
* T2I-Adapter-XL
* update
* update
* add pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify modeling_text_unet
* fix styling.
* fix: copies.
* adapter settings
* new test case
* new test case
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* revert prints.
* new test case
* remove print
* org test case
* add test_pipeline
* styling.
* fix copies.
* modify test parameter
* style.
* add adapter-xl doc
* double quotes in docs
* Fix potential type mismatch
* style.
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
2023-08-29 10:34:07 +05:30
YiYi Xu
5eeedd9e33
add StableDiffusionXLControlNetImg2ImgPipeline ( #4592 )
...
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-28 08:16:27 -10:00
YiYi Xu
a971c598b5
fix auto_pipeline: pass kwargs to load_config ( #4793 )
...
* fix
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-28 07:42:16 -10:00
YiYi Xu
934d439a42
fix bug in StableDiffusionXLControlNetPipeline when use guess_mode ( #4799 )
...
* fix
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-28 06:51:17 -10:00
Dhruv Nair
e3f3672f46
Fix Disentangle ONNX and non-ONNX pipeline ( #4656 )
...
* initial commit to fix inheritance issue
* clean up sd onnx upscale
* clean up
2023-08-28 21:14:49 +05:30
Patrick von Platen
766aa50f70
[LoRA Attn Processors] Refactor LoRA Attn Processors ( #4765 )
...
* [LoRA Attn] Refactor LoRA attn
* correct for network alphas
* fix more
* fix more tests
* fix more tests
* Move below
* Finish
* better version
* correct serialization format
* fix
* fix more
* fix more
* fix more
* Apply suggestions from code review
* Update src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
* deprecation
* relax atol for slow test slighly
* Finish tests
* make style
* make style
2023-08-28 10:38:09 +05:30
Patrick von Platen
c4d2823601
[SDXL Lora] Fix last ben sdxl lora ( #4797 )
...
* Fix last ben sdxl lora
* Correct typo
* make style
2023-08-26 23:31:56 +02:00
Sayak Paul
3be0ff9056
[Core] Support negative conditions in SDXL ( #4774 )
...
* add: support negative conditions.
* fix: key
* add: tests
* address PR feedback.
* add documentation
* add img2img support.
* add inpainting support.
* ad controlnet support
* Apply suggestions from code review
* modify wording in the doc.
2023-08-26 09:13:44 +05:30
YiYi Xu
b7b1a30bc4
refactor prepare_mask_and_masked_image with VaeImageProcessor ( #4444 )
...
* refactor image processor for mask
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-08-25 08:18:48 -10:00
YiYi Xu
b3b2d30cd8
fix a bug in from_pretrained when load optional components ( #4745 )
...
* fix
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-25 06:25:48 -10:00
Dhruv Nair
3bba44d74e
[WIP ] Proposal to address precision issues in CI ( #4775 )
...
* proposal for flaky tests
* clean up
2023-08-25 19:12:09 +05:30
Sanchit Gandhi
b1290d3fb8
Convert MusicLDM ( #4579 )
...
* from audioldm
* fix vae
* move to new pipeline
* copied from audioldm
* remove redundant control flow
* iterate
* fix docstring
* finish pipeline
* tests: from audioldm2
* iterate
* finish fast tests
* finish slow integration tests
* add docs
* remove dtype test
* update toctree
* "copied from" in conversion (where possible)
* Update docs/source/en/api/pipelines/musicldm.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix docstring
* make nightly
* style
* fix dtype test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-25 13:31:00 +01:00
Sanchit Gandhi
29a11c2a94
[AudioLDM 2] Pipeline fixes ( #4738 )
...
* fix docs
* fix unet docs
* use image output for latents
* fix hub checkpoints
* fix pipeline example
* update example
* return_dict = False
* revert image pipeline output
* revert doc changes
* remove dtype test
* make style
* remove docstring updates
* remove unet docstring update
* Empty commit to re-trigger CI
* fix cpu offload
* fix dtype test
* add offload test
2023-08-25 11:38:10 +01:00
Will Berman
3105c710ba
[fix] multi t2i adapter set total_downscale_factor ( #4621 )
...
* [fix] multi t2i adapter set total_downscale_factor
* move image checks into check inputs
* remove copied from
2023-08-24 12:01:23 -07:00
Dhruv Nair
4f05058bb7
Clean up flaky behaviour on Slow CUDA Pytorch Push Tests ( #4759 )
...
use max diff to compare model outputs
2023-08-24 18:58:02 +05:30
YiYi Xu
cd21b965d1
add a step_index counter ( #4347 )
...
add self.step_index
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-23 10:49:54 -10:00
Ollin Boer Bohan
052bf3280b
Fix AutoencoderTiny encoder scaling convention ( #4682 )
...
* Fix AutoencoderTiny encoder scaling convention
* Add [-1, 1] -> [0, 1] rescaling to EncoderTiny
* Move [0, 1] -> [-1, 1] rescaling from AutoencoderTiny.decode to DecoderTiny
(i.e. immediately after the final conv, as early as possible)
* Fix missing [0, 255] -> [0, 1] rescaling in AutoencoderTiny.forward
* Update AutoencoderTinyIntegrationTests to protect against scaling issues.
The new test constructs a simple image, round-trips it through AutoencoderTiny,
and confirms the decoded result is approximately equal to the source image.
This test checks behavior with and without tiling enabled.
This test will fail if new AutoencoderTiny scaling issues are introduced.
* Context: Raw TAESD weights expect images in [0, 1], but diffusers'
convention represents images with zero-centered values in [-1, 1],
so AutoencoderTiny needs to scale / unscale images at the start of
encoding and at the end of decoding in order to work with diffusers.
* Re-add existing AutoencoderTiny test, update golden values
* Add comments to AutoencoderTiny.forward
2023-08-23 08:38:37 +05:30
Patrick von Platen
38efac9f61
Revert "Move controlnet load local tests to nightly ( #4543 )" ( #4713 )
...
This reverts commit 7b07f9812a
2023-08-22 19:55:15 +02:00
Sayak Paul
9141c1f9d5
[Core] enable lora for sdxl controlnets too and add slow tests. ( #4666 )
...
* enable lora for sdxl controlnets too.
* add: tests
* fix: assertion values.
2023-08-22 07:13:23 +05:30
Sanchit Gandhi
7a24977ce3
Add AudioLDM 2 ( #4549 )
...
* from audioldm
* unet down + mid
* vae, clap, flan-t5
* start sequence audio mae
* iterate on audioldm encoder
* finish encoder
* finish weight conversion
* text pre-processing
* gpt2 pre-processing
* fix projection model
* working
* unet equivalence
* finish in base
* add unet cond
* finish unet
* finish custom unet
* start clean-up
* revert base unet changes
* refactor pre-processing
* tests: from audioldm
* fix some tests
* more fixes
* iterate on tests
* make fix copies
* harden fast tests
* slow integration tests
* finish tests
* update checkpoint
* update copyright
* docs
* remove outdated method
* add docstring
* make style
* remove decode latents
* enable cpu offload
* (text_encoder_1, tokenizer_1) -> (text_encoder, tokenizer)
* more clean up
* more refactor
* build pr docs
* Update docs/source/en/api/pipelines/audioldm2.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* small clean
* tidy conversion
* update for large checkpoint
* generate -> generate_language_model
* full clap model
* shrink clap-audio in tests
* fix large integration test
* fix fast tests
* use generation config
* make style
* update docs
* finish docs
* finish doc
* update tests
* fix last test
* syntax
* finalise tests
* refactor projection model in prep for TTS
* fix fast tests
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-21 12:34:21 +01:00
Isotr0py
67ea2b7afa
Support tiled encode/decode for AutoencoderTiny ( #4627 )
...
* Impl tae slicing and tiling
* add tae tiling test
* add parameterized test
* formatted code
* fix failed test
* style docs
2023-08-18 09:12:55 +05:30
Sayak Paul
a10107f92b
fix: lora sdxl tests ( #4652 )
2023-08-17 15:59:50 +05:30
Jacqui Wei
7c3e7fedcd
Fix use_onnx parameter usage in from_pretrained func and update test_download_no_onnx_by_default test ( #4508 )
...
* add missing use_onnx in from_pretrained func
* fix test_download_no_onnx_by_default test func
* address comments
* split test cases
2023-08-17 11:49:32 +05:30
Patrick von Platen
029fb41695
[Safetensors] Make safetensors the default way of saving weights ( #4235 )
...
* make safetensors default
* set default save method as safetensors
* update tests
* update to support saving safetensors
* update test to account for safetensors default
* update example tests to use safetensors
* update example to support safetensors
* update unet tests for safetensors
* fix failing loader tests
* fix qc issues
* fix pipeline tests
* fix example test
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-17 10:54:28 +05:30
Batuhan Taskaya
852dc76d6d
Support higher dimension LoRAs ( #4625 )
...
* Support higher dimension LoRAs
* add: tests
* fix: assertion values.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-17 10:07:07 +05:30
Scott Lessans
064f150813
Fix UnboundLocalError during LoRA loading ( #4523 )
...
* fixed
* add: tests
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-17 09:33:35 +05:30
Sayak Paul
5049599143
[Core] feat: MultiControlNet support for SDXL ControlNet pipeline ( #4597 )
...
* core: add multicontrolnet support to sdxl controlnet
* modify checks.
* fix: original_size determination
* add: tests for multi controlnet sdxl.
* remove unnecessary prints.
2023-08-16 20:30:39 +05:30
Dirk Morris
a7de96505b
Fix unipc use_karras_sigmas exception - fixes huggingface/diffusers#4580 ( #4581 )
...
* Fix unipc karras sigmas exception - fixes huggingface/diffusers#4580
* Add unipc scheduler tests for karras sigmas
2023-08-16 10:01:53 +05:30
nikhil-masterful
da5ab51d54
Add GLIGEN implementation ( #4441 )
...
* Add GLIGEN implementation
* GLIGEN: Fix code quality check failures
* GLIGEN: Fix Import block un-sorted or un-formatted failures
* GLIGEN: Fix check_repository_consistency failures
* GLIGEN: Add 'PositionNet' to versatile_diffusion/modeling_text_unet.py
* GLIGEN: check_repository_consistency: fix 'copy does not match' error
* GLIGEN: Fix review comments (1)
* GLIGEN: Fix E721 Do not compare types, use `isinstance()` failures
* GLIGEN : Ensure _encode_prompt() copy matches to StableDiffusionPipeline
* GLIGEN: Fix ruff E721 failure in unidiffuser/test_unidiffuser.py
* GLIGEN: doc_builder: restyle pipeline_stable_diffusion_gligen.py
* GIGLEN: reset files unrelated to gligen
* GLIGEN: Fix documentation comments (1)
* GLIGEN: Fix review comments (2)
* GLIGEN: Added FastTest
* GLIGEN: Fix review comments (3)
2023-08-16 09:34:17 +05:30
Sayak Paul
15782fd506
[Pipeline utils] feat: implement push_to_hub for standalone models, schedulers as well as pipelines ( #4128 )
...
* feat: implement push_to_hub for standalone models.
* address PR feedback.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* remove max_shard_size.
* add: support for scheduler push_to_hub
* enable push_to_hub support for flax schedulers.
* enable push_to_hub for pipelines.
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com >
* reflect pr feedback.
* address another round of deedback.
* better handling of kwargs.
* add: tests
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com >
* setting hub staging to False for now.
* incorporate staging test as a separate job.
Co-authored-by: ydshieh <2521628+ydshieh@users.noreply.github.com >
* fix: tokenizer loading.
* fix: json dumping.
* move is_staging_test to a better location.
* better treatment to tokens.
* define repo_id to better handle concurrency
* style
* explicitly set token
* Empty-Commit
* move SUER, TOKEN to test
* collate org_repo_id
* delete repo
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Lucain <lucainp@gmail.com >
Co-authored-by: ydshieh <2521628+ydshieh@users.noreply.github.com >
2023-08-15 07:39:22 +05:30
Abhipsha Das
c8d86e9f0a
Remove code snippets containing is_safetensors_available() ( #4521 )
...
* [WIP] Remove code snippets containing `is_safetensors_available()`
* Modifying `import_utils.py`
* update pipeline tests for safetensor default
* fix test related to cached requests
* address import nits
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-11 11:05:22 +05:30
Dhruv Nair
7b07f9812a
Move controlnet load local tests to nightly ( #4543 )
...
move controlnet load local tests to nihghtly
2023-08-09 23:00:42 +05:30
Steven Liu
16ad13b61d
[docs] Clean scheduler api ( #4204 )
...
* clean scheduler mixin
* up to dpmsolvermultistep
* finish cleaning
* first draft
* fix overview table
* apply feedback
* update reference code
2023-08-09 09:00:35 -07:00
Dhruv Nair
a67ff32301
Move slow tests to nightly ( #4526 )
...
* move slow pix2pixzero tests to nightly
* move slow panorama tests to nightly
* move txt2video full test to nightly
* clean up
* remove nightly test from text to video pipeline
2023-08-09 12:38:15 +02:00
Dhruv Nair
71c8224159
Moving certain pipelines slow tests to nightly ( #4469 )
...
* move audioldm tests to nightly
* move kandinsky im2img ddpm test to nightly
* move flax dpm test to nightly
* move diffedit dpm test to nightly
* move fp16 slow tests to nightly
2023-08-07 17:28:56 +02:00
Patrick von Platen
ea1fcc28a4
[SDXL] Allow SDXL LoRA to be run with less than 16GB of VRAM ( #4470 )
...
* correct
* correct blocks
* finish
* finish
* finish
* Apply suggestions from code review
* fix
* up
* up
* up
* Update examples/dreambooth/README_sdxl.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Apply suggestions from code review
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-04 20:06:38 +02:00
Sayak Paul
06f73bd6d1
[Tests] Adds integration tests for SDXL LoRAs ( #4462 )
...
* add: integration tests for SDXL LoRAs.
* change pipeline class.
* fix assertion values.
* print values again.
* let's see.
* let's see.
* let's see.
* finish
2023-08-04 16:25:53 +05:30
Dhruv Nair
801a5e2199
Cleanup Pass on flaky slow tests for Stable Diffusion ( #4455 )
...
* lower num inference steps and precision checkk
* fix flaky inpaint tests
* remove unsued imports
* set unet default attn processor
2023-08-04 10:24:56 +02:00
YiYi Xu
29ece0db79
a few fix for kandinsky combined pipeline ( #4352 )
...
* add xformer
* enable_sequential_cpu_offload
* style
* Update src/diffusers/pipelines/kandinsky/pipeline_kandinsky_combined.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-03 15:10:41 -10:00
Dhruv Nair
1d2587bb34
move tests to nightly ( #4451 )
...
* move tests to nightly
* clean up code quality issues
* more clean up
2023-08-03 15:25:28 +02:00
