Sanchit Gandhi
7a24977ce3
Add AudioLDM 2 ( #4549 )
...
* from audioldm
* unet down + mid
* vae, clap, flan-t5
* start sequence audio mae
* iterate on audioldm encoder
* finish encoder
* finish weight conversion
* text pre-processing
* gpt2 pre-processing
* fix projection model
* working
* unet equivalence
* finish in base
* add unet cond
* finish unet
* finish custom unet
* start clean-up
* revert base unet changes
* refactor pre-processing
* tests: from audioldm
* fix some tests
* more fixes
* iterate on tests
* make fix copies
* harden fast tests
* slow integration tests
* finish tests
* update checkpoint
* update copyright
* docs
* remove outdated method
* add docstring
* make style
* remove decode latents
* enable cpu offload
* (text_encoder_1, tokenizer_1) -> (text_encoder, tokenizer)
* more clean up
* more refactor
* build pr docs
* Update docs/source/en/api/pipelines/audioldm2.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* small clean
* tidy conversion
* update for large checkpoint
* generate -> generate_language_model
* full clap model
* shrink clap-audio in tests
* fix large integration test
* fix fast tests
* use generation config
* make style
* update docs
* finish docs
* finish doc
* update tests
* fix last test
* syntax
* finalise tests
* refactor projection model in prep for TTS
* fix fast tests
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-21 12:34:21 +01:00
Isotr0py
67ea2b7afa
Support tiled encode/decode for AutoencoderTiny ( #4627 )
...
* Impl tae slicing and tiling
* add tae tiling test
* add parameterized test
* formatted code
* fix failed test
* style docs
2023-08-18 09:12:55 +05:30
Sayak Paul
a10107f92b
fix: lora sdxl tests ( #4652 )
2023-08-17 15:59:50 +05:30
Jacqui Wei
7c3e7fedcd
Fix use_onnx parameter usage in from_pretrained func and update test_download_no_onnx_by_default test ( #4508 )
...
* add missing use_onnx in from_pretrained func
* fix test_download_no_onnx_by_default test func
* address comments
* split test cases
2023-08-17 11:49:32 +05:30
Patrick von Platen
029fb41695
[Safetensors] Make safetensors the default way of saving weights ( #4235 )
...
* make safetensors default
* set default save method as safetensors
* update tests
* update to support saving safetensors
* update test to account for safetensors default
* update example tests to use safetensors
* update example to support safetensors
* update unet tests for safetensors
* fix failing loader tests
* fix qc issues
* fix pipeline tests
* fix example test
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-17 10:54:28 +05:30
Batuhan Taskaya
852dc76d6d
Support higher dimension LoRAs ( #4625 )
...
* Support higher dimension LoRAs
* add: tests
* fix: assertion values.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-17 10:07:07 +05:30
Scott Lessans
064f150813
Fix UnboundLocalError during LoRA loading ( #4523 )
...
* fixed
* add: tests
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-17 09:33:35 +05:30
Sayak Paul
5049599143
[Core] feat: MultiControlNet support for SDXL ControlNet pipeline ( #4597 )
...
* core: add multicontrolnet support to sdxl controlnet
* modify checks.
* fix: original_size determination
* add: tests for multi controlnet sdxl.
* remove unnecessary prints.
2023-08-16 20:30:39 +05:30
Dirk Morris
a7de96505b
Fix unipc use_karras_sigmas exception - fixes huggingface/diffusers#4580 ( #4581 )
...
* Fix unipc karras sigmas exception - fixes huggingface/diffusers#4580
* Add unipc scheduler tests for karras sigmas
2023-08-16 10:01:53 +05:30
nikhil-masterful
da5ab51d54
Add GLIGEN implementation ( #4441 )
...
* Add GLIGEN implementation
* GLIGEN: Fix code quality check failures
* GLIGEN: Fix Import block un-sorted or un-formatted failures
* GLIGEN: Fix check_repository_consistency failures
* GLIGEN: Add 'PositionNet' to versatile_diffusion/modeling_text_unet.py
* GLIGEN: check_repository_consistency: fix 'copy does not match' error
* GLIGEN: Fix review comments (1)
* GLIGEN: Fix E721 Do not compare types, use `isinstance()` failures
* GLIGEN : Ensure _encode_prompt() copy matches to StableDiffusionPipeline
* GLIGEN: Fix ruff E721 failure in unidiffuser/test_unidiffuser.py
* GLIGEN: doc_builder: restyle pipeline_stable_diffusion_gligen.py
* GIGLEN: reset files unrelated to gligen
* GLIGEN: Fix documentation comments (1)
* GLIGEN: Fix review comments (2)
* GLIGEN: Added FastTest
* GLIGEN: Fix review comments (3)
2023-08-16 09:34:17 +05:30
Sayak Paul
15782fd506
[Pipeline utils] feat: implement push_to_hub for standalone models, schedulers as well as pipelines ( #4128 )
...
* feat: implement push_to_hub for standalone models.
* address PR feedback.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* remove max_shard_size.
* add: support for scheduler push_to_hub
* enable push_to_hub support for flax schedulers.
* enable push_to_hub for pipelines.
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com >
* reflect pr feedback.
* address another round of deedback.
* better handling of kwargs.
* add: tests
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com >
* setting hub staging to False for now.
* incorporate staging test as a separate job.
Co-authored-by: ydshieh <2521628+ydshieh@users.noreply.github.com >
* fix: tokenizer loading.
* fix: json dumping.
* move is_staging_test to a better location.
* better treatment to tokens.
* define repo_id to better handle concurrency
* style
* explicitly set token
* Empty-Commit
* move SUER, TOKEN to test
* collate org_repo_id
* delete repo
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Lucain <lucainp@gmail.com >
Co-authored-by: ydshieh <2521628+ydshieh@users.noreply.github.com >
2023-08-15 07:39:22 +05:30
Abhipsha Das
c8d86e9f0a
Remove code snippets containing is_safetensors_available() ( #4521 )
...
* [WIP] Remove code snippets containing `is_safetensors_available()`
* Modifying `import_utils.py`
* update pipeline tests for safetensor default
* fix test related to cached requests
* address import nits
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-11 11:05:22 +05:30
Dhruv Nair
7b07f9812a
Move controlnet load local tests to nightly ( #4543 )
...
move controlnet load local tests to nihghtly
2023-08-09 23:00:42 +05:30
Steven Liu
16ad13b61d
[docs] Clean scheduler api ( #4204 )
...
* clean scheduler mixin
* up to dpmsolvermultistep
* finish cleaning
* first draft
* fix overview table
* apply feedback
* update reference code
2023-08-09 09:00:35 -07:00
Dhruv Nair
a67ff32301
Move slow tests to nightly ( #4526 )
...
* move slow pix2pixzero tests to nightly
* move slow panorama tests to nightly
* move txt2video full test to nightly
* clean up
* remove nightly test from text to video pipeline
2023-08-09 12:38:15 +02:00
Dhruv Nair
71c8224159
Moving certain pipelines slow tests to nightly ( #4469 )
...
* move audioldm tests to nightly
* move kandinsky im2img ddpm test to nightly
* move flax dpm test to nightly
* move diffedit dpm test to nightly
* move fp16 slow tests to nightly
2023-08-07 17:28:56 +02:00
Patrick von Platen
ea1fcc28a4
[SDXL] Allow SDXL LoRA to be run with less than 16GB of VRAM ( #4470 )
...
* correct
* correct blocks
* finish
* finish
* finish
* Apply suggestions from code review
* fix
* up
* up
* up
* Update examples/dreambooth/README_sdxl.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Apply suggestions from code review
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-04 20:06:38 +02:00
Sayak Paul
06f73bd6d1
[Tests] Adds integration tests for SDXL LoRAs ( #4462 )
...
* add: integration tests for SDXL LoRAs.
* change pipeline class.
* fix assertion values.
* print values again.
* let's see.
* let's see.
* let's see.
* finish
2023-08-04 16:25:53 +05:30
Dhruv Nair
801a5e2199
Cleanup Pass on flaky slow tests for Stable Diffusion ( #4455 )
...
* lower num inference steps and precision checkk
* fix flaky inpaint tests
* remove unsued imports
* set unet default attn processor
2023-08-04 10:24:56 +02:00
YiYi Xu
29ece0db79
a few fix for kandinsky combined pipeline ( #4352 )
...
* add xformer
* enable_sequential_cpu_offload
* style
* Update src/diffusers/pipelines/kandinsky/pipeline_kandinsky_combined.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-03 15:10:41 -10:00
Dhruv Nair
1d2587bb34
move tests to nightly ( #4451 )
...
* move tests to nightly
* clean up code quality issues
* more clean up
2023-08-03 15:25:28 +02:00
cmdr2
4c4fe042a7
Accept pooled_prompt_embeds in the SDXL Controlnet pipeline. Fixes an error if prompt_embeds are passed. ( #4309 )
...
* Accept pooled_prompt_embeds in the SDXL Controlnet pipeline. Fixes an error if prompt_embeds are passed.
* Add a test for pooled prompt embeds
2023-08-03 13:05:19 +05:30
Sayak Paul
18fc40c169
[Feat] add tiny Autoencoder for (almost) instant decoding ( #4384 )
...
* add: model implementation of tiny autoencoder.
* add: inits.
* push the latest devs.
* add: conversion script and finish.
* add: scaling factor args.
* debugging
* fix denormalization.
* fix: positional argument.
* handle use_torch_2_0_or_xformers.
* handle post_quant_conv
* handle dtype
* fix: sdxl image processor for tiny ae.
* fix: sdxl image processor for tiny ae.
* unify upcasting logic.
* copied from madness.
* remove trailing whitespace.
* set is_tiny_vae = False
* address PR comments.
* change to AutoencoderTiny
* make act_fn an str throughout
* fix: apply_forward_hook decorator call
* get rid of the special is_tiny_vae flag.
* directly scale the output.
* fix dummies?
* fix: act_fn.
* get rid of the Clamp() layer.
* bring back copied from.
* movement of the blocks to appropriate modules.
* add: docstrings to AutoencoderTiny
* add: documentation.
* changes to the conversion script.
* add doc entry.
* settle tests.
* style
* add one slow test.
* fix
* fix 2
* fix 2
* fix: 4
* fix: 5
* finish integration tests
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* style
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-02 23:58:05 +05:30
Sayak Paul
816ca0048f
[LoRA] Fix SDXL text encoder LoRAs ( #4371 )
...
* temporarily disable text encoder loras.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debbuging.
* modify doc.
* rename tests.
* print slices.
* fix: assertions
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-02 17:00:56 +05:30
YiYi Xu
c10861ee1b
fix test_float16_inference ( #4412 )
...
fix
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-08-01 07:49:50 -10:00
Dhruv Nair
6f4355f89f
Cleanup pass for flaky Slow Tests for Stable diffusion ( #4415 )
...
* update expected slice so img2img compile tests pass
* use default attn processor
* use default attn processor and update expected slice value to pass test
* use default attn processor
* set default attn processor and update expected slice
* set default attn processor and change precision for check
* set unet to use default attn processor
2023-08-01 18:21:14 +02:00
Sayak Paul
4a4cdd6b07
[Feat] Support SDXL Kohya-style LoRA ( #4287 )
...
* sdxl lora changes.
* better name replacement.
* better replacement.
* debugging
* debugging
* debugging
* debugging
* debugging
* remove print.
* print state dict keys.
* print
* distingisuih better
* debuggable.
* fxi: tyests
* fix: arg from training script.
* access from class.
* run style
* debug
* save intermediate
* some simplifications for SDXL LoRA
* styling
* unet config is not needed in diffusers format.
* fix: dynamic SGM block mapping for SDXL kohya loras (#4322 )
* Use lora compatible layers for linear proj_in/proj_out (#4323 )
* improve condition for using the sgm_diffusers mapping
* informative comment.
* load compatible keys and embedding layer maaping.
* Get SDXL 1.0 example lora to load
* simplify
* specif ranks and hidden sizes.
* better handling of k rank and hidden
* debug
* debug
* debug
* debug
* debug
* fix: alpha keys
* add check for handling LoRAAttnAddedKVProcessor
* sanity comment
* modifications for text encoder SDXL
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* denugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* up
* up
* up
* up
* up
* up
* unneeded comments.
* unneeded comments.
* kwargs for the other attention processors.
* kwargs for the other attention processors.
* debugging
* debugging
* debugging
* debugging
* improve
* debugging
* debugging
* more print
* Fix alphas
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* clean up
* clean up.
* debugging
* fix: text
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Batuhan Taskaya <batuhan@python.org >
2023-07-28 19:49:49 +02:00
Patrick von Platen
306a7bd047
[ONNX] Don't download ONNX model by default ( #4338 )
...
* [Download] Don't download ONNX weights by default
* [Download] Don't download ONNX weights by default
* [Download] Don't download ONNX weights by default
* fix more
* finish
* finish
* finish
2023-07-28 14:02:48 +02:00
Patrick von Platen
18b018c864
[SDXL Refiner] Fix refiner forward pass for batched input ( #4327 )
...
* fix_batch_xl
* Fix other pipelines as well
* up
* up
* Update tests/pipelines/stable_diffusion_xl/test_stable_diffusion_xl_inpaint.py
* sort
* up
* Finish it all up Co-authored-by: Bagheera <bghira@users.github.com >
* Co-authored-by: Bagheera bghira@users.github.com
* Co-authored-by: Bagheera <bghira@users.github.com >
* Finish it all up Co-authored-by: Bagheera <bghira@users.github.com >
2023-07-28 12:34:18 +02:00
Sayak Paul
7d0d073261
[Tests] add test for pipeline import. ( #4276 )
...
* add test for pipeline import.
* Update tests/others/test_dependencies.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* address suggestions
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-07-28 00:08:15 +05:30
Patrick von Platen
1926331eaf
[Local loading] Correct bug with local files only ( #4318 )
...
* [Local loading] Correct bug with local files only
* file not found error
* fix
* finish
2023-07-27 16:16:46 +02:00
YiYi Xu
5fd3dca5f3
fix a bug in StableDiffusionUpscalePipeline when prompt is None ( #4278 )
...
* fix batch_size
* add test
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-07-27 15:07:50 +02:00
Patrick von Platen
b3e5cd6b4d
[Kandinsky] Add combined pipelines / Fix cpu model offload / Fix inpainting ( #4207 )
...
* Add combined pipeline
* Download readme
* Upload
* up
* up
* fix final
* Add enable model cpu offload kandinsky
* finish
* finish
* Fix
* fix more
* make style
* fix kandinsky mask
* fix inpainting test
* add callbacks
* add tests
* fix tests
* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* docs
* docs
* correct docs
* fix tests
* add warning
* correct docs
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2023-07-26 17:13:55 +02:00
Batuhan Taskaya
ff8f58086b
Load Kohya-ss style LoRAs with auxilary states ( #4147 )
...
* Support to load Kohya-ss style LoRA file format (without restrictions)
Co-Authored-By: Takuma Mori <takuma104@gmail.com >
Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com >
* tmp: add sdxl to mlp_modules
---------
Co-authored-by: Takuma Mori <takuma104@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-07-26 00:24:19 +02:00
Harutatsu Akiyama
428dbfecd9
[SDXL and IP2P]: instruction pix2pix XL training and pipeline ( #4079 )
...
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* [Community] Implementation of the IADB community pipeline (#3996 )
* community pipeline: implementation of iadb
* iadb.py: reformat using black
* iadb.py: linting update
* add kandinsky to readme table (#4081 )
Co-authored-by: yiyixuxu <yixu310@gmail,com>
* [From Single File] Force accelerate to be installed (#4078 )
force accelerate to be installed
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Clean up IP2P SDXL code
* Clean up IP2P SDXL code
* [IP2P and SDXL] clean up code
* [IP2P and SDXL] clean up code
* [IP2P and SDXL] clean up code
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Add README_SDXL
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Fix the copy problems
* [IP2P SDXL] Add license
* [IP2P SDXL] Add license
* [IP2P SDXL] Add license
* [IP2P SDXL] Address code reivew for selecting VAE andd others
* [IP2P SDXL] Update README_sdxl
* [IP2P SDXL] Update __init__
* [IP2P SDXL] Update dummy_torch_and_transformers_and_invisible_watermark_objects
* address patrick's comments and some additions to readmes.
---------
Co-authored-by: Harutatsu Akiyama <kf.zy.qin@gmail.com >
Co-authored-by: Thomas Chambon <36728882+tchambon@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-07-25 18:19:35 +05:30
YiYi Xu
c11d11d63d
[draft v2] AutoPipeline ( #4138 )
...
* initial
* style
* from ...pipelines -> from ..pipeline_util
* make style
* fix-copies
* fix value_guided_sampling oops
* style
* add test
* Show failing test
* update from_pipe
* fix
* add controlnet, additional test and register unused original config
* update for controlnet
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* store unused config as private attribute and pass if can
* add doc
* kandinsky inpaint pipeline does not work with decoder checkpoint
* update doc
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* style
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix
* Apply suggestions from code review
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-07-25 13:20:35 +02:00
Sayak Paul
365e8461ac
[SDXL DreamBooth LoRA] add support for text encoder fine-tuning ( #4097 )
...
* Allow low precision sd xl
* finish
* finish
* feat: initial draft for supporting text encoder lora finetuning for SDXL DreamBooth
* fix: variable assignments.
* add: autocast block.
* add debugging
* vae dtype hell
* fix: vae dtype hell.
* fix: vae dtype hell 3.
* clean up
* lora text encoder loader.
* fix: unwrapping models.
* add: tests.
* docs.
* handle unexpected keys.
* fix vae dtype in the final inference.
* fix scope problem.
* fix: save_model_card args.
* initialize: prefix to None.
* fix: dtype issues.
* apply gixes.
* debgging.
* debugging
* debugging
* debugging
* debugging
* debugging
* add: fast tests.
* pre-tokenize.
* address: will's comments.
* fix: loader and tests.
* fix: dataloader.
* simplify dataloader.
* length.
* simplification.
* make style && make quality
* simplify state_dict munging
* fix: tests.
* fix: state_dict packing.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-07-25 05:35:48 +05:30
39th president of the United States, probably
e98fabc550
Allow specifying denoising_start and denoising_end as integers representing the discrete timesteps, fixing the XL ensemble not working for many schedulers ( #4115 )
...
* Fix the XL ensemble not working for any kerras scheduler sigmas and having an off by one bug
* Update src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py
* make sytle
---------
Co-authored-by: Jimmy <39@🇺🇸 .com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-07-24 19:44:35 +02:00
Patrick von Platen
3ba36f97b8
[SD-XL] Fix sdxl controlnet inference ( #4238 )
...
* Fix controlnet xl inference
* correct some sd xl control inference
2023-07-24 18:43:35 +02:00
Patrick von Platen
b288684d25
[SDXL] Fix sd xl encode prompt ( #4237 )
...
* [SDXL] Fix sd xl encode prompt
* add tests
2023-07-24 18:37:07 +02:00
Hu Ye
8e5921cac1
fix a bug of prompt embeds in sdxl ( #4099 )
...
* fix bug in sdxl
* Update pipeline_stable_diffusion_xl_img2img.py
* Update pipeline_stable_diffusion_xl.py
* Update pipeline_stable_diffusion_xl_img2img.py
* Update pipeline_stable_diffusion_xl_inpaint.py
* Update pipeline_stable_diffusion_xl.py
* Update pipeline_stable_diffusion_xl_img2img.py
* Update pipeline_stable_diffusion_xl_inpaint.py
* Update pipeline_stable_diffusion_xl_img2img.py
* Update pipeline_controlnet_sd_xl.py
* Update pipeline_controlnet_sd_xl.py
* Update pipeline_stable_diffusion_xl.py
* Update pipeline_stable_diffusion_xl_img2img.py
* Update pipeline_stable_diffusion_xl_inpaint.py
* Update test_stable_diffusion_xl.py
* Update test_stable_diffusion_xl.py
* Update test_stable_diffusion_xl.py
add test on prompt_embeds
* add test on prompt_embeds
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-07-24 10:14:20 +02:00
YiYi Xu
8e8954bd15
fix no CFG for kandinsky pipelines ( #4193 )
...
* fix bug when no cfg
* style
* fix no cfg for shap-e and cycle
* style
* fix no cfg for sdxl
* fix copies
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-07-24 10:10:14 +02:00
apolinário
aed30dff6b
Allow passing different prompts to each text_encoder on stable_diffusion_xl pipelines ( #4156 )
...
* sdxl prompt2
* Improve checks
* doc linting
* whoops
* remove cat
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Add other pipelines and tests
* Add multi-prompting to docs
* doc and copies check
* Fix copied froms
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Bring back the original code for unrelated files
* Fix tests
* Fix img2img
* Fix all
* fix
---------
Co-authored-by: multimodalart <joaopaulo.passos+multimodal@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-07-21 14:50:22 +02:00
Batuhan Taskaya
ad787082e2
Fix unloading of LoRAs when xformers attention procs are in use ( #4179 )
2023-07-21 14:29:20 +05:30
YiYi Xu
47b3346422
Shap-E: add support for mesh output ( #4062 )
...
* add output_type=mesh
* update img2img
* make style
* add doc
* make style
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* add docstring for output_type
* add a section in doc about hub mesh visualization/ rotation
* update conversion script so default background is white
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update src/diffusers/pipelines/shap_e/pipeline_shap_e_img2img.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* renderer -> shap_e_renderer
* img2img renderer -> shap_e_renderer
* fix tests
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-07-20 18:05:13 +02:00
Ruslan Vorovchenko
07f1fbb18e
Asymmetric vqgan ( #3956 )
...
* added AsymmetricAutoencoderKL
* fixed copies+dummy
* added script to convert original asymmetric vqgan
* added docs
* updated docs
* fixed style
* fixes, added tests
* update doc
* fixed doc
* fixed tests
* naming
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* naming
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* udpated code example
* updated doc
* comments fixes
* added docstring
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* comments fixes
* added inpaint pipeline tests
* comment suggestion: delete method
* yet another fixes
---------
Co-authored-by: Ruslan Vorovchenko <r.vorovchenko@prequelapp.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-07-20 17:51:06 +02:00
Lim Swee Kiat
2551b73670
Fix bug in ControlNetPipelines with MultiControlNetModel of length 1 ( #4032 )
...
* Fix bug in ControlNetPipelines with MultiControlNetModel of length 1
* Add tests for varying number of ControlNet models
* Fix missing indexing for control_guidance_start and control_guidance_end
* Fix code quality
* Separate test for MultiControlNet with one model
* Revert formatting of earlier test
2023-07-20 17:45:08 +02:00
Patrick von Platen
6b1abba18d
Add controlnet and vae from single file ( #4084 )
...
* Add controlnet from single file
* Updates
* make style
* finish
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-07-19 14:50:27 +02:00
Saurav Maheshkar
470f51cd26
feat: add act_fn param to OutValueFunctionBlock ( #3994 )
...
* feat: add act_fn param to OutValueFunctionBlock
* feat: update unet1d tests to not use mish
* feat: add `mish` as the default activation function
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* feat: drop mish tests from unet1d
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-07-19 14:44:44 +02:00
Sayak Paul
3eb498e7b4
[Core] add: controlnet support for SDXL ( #4038 )
...
* add: controlnet sdxl.
* modifications to controlnet.
* run styling.
* add: __init__.pys
* incorporate https://github.com/huggingface/diffusers/pull/4019 changes.
* run make fix-copies.
* resize the conditioning images.
* remove autocast.
* run styling.
* disable autocast.
* debugging
* device placement.
* back to autocast.
* remove comment.
* save some memory by reusing the vae and unet in the pipeline.
* apply styling.
* Allow low precision sd xl
* finish
* finish
* changes to accommodate the improved VAE.
* modifications to how we handle vae encoding in the training.
* make style
* make existing controlnet fast tests pass.
* change vae checkpoint cli arg.
* fix: vae pretrained paths.
* fix: steps in get_scheduler().
* debugging.
* debugging./
* fix: weight conversion.
* add: docs.
* add: limited tests./
* add: datasets to the requirements.
* update docstrings and incorporate the usage of watermarking.
* incorporate fix from #4083
* fix watermarking dependency handling.
* run make-fix-copies.
* Empty-Commit
* Update requirements_sdxl.txt
* remove vae upcasting part.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* run make style
* run make fix-copies.
* disable suppot for multicontrolnet.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* run make fix-copies.
* dtyle/.
* fix-copies.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-07-18 18:25:34 +05:30
