Carson Katri
6281d2066b
Add callbacks to WuerstchenDecoderPipeline and WuerstchenCombinedPipeline ( #5154 )
2023-09-25 19:26:53 +02:00
Anh71me
28254c79b6
Fix type annotation ( #5146 )
...
* Fix type annotation on Scheduler.from_pretrained
* Fix type annotation on PIL.Image
2023-09-25 19:26:39 +02:00
MLRichter
0bc6be6960
Update wuerstchen.md ( #5156 )
2023-09-25 18:43:08 +02:00
Patrick von Platen
144c3a8b7c
[Imports] Fix many import bugs and make sure that doc builder CI test works correctly ( #5176 )
...
* [Doc builder] Ensure slow import for doc builder
* Apply suggestions from code review
* env for doc builder
* fix more
* [Diffusers] Set import to slow as env variable
* fix docs
* fix docs
* Apply suggestions from code review
* Apply suggestions from code review
* fix docs
* fix docs
2023-09-25 18:06:51 +02:00
Patrick von Platen
30a512ea69
[Core] Improve .to(...) method, fix offloads multi-gpu, add docstring, add dtype ( #5132 )
...
* fix cpu offload
* fix
* fix
* Update src/diffusers/pipelines/pipeline_utils.py
* make style
* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* fix more
* fix more
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2023-09-25 14:10:18 +02:00
Dhruv Nair
92f15f5bd4
Model CPU offload fix for BLIPDiffusion ( #5174 )
...
cpu offload fix for blip diffusion
2023-09-25 17:07:32 +05:30
Patrick von Platen
22b19d578e
[Tests] Add is flaky decorator ( #5139 )
...
* add is flaky decorator
* fix more
2023-09-25 13:24:44 +02:00
Sayak Paul
787195fe20
Fix/controlnet lora ( #5157 )
...
* print
* print
* print
* print
* print
* debugging
* debugging
* debugging
* debugging
* safer condition.
* remove prints and try excepts.
* Empty-Commit
* Apply suggestions from code review
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-25 12:08:05 +02:00
Mishig
48664d62b8
Delete duplicatd doc file ( #5169 )
2023-09-24 19:58:13 +02:00
YiYi Xu
5b11c5dc77
fix the add_noise function for dpm-multi et al ( #5158 )
...
* remove to _device() for sigmas
* update add_noise to use simgas
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-23 09:07:50 -10:00
Sayak Paul
310cf32801
add: note on whom to tag for issues related to community pipelines. ( #5083 )
2023-09-23 17:01:37 +01:00
Steven Liu
06b316ef5c
[docs] Improved image-to-image guide ( #5020 )
...
* finish first draft
* feedback
* feedback
2023-09-22 13:20:30 -07:00
Pedro Cuenca
3651b14cf4
SDXL flax ( #4254 )
...
* support transformer_layers_per block in flax UNet
* add support for text_time additional embeddings to Flax UNet
* rename attention layers for VAE
* add shape asserts when renaming attention layers
* transpose VAE attention layers
* add pipeline flax SDXL code [WIP]
* continue add pipeline flax SDXL code [WIP]
* cleanup
* Working on JIT support
Fixed prompt embedding shapes so they work in parallel mode. Assuming we
always have both text encoders for now, for simplicity.
* Fixing embeddings (untested)
* Remove spurious line
* Shard guidance_scale when jitting.
* Decode images
* Fix sharding
* style
* Refiner UNet can be loaded.
* Refiner / img2img pipeline
* Allow latent outputs from base and latent inputs in refiner
This makes it possible to chain base + refiner without having to use the
vae decoder in the base model, the vae encoder in the refiner, skipping
conversions to/from PIL, and avoiding TPU <-> CPU memory copies.
* Adapt to FlaxCLIPTextModelOutput
* Update Flax XL pipeline to FlaxCLIPTextModelOutput
* make fix-copies
* make style
* add euler scheduler
* Fix import
* Fix copies, comment unused code.
* Fix SDXL Flax imports
* Fix euler discrete begin
* improve init import
* finish
* put discrete euler in init
* fix flax euler
* Fix more
* make style
* correct init
* correct init
* Temporarily remove FlaxStableDiffusionXLImg2ImgPipeline
* correct pipelines
* finish
---------
Co-authored-by: Martin Müller <martin.muller.me@gmail.com >
Co-authored-by: patil-suraj <surajp815@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-22 18:34:04 +02:00
Pedro Cuenca
2e860e89eb
SDXL: update links to refine docs ( #5101 )
...
* SDXL: update links to refine docs
* make style
2023-09-22 13:17:17 +02:00
Younes Belkada
493f9529d7
[PEFT / LoRA] PEFT integration - text encoder ( #5058 )
...
* more fixes
* up
* up
* style
* add in setup
* oops
* more changes
* v1 rzfactor CI
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* few todos
* protect torch import
* style
* fix fuse text encoder
* Update src/diffusers/loaders.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* replace with `recurse_replace_peft_layers`
* keep old modules for BC
* adjustments on `adjust_lora_scale_text_encoder`
* nit
* move tests
* add conversion utils
* remove unneeded methods
* use class method instead
* oops
* use `base_version`
* fix examples
* fix CI
* fix weird error with python 3.8
* fix
* better fix
* style
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* add comment
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* conv2d support for recurse remove
* added docstrings
* more docstring
* add deprecate
* revert
* try to fix merge conflicts
* v1 tests
* add new decorator
* add saving utilities test
* adapt tests a bit
* add save / from_pretrained tests
* add saving tests
* add scale tests
* fix deps tests
* fix lora CI
* fix tests
* add comment
* fix
* style
* add slow tests
* slow tests pass
* style
* Update src/diffusers/utils/import_utils.py
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com >
* Apply suggestions from code review
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com >
* circumvents pattern finding issue
* left a todo
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* update hub path
* add lora workflow
* fix
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com >
2023-09-22 13:03:39 +02:00
hysts
b32555a2da
[docs] Add missing parenthesis in the sample code of BLIP Diffusion ( #5144 )
...
Add missing parenthesis in the sample code of BLIP Diffusion
2023-09-22 09:38:17 +01:00
YiYi Xu
80c00e5451
add use_karras_sigmas to KDPM2DiscreteScheduler and KDPM2AncestralDiscreteScheduler ( #5111 )
...
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-21 13:50:41 -10:00
YiYi Xu
2badddfdb6
add multi adapter support to StableDiffusionXLAdapterPipeline ( #5127 )
...
fix and add tests
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-21 12:54:59 -10:00
Bagheera
d558811b26
Min-SNR gamma support for Dreambooth training ( #5107 )
...
* min-SNR gamma for Dreambooth training
* Align the mse_loss_weights style with SDXL training example
---------
Co-authored-by: bghira <bghira@users.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-21 22:53:06 +01:00
Ayush Mangal
157c9011d8
Add BLIP Diffusion ( #4388 )
...
* Add BLIP Diffusion skeleton
* Add other model components
* Add BLIP2, need to change it for now
* Fix pipeline imports
* Load pretrained ViT
* Make qformer fwd pass same
* Replicate fwd passes
* Fix device bug
* Add accelerate functions
* Remove extra functions from Blip2
* Minor bug
* Integrate initial review changes
* Refactoring
* Refactoring
* Refactor
* Add controlnet
* Refactor
* Update conversion script
* Add image processor
* Shift postprocessing to ImageProcessor
* Refactor
* Fix device
* Add fast tests
* Update conversion script
* Fix checkpoint conversion script
* Integrate review changes
* Integrate reivew changes
* Remove unused functions from test
* Reuse HF image processor in Cond image
* Create new BlipImageProcessor based on transfomers
* Fix image preprocessor
* Minor
* Minor
* Add canny preprocessing
* Fix controlnet preprocessing
* Fix blip diffusion test
* Add controlnet test
* Add initial doc strings
* Integrate review changes
* Refactor
* Update examples
* Remove DDIM comments
* Add copied from for prepare_latents
* Add type anotations
* Add docstrings
* Do black formatting
* Add batch support
* Make tests pass
* Make controlnet tests pass
* Black formatting
* Fix progress bar
* Fix some licensing comments
* Fix imports
* Refactor controlnet
* Make tests faster
* Edit examples
* Black formatting/Ruff
* Add doc
* Minor
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Move controlnet pipeline
* Make tests faster
* Fix imports
* Fix formatting
* Fix make errors
* Fix make errors
* Minor
* Add suggested doc changes
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Edit docs
* Fix 16 bit loading
* Update examples
* Edit toctree
* Update docs/source/en/api/pipelines/blip_diffusion.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Minor
* Add tips
* Edit examples
* Update model paths
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-21 17:05:35 +01:00
Bagheera
24563ca654
SNR gamma fixes for v_prediction training ( #5106 )
...
Co-authored-by: bghira <bghira@users.github.com >
2023-09-20 21:18:56 +01:00
Younes Belkada
914586f5b6
[core] Use python 3.8 in workflow and setup file ( #5122 )
...
* use python 3.7 instead
* Update setup.py
2023-09-20 20:57:06 +02:00
김태민
5b78141fd3
[FIX BUG] add config_files parser #5114 ( #5115 )
...
* add config_files parser #5114
* add config_files parser_fix #5114
2023-09-20 16:17:47 +02:00
Sayak Paul
e312b2302b
[LoRA] support LyCORIS ( #5102 )
...
* better condition.
* debugging
* how about now?
* how about now?
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* support for lycoris.
* style
* add: lycoris test
* fix from_pretrained call.
* fix assertion values.
2023-09-20 10:30:18 +01:00
YiYi Xu
8263cf00f8
refactor DPMSolverMultistepScheduler using sigmas ( #4986 )
...
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-19 11:21:49 -10:00
Bagheera
74e43a4fbd
Resolve v_prediction issue for min-SNR gamma weighted loss function ( #5096 )
...
* Resolve v_prediction issue for min-SNR gamma weighted loss function
* Combine MSE loss calculation of epsilon and velocity, with a note about the application of the epsilon code to sample prediction
* style
---------
Co-authored-by: bghira <bghira@users.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-19 17:31:27 +01:00
Bagheera
81331f3b7d
Add x-prediction / prediction_type=sample support for SDXL fine-tuning ( #5095 )
...
Co-authored-by: bghira <bghira@users.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-19 16:57:44 +01:00
Dhruv Nair
29970757de
Fast Tests on PR improvements: Batch Tests fixes ( #5080 )
...
* fix test
* initial commit
* change test
* updates:
* fix tests
* test fix
* test fix
* fix tests
* make test faster
* clean up
* fix precision in test
* fix precision
* Fix tests
* Fix logging test
* fix test
* fix test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-19 18:31:21 +05:30
Dhruv Nair
c2787c11c2
Fixes for Float16 inference Fast CUDA Tests ( #5097 )
...
* wip
* fix tests
2023-09-19 17:25:48 +05:30
Dhruv Nair
79a3f39eb5
Move to slow tests to nightly ( #5093 )
...
* move slow tests to nightly
* move slow tests to nightly
2023-09-19 16:04:26 +05:30
Dhruv Nair
431dd2f4d6
Fix precision related issues in Kandinsky Pipelines ( #5098 )
...
* fix failing tests
* make style
2023-09-19 16:02:21 +05:30
Sayak Paul
edcbb6f42e
[WIP] core: add support for clip skip to SDXL ( #5057 )
...
* core: add support for clip ckip to SDXL
* add clip_skip support to the rest of the pipeline.
* Empty-Commit
2023-09-19 10:51:36 +01:00
Patrick von Platen
5a287d3f23
[SDXL] Make sure multi batch prompt embeds works ( #5073 )
...
* [SDXL] Make sure multi batch prompt embeds works
* [SDXL] Make sure multi batch prompt embeds works
* improve more
* improve more
* Apply suggestions from code review
2023-09-19 11:49:49 +02:00
maksymbekuzarovSC
65c162a5b3
Fixed get_word_inds mistake/typo in P2P community pipeline (breaking code examples) ( #5089 )
...
Fixed `get_word_inds` mistake/typo in P2P community pipeline
The function `get_word_inds` was taking a string of text and either a word (str) or a word index (int) and returned the indices of token(s) the word would be encoded to.
However, there was a typo, in which in the second `if` branch the word was checked to be a `str` **again**, not `int`, which resulted in an [example code from the docs](https://github.com/huggingface/diffusers/tree/main/examples/community#prompt2prompt-pipeline ) to result in an error
2023-09-19 11:34:49 +02:00
Sayak Paul
04d696d650
[Core] Add support for CLIP-skip ( #4901 )
...
* add support for clip skip
* fix condition
* fix
* add clip_output_layer_to_default
* expose
* remove the previous functions.
* correct condition.
* apply final layer norm
* address feedback
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* refactor clip_skip.
* port to the other pipelines.
* fix copies one more time
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-19 10:30:53 +01:00
Sayak Paul
ed507680e3
[LoRA] don't break offloading for incompatible lora ckpts. ( #5085 )
...
* don't break offloading for incompatible lora ckpts.
* debugging
* better condition.
* fix
* fix
* fix
* fix
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-18 23:46:28 +02:00
Will Berman
7974fad13b
remove unused adapter weights in constructor ( #5088 )
...
remove adapter weights in MultiAdapter constructor
2023-09-18 22:36:55 +02:00
Will Berman
6d7279adad
t2i Adapter community member fix ( #5090 )
...
* convert tensorrt controlnet
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix number controlnet condition
* Add convert SD XL to onnx
* Add convert SD XL to tensorrt
* Add convert SD XL to tensorrt
* Add examples in comments
* Add examples in comments
* Add test onnx controlnet
* Add tensorrt test
* Remove copied
* Move file test to examples/community
* Remove script
* Remove script
* Remove text
* Fix import
* Fix T2I MultiAdapter
* fix tests
---------
Co-authored-by: dotieuthien <thien.do@mservice.com.vn >
Co-authored-by: dotieuthien <dotieuthien9997@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: dotieuthien <hades@cinnamon.is >
2023-09-18 22:35:49 +02:00
Patrick von Platen
119ad2c3dc
[LoRA] Centralize LoRA tests ( #5086 )
...
* [LoRA] Centralize LoRA tests
* [LoRA] Centralize LoRA tests
* [LoRA] Centralize LoRA tests
* [LoRA] Centralize LoRA tests
* [LoRA] Centralize LoRA tests
2023-09-18 17:54:33 +02:00
Ruoxi
16b9a57d29
Implement CustomDiffusionAttnProcessor2_0. ( #4604 )
...
* Implement `CustomDiffusionAttnProcessor2_0`
* Doc-strings and type annotations for `CustomDiffusionAttnProcessor2_0`. (#1 )
* Update attnprocessor.md
* Update attention_processor.py
* Interops for `CustomDiffusionAttnProcessor2_0`.
* Formatted `attention_processor.py`.
* Formatted doc-string in `attention_processor.py`
* Conditional CustomDiffusion2_0 for training example.
* Remove unnecessary reference impl in comments.
* Fix `save_attn_procs`.
2023-09-18 14:49:00 +02:00
Patrick von Platen
7b39f43c06
[Textual inversion] Refactor textual inversion to make it cleaner ( #5076 )
...
* [Textual inversion] Clean loading
* [Textual inversion] Clean loading
* [Textual inversion] Clean up
* [Textual inversion] Clean up
* [Textual inversion] Clean up
* [Textual inversion] Clean up
2023-09-18 14:30:34 +02:00
Sayak Paul
bfc606301f
add doc around fusing multiple loras. ( #5056 )
...
* add doc around fusing multiple loras.
* Apply suggestions from code review
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
* address poli's comments.
---------
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
2023-09-18 12:42:58 +01:00
YiYi Xu
6886e28fd8
fix a bug in inpaint pipeline when use regular text2image unet ( #5033 )
...
* fix
* fix num_images_per_prompt >1
* other pipelines
* add fast tests for inpaint pipelines
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-18 13:40:11 +02:00
Lee Dong Joo
b089102a8e
fix guidance_rescale docstring ( #5063 )
2023-09-18 13:39:12 +02:00
Kashif Rasul
73bb97adfc
[LoRA] fix typo in attention_processor.py ( #5066 )
...
* [LoRA] fix typo in attention_processor.py
fixes #5062
* make style
* make fix-copies, logger comented for torch compile
2023-09-16 14:43:18 +02:00
Sayak Paul
38a664a3d6
fix: validation_image arg ( #5053 )
2023-09-15 12:20:50 +01:00
Kashif Rasul
427feb5359
[Wuerstchen] fix typos in docs ( #5051 )
...
* fix typos in docs
* fix for issue #5023
2023-09-15 12:53:25 +02:00
Gang Wu
9f40d7970e
[FIX BUG] type of args in train_instruct_pix2pix_sdxl.py ( #4955 )
2023-09-15 12:53:07 +02:00
Bagheera
a0198676d7
Remove logger.info statement from Unet2DCondition code to ensure torch compile reliably succeeds ( #4982 )
...
* Remove logger.info statement from Unet2DCondition code to ensure torch compile reliably succeeds
* Convert logging statement to a comment for future archaeologists
* Update src/diffusers/models/unet_2d_condition.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: bghira <bghira@users.github.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-15 12:52:46 +02:00
Patrick von Platen
abc47dece6
[SDXL, Docs] Textual inversion ( #5039 )
...
* [SDXL, Docs] Textual inversion
* Update docs/source/en/using-diffusers/sdxl.md
* finish
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-09-15 12:51:36 +02:00
