Sayak Paul
1d1e1a2888
Some minor updates to the nightly and push workflows ( #9759 )
...
* move lora integration tests to nightly./
* remove slow marker in the workflow where not needed.
2024-10-24 23:49:09 +09:00
Dhruv Nair
76c00c7236
is_safetensors_compatible fix ( #9741 )
...
update
2024-10-22 19:35:03 +05:30
Sayak Paul
60ffa84253
[bitsandbbytes] follow-ups ( #9730 )
...
* bnb follow ups.
* add a warning when dtypes mismatch.
* fx-copies
* clear cache.
* check_if_quantized_param
* add a check on shape.
* updates
* docs
* improve readability.
* resources.
* fix
2024-10-22 16:00:05 +05:30
YiYi Xu
e2d037bbf1
minor doc/test update ( #9734 )
...
* update some docs and tests!
---------
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
2024-10-21 13:06:13 -10:00
Sayak Paul
b821f006d0
[Quantization] Add quantization support for bitsandbytes ( #9213 )
...
* quantization config.
* fix-copies
* fix
* modules_to_not_convert
* add bitsandbytes utilities.
* make progress.
* fixes
* quality
* up
* up
rotary embedding refactor 2: update comments, fix dtype for use_real=False (#9312 )
fix notes and dtype
up
up
* minor
* up
* up
* fix
* provide credits where due.
* make configurations work.
* fixes
* fix
* update_missing_keys
* fix
* fix
* make it work.
* fix
* provide credits to transformers.
* empty commit
* handle to() better.
* tests
* change to bnb from bitsandbytes
* fix tests
fix slow quality tests
SD3 remark
fix
complete int4 tests
add a readme to the test files.
add model cpu offload tests
warning test
* better safeguard.
* change merging status
* courtesy to transformers.
* move upper.
* better
* make the unused kwargs warning friendlier.
* harmonize changes with https://github.com/huggingface/transformers/pull/33122
* style
* trainin tests
* feedback part i.
* Add Flux inpainting and Flux Img2Img (#9135 )
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Update `UNet2DConditionModel`'s error messages (#9230 )
* refactor
[CI] Update Single file Nightly Tests (#9357 )
* update
* update
feedback.
improve README for flux dreambooth lora (#9290 )
* improve readme
* improve readme
* improve readme
* improve readme
fix one uncaught deprecation warning for accessing vae_latent_channels in VaeImagePreprocessor (#9372 )
deprecation warning vae_latent_channels
add mixed int8 tests and more tests to nf4.
[core] Freenoise memory improvements (#9262 )
* update
* implement prompt interpolation
* make style
* resnet memory optimizations
* more memory optimizations; todo: refactor
* update
* update animatediff controlnet with latest changes
* refactor chunked inference changes
* remove print statements
* update
* chunk -> split
* remove changes from incorrect conflict resolution
* remove changes from incorrect conflict resolution
* add explanation of SplitInferenceModule
* update docs
* Revert "update docs"
This reverts commit c55a50a271#9135 )"
This reverts commit 5799954dd459462357+stevhliu@users.noreply.github.com >
* contribution guide.
* changes
* empty
* fix tests
* harmonize with https://github.com/huggingface/transformers/pull/33546 .
* numpy_cosine_distance
* config_dict modification.
* remove if config comment.
* note for load_state_dict changes.
* float8 check.
* quantizer.
* raise an error for non-True low_cpu_mem_usage values when using quant.
* low_cpu_mem_usage shenanigans when using fp32 modules.
* don't re-assign _pre_quantization_type.
* make comments clear.
* remove comments.
* handle mixed types better when moving to cpu.
* add tests to check if we're throwing warning rightly.
* better check.
* fix 8bit test_quality.
* handle dtype more robustly.
* better message when keep_in_fp32_modules.
* handle dtype casting.
* fix dtype checks in pipeline.
* fix warning message.
* Update src/diffusers/models/modeling_utils.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* mitigate the confusing cpu warning
---------
Co-authored-by: Vishnu V Jaddipal <95531133+Gothos@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-21 10:11:57 +05:30
bonlime
5d3e7bdaaa
Fix bug in Textual Inversion Unloading ( #9304 )
...
* Update textual_inversion.py
* add unload test
* add comment
* fix style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Your Name <you@example.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-19 02:37:32 -10:00
Aryan
5704376d03
[refactor] DiffusionPipeline.download ( #9557 )
...
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-10-17 12:38:06 -10:00
Aryan
d9029f2c59
[tests] fix name and unskip CogI2V integration test ( #9683 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-16 16:28:19 +05:30
Aryan
8cabd4a0db
[pipeline] CogVideoX-Fun Control ( #9671 )
...
* cogvideox-fun control
* make style
* make fix-copies
* karras schedulers
* Update src/diffusers/pipelines/cogvideo/pipeline_cogvideox_fun_control.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/api/pipelines/cogvideox.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* apply suggestions from review
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-16 16:21:09 +05:30
Sayak Paul
cef4f65cf7
[LoRA] log a warning when there are missing keys in the LoRA loading. ( #9622 )
...
* log a warning when there are missing keys in the LoRA loading.
* handle missing keys and unexpected keys better.
* add tests
* fix-copies.
* updates
* tests
* concat warning.
* Add Differential Diffusion to Kolors (#9423 )
* Added diff diff support for kolors img2img
* Fized relative imports
* Fized relative imports
* Added diff diff support for Kolors
* Fized import issues
* Added map
* Fized import issues
* Fixed naming issues
* Added diffdiff support for Kolors img2img pipeline
* Removed example docstrings
* Added map input
* Updated latents
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* Updated `original_with_noise`
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* Improved code quality
---------
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* FluxMultiControlNetModel (#9647 )
* tests
* Update src/diffusers/loaders/lora_pipeline.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* fix
---------
Co-authored-by: M Saqlain <118016760+saqlain2204@users.noreply.github.com >
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
Co-authored-by: hlky <hlky@hlky.ac >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-16 07:46:12 +05:30
hlky
9d0616189e
Slight performance improvement to Euler, EDMEuler, FlowMatchHeun, KDPM2Ancestral ( #9616 )
...
* Slight performance improvement to Euler
* Slight performance improvement to EDMEuler
* Slight performance improvement to FlowMatchHeun
* Slight performance improvement to KDPM2Ancestral
* Update KDPM2AncestralDiscreteSchedulerTest
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-14 19:34:25 -10:00
SahilCarterr
22ed39f571
Added Lora Support to SD3 Img2Img Pipeline ( #9659 )
...
* add lora
2024-10-14 11:39:20 -10:00
Yuxuan.Zhang
8d81564b27
CogView3Plus DiT ( #9570 )
...
* merge 9588
* max_shard_size="5GB" for colab running
* conversion script updates; modeling test; refactor transformer
* make fix-copies
* Update convert_cogview3_to_diffusers.py
* initial pipeline draft
* make style
* fight bugs 🐛 🪳
* add example
* add tests; refactor
* make style
* make fix-copies
* add co-author
YiYi Xu <yixu310@gmail.com >
* remove files
* add docs
* add co-author
Co-Authored-By: YiYi Xu <yixu310@gmail.com >
* fight docs
* address reviews
* make style
* make model work
* remove qkv fusion
* remove qkv fusion tets
* address review comments
* fix make fix-copies error
* remove None and TODO
* for FP16(draft)
* make style
* remove dynamic cfg
* remove pooled_projection_dim as a parameter
* fix tests
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-14 19:30:36 +05:30
Sayak Paul
86bcbc389e
[Tests] increase transformers version in test_low_cpu_mem_usage_with_loading ( #9662 )
...
increase transformers version in test_low_cpu_mem_usage_with_loading
2024-10-13 22:39:38 +05:30
Sayak Paul
e16fd93d0a
[LoRA] fix dora test to catch the warning properly. ( #9627 )
...
fix dora test.
2024-10-10 11:47:49 +05:30
SahilCarterr
af28ae2d5b
add PAG support for SD Img2Img ( #9463 )
...
* added pag to sd img2img pipeline
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-09 10:40:58 -10:00
Sayak Paul
31058cdaef
[LoRA] allow loras to be loaded with low_cpu_mem_usage. ( #9510 )
...
* allow loras to be loaded with low_cpu_mem_usage.
* add flux support but note https://github.com/huggingface/diffusers/pull/9510\#issuecomment-2378316687
* low_cpu_mem_usage.
* fix-copies
* fix-copies again
* tests
* _LOW_CPU_MEM_USAGE_DEFAULT_LORA
* _peft_version default.
* version checks.
* version check.
* version check.
* version check.
* require peft 0.13.1.
* explicitly specify low_cpu_mem_usage=False.
* docs.
* transformers version 4.45.2.
* update
* fix
* empty
* better name initialize_dummy_state_dict.
* doc todos.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* style
* fix-copies
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-10-09 10:57:16 +05:30
Sayak Paul
02eeb8e77e
[LoRA] Handle DoRA better ( #9547 )
...
* handle dora.
* print test
* debug
* fix
* fix-copies
* update logits
* add warning in the test.
* make is_dora check consistent.
* fix-copies
2024-10-08 21:47:44 +05:30
Sayak Paul
31010ecc45
[Chore] add a note on the versions in Flux LoRA integration tests ( #9598 )
...
add a note on the versions.
2024-10-07 17:43:48 +05:30
Darren Hsu
61d37640ad
Support bfloat16 for Upsample2D ( #9480 )
...
* Support bfloat16 for Upsample2D
* Add test and use is_torch_version
* Resolve comments and add decorator
* Simplify require_torch_version_greater_equal decorator
* Run make style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-01 16:08:12 -10:00
JuanCarlosPi
33fafe3d14
Add PAG support to StableDiffusionControlNetPAGInpaintPipeline ( #8875 )
...
* Add pag to controlnet inpainting pipeline
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-30 20:04:42 -10:00
Sayak Paul
f9fd511466
[LoRA] support Kohya Flux LoRAs that have text encoders as well ( #9542 )
...
* support kohya flux loras that have tes.
2024-09-30 07:59:39 -10:00
Sayak Paul
11542431a5
[Core] fix variant-identification. ( #9253 )
...
* fix variant-idenitification.
* fix variant
* fix sharded variant checkpoint loading.
* Apply suggestions from code review
* fixes.
* more fixes.
* remove print.
* fixes
* fixes
* comments
* fixes
* apply suggestions.
* hub_utils.py
* fix test
* updates
* fixes
* fixes
* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* updates.
* removep patch file.
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-28 09:57:31 +05:30
Sayak Paul
81cf3b2f15
[Tests] [LoRA] clean up the serialization stuff. ( #9512 )
...
* clean up the serialization stuff.
* better
2024-09-27 07:57:09 -10:00
Sayak Paul
2daedc0ad3
[LoRA] make set_adapters() method more robust. ( #9535 )
...
* make set_adapters() method more robust.
* remove patch
* better and concise code.
* Update src/diffusers/loaders/lora_base.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-27 07:32:43 +05:30
YiYi Xu
bac8a2412d
a few fix for SingleFile tests ( #9522 )
...
* update sd15 repo
* update more
2024-09-24 13:36:53 -10:00
M Saqlain
14f6464bef
[Tests] Reduce the model size in the lumina test ( #8985 )
...
* Reduced model size for lumina-tests
* Handled failing tests
2024-09-23 20:35:50 +05:30
Sayak Paul
aa73072f1f
[CI] fix nightly model tests ( #9483 )
...
* check if default attn procs fix it.
* print
* print
* replace
* style./
* replace revision with variant.
* replace with stable-diffusion-v1-5/stable-diffusion-inpainting.
* replace with stable-diffusion-v1-5/stable-diffusion-v1-5.
* fix
2024-09-21 07:44:47 +05:30
Aryan
e5d0a328d6
[refactor] LoRA tests ( #9481 )
...
* refactor scheduler class usage
* reorder to make tests more readable
* remove pipeline specific checks and skip tests directly
* rewrite denoiser conditions cleaner
* bump tolerance for cog test
2024-09-21 07:10:36 +05:30
Aryan
2b443a5d62
[training] CogVideoX Lora ( #9302 )
...
* cogvideox lora training draft
* update
* update
* update
* update
* update
* make fix-copies
* update
* update
* apply suggestions from review
* apply suggestions from reveiw
* fix typo
* Update examples/cogvideo/train_cogvideox_lora.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* fix lora alpha
* use correct lora scaling for final test pipeline
* Update examples/cogvideo/train_cogvideox_lora.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* apply suggestions from review; prodigy optimizer
YiYi Xu <yixu310@gmail.com >
* add tests
* make style
* add README
* update
* update
* make style
* fix
* update
* add test skeleton
* revert lora utils changes
* add cleaner modifications to lora testing utils
* update lora tests
* deepspeed stuff
* add requirements.txt
* deepspeed refactor
* add lora stuff to img2vid pipeline to fix tests
* fight tests
* add co-authors
Co-Authored-By: Fu-Yun Wang <1697256461@qq.com >
Co-Authored-By: zR <2448370773@qq.com >
* fight lora runner tests
* import Dummy optim and scheduler only wheh required
* update docs
* add coauthors
Co-Authored-By: Fu-Yun Wang <1697256461@qq.com >
* remove option to train text encoder
Co-Authored-By: bghira <bghira@users.github.com >
* update tests
* fight more tests
* update
* fix vid2vid
* fix typo
* remove lora tests; todo in follow-up PR
* undo img2vid changes
* remove text encoder related changes in lora loader mixin
* Revert "remove text encoder related changes in lora loader mixin"
This reverts commit f8a8444487yixu310@gmail.com >
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: zR <2448370773@qq.com >
Co-authored-by: Fu-Yun Wang <1697256461@qq.com >
Co-authored-by: bghira <bghira@users.github.com >
2024-09-19 14:37:57 +05:30
Sayak Paul
d13b0d63c0
[Flux] add lora integration tests. ( #9353 )
...
* add lora integration tests.
* internal note
* add a skip marker.
2024-09-19 09:21:28 +05:30
Aryan
ba06124e4a
Remove CogVideoX mentions from single file docs; Test updates ( #9444 )
...
* remove mentions from single file
* update tests
* update
2024-09-17 10:05:45 -10:00
Subho Ghosh
bb1b0fa1f9
Feature flux controlnet img2img and inpaint pipeline ( #9408 )
...
* Implemented FLUX controlnet support to Img2Img pipeline
2024-09-17 09:43:54 -10:00
Yuxuan.Zhang
8336405e50
CogVideoX-5b-I2V support ( #9418 )
...
* draft Init
* draft
* vae encode image
* make style
* image latents preparation
* remove image encoder from conversion script
* fix minor bugs
* make pipeline work
* make style
* remove debug prints
* fix imports
* update example
* make fix-copies
* add fast tests
* fix import
* update vae
* update docs
* update image link
* apply suggestions from review
* apply suggestions from review
* add slow test
* make use of learned positional embeddings
* apply suggestions from review
* doc change
* Update convert_cogvideox_to_diffusers.py
* make style
* final changes
* make style
* fix tests
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-09-16 14:46:24 +05:30
Dhruv Nair
1e8cf2763d
[CI] Nightly Test Updates ( #9380 )
...
* update
* update
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-12 20:21:28 +05:30
Sayak Paul
adf1f911f0
[Tests] fix some fast gpu tests. ( #9379 )
...
fix some fast gpu tests.
2024-09-11 06:50:02 +05:30
Igor Filippov
a7361dccdc
[Pipeline] animatediff + vid2vid + controlnet ( #9337 )
...
* add animatediff + vid2vide + controlnet
* post tests fixes
* PR discussion fixes
* update docs
* change input video to links on HF + update an example
* make quality fix
* fix ip adapter test
* fix ip adapter test input
* update ip adapter test
2024-09-09 22:48:21 +05:30
YiYi Xu
8cdcdd9e32
add flux inpaint + img2img + controlnet to auto pipeline ( #9367 )
2024-09-06 07:14:48 -10:00
Dhruv Nair
d269cc8a4e
[CI] Quick fix for Cog Video Test ( #9373 )
...
update
2024-09-06 15:25:53 +05:30
Aryan
6dfa49963c
[core] Freenoise memory improvements ( #9262 )
...
* update
* implement prompt interpolation
* make style
* resnet memory optimizations
* more memory optimizations; todo: refactor
* update
* update animatediff controlnet with latest changes
* refactor chunked inference changes
* remove print statements
* update
* chunk -> split
* remove changes from incorrect conflict resolution
* remove changes from incorrect conflict resolution
* add explanation of SplitInferenceModule
* update docs
* Revert "update docs"
This reverts commit c55a50a271
2024-09-06 12:51:20 +05:30
Dhruv Nair
53051cf282
[CI] Update Single file Nightly Tests ( #9357 )
...
* update
* update
2024-09-05 14:33:44 +05:30
Vishnu V Jaddipal
249a9e48e8
Add Flux inpainting and Flux Img2Img ( #9135 )
...
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-09-04 10:31:43 -10:00
Fanli Lin
2ee3215949
[tests] make 2 tests device-agnostic ( #9347 )
...
* enabel on xpu
* fix style
2024-09-03 16:34:03 -10:00
Aryan
24053832b5
[tests] remove/speedup some low signal tests ( #9285 )
...
* remove 2 shapes from SDFunctionTesterMixin::test_vae_tiling
* combine freeu enable/disable test to reduce many inference runs
* remove low signal unet test for signature
* remove low signal embeddings test
* remove low signal progress bar test from PipelineTesterMixin
* combine ip-adapter single and multi tests to save many inferences
* fix broken tests
* Update tests/pipelines/test_pipelines_common.py
* Update tests/pipelines/test_pipelines_common.py
* add progress bar tests
2024-09-03 13:59:18 +05:30
Dhruv Nair
f6f16a0c11
[CI] More Fast GPU Test Fixes ( #9346 )
...
* update
* update
* update
* update
2024-09-03 13:22:38 +05:30
Dhruv Nair
007ad0e2aa
[CI] More fixes for Fast GPU Tests on main ( #9300 )
...
update
2024-09-02 17:51:48 +05:30
Aryan
0e6a8403f6
[core] Support VideoToVideo with CogVideoX ( #9333 )
...
* add vid2vid pipeline for cogvideox
* make fix-copies
* update docs
* fake context parallel cache, vae encode tiling
* add test for cog vid2vid
* use video link from HF docs repo
* add copied from comments; correctly rename test class
2024-09-02 16:54:58 +05:30
Aryan
cbc2ec8f44
AnimateDiff prompt travel ( #9231 )
...
* update
* implement prompt interpolation
* make style
* resnet memory optimizations
* more memory optimizations; todo: refactor
* update
* update animatediff controlnet with latest changes
* refactor chunked inference changes
* remove print statements
* undo memory optimization changes
* update docstrings
* fix tests
* fix pia tests
* apply suggestions from review
* add tests
* update comment
2024-08-28 14:48:12 +05:30
Sayak Paul
2d9ccf39b5
[Core] fuse_qkv_projection() to Flux ( #9185 )
...
* start fusing flux.
* test
* finish fusion
* fix-copues
2024-08-23 10:54:13 +05:30
zR
960c149c77
Cogvideox-5B Model adapter change ( #9203 )
...
* draft of embedding
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-22 16:03:29 -10:00
