Sayak Paul
f9fd511466
[LoRA] support Kohya Flux LoRAs that have text encoders as well ( #9542 )
...
* support kohya flux loras that have tes.
2024-09-30 07:59:39 -10:00
Sayak Paul
11542431a5
[Core] fix variant-identification. ( #9253 )
...
* fix variant-idenitification.
* fix variant
* fix sharded variant checkpoint loading.
* Apply suggestions from code review
* fixes.
* more fixes.
* remove print.
* fixes
* fixes
* comments
* fixes
* apply suggestions.
* hub_utils.py
* fix test
* updates
* fixes
* fixes
* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* updates.
* removep patch file.
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-28 09:57:31 +05:30
Sayak Paul
81cf3b2f15
[Tests] [LoRA] clean up the serialization stuff. ( #9512 )
...
* clean up the serialization stuff.
* better
2024-09-27 07:57:09 -10:00
Sayak Paul
2daedc0ad3
[LoRA] make set_adapters() method more robust. ( #9535 )
...
* make set_adapters() method more robust.
* remove patch
* better and concise code.
* Update src/diffusers/loaders/lora_base.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-27 07:32:43 +05:30
YiYi Xu
bac8a2412d
a few fix for SingleFile tests ( #9522 )
...
* update sd15 repo
* update more
2024-09-24 13:36:53 -10:00
M Saqlain
14f6464bef
[Tests] Reduce the model size in the lumina test ( #8985 )
...
* Reduced model size for lumina-tests
* Handled failing tests
2024-09-23 20:35:50 +05:30
Sayak Paul
aa73072f1f
[CI] fix nightly model tests ( #9483 )
...
* check if default attn procs fix it.
* print
* print
* replace
* style./
* replace revision with variant.
* replace with stable-diffusion-v1-5/stable-diffusion-inpainting.
* replace with stable-diffusion-v1-5/stable-diffusion-v1-5.
* fix
2024-09-21 07:44:47 +05:30
Aryan
e5d0a328d6
[refactor] LoRA tests ( #9481 )
...
* refactor scheduler class usage
* reorder to make tests more readable
* remove pipeline specific checks and skip tests directly
* rewrite denoiser conditions cleaner
* bump tolerance for cog test
2024-09-21 07:10:36 +05:30
Aryan
2b443a5d62
[training] CogVideoX Lora ( #9302 )
...
* cogvideox lora training draft
* update
* update
* update
* update
* update
* make fix-copies
* update
* update
* apply suggestions from review
* apply suggestions from reveiw
* fix typo
* Update examples/cogvideo/train_cogvideox_lora.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* fix lora alpha
* use correct lora scaling for final test pipeline
* Update examples/cogvideo/train_cogvideox_lora.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* apply suggestions from review; prodigy optimizer
YiYi Xu <yixu310@gmail.com >
* add tests
* make style
* add README
* update
* update
* make style
* fix
* update
* add test skeleton
* revert lora utils changes
* add cleaner modifications to lora testing utils
* update lora tests
* deepspeed stuff
* add requirements.txt
* deepspeed refactor
* add lora stuff to img2vid pipeline to fix tests
* fight tests
* add co-authors
Co-Authored-By: Fu-Yun Wang <1697256461@qq.com >
Co-Authored-By: zR <2448370773@qq.com >
* fight lora runner tests
* import Dummy optim and scheduler only wheh required
* update docs
* add coauthors
Co-Authored-By: Fu-Yun Wang <1697256461@qq.com >
* remove option to train text encoder
Co-Authored-By: bghira <bghira@users.github.com >
* update tests
* fight more tests
* update
* fix vid2vid
* fix typo
* remove lora tests; todo in follow-up PR
* undo img2vid changes
* remove text encoder related changes in lora loader mixin
* Revert "remove text encoder related changes in lora loader mixin"
This reverts commit f8a8444487yixu310@gmail.com >
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: zR <2448370773@qq.com >
Co-authored-by: Fu-Yun Wang <1697256461@qq.com >
Co-authored-by: bghira <bghira@users.github.com >
2024-09-19 14:37:57 +05:30
Sayak Paul
d13b0d63c0
[Flux] add lora integration tests. ( #9353 )
...
* add lora integration tests.
* internal note
* add a skip marker.
2024-09-19 09:21:28 +05:30
Aryan
ba06124e4a
Remove CogVideoX mentions from single file docs; Test updates ( #9444 )
...
* remove mentions from single file
* update tests
* update
2024-09-17 10:05:45 -10:00
Subho Ghosh
bb1b0fa1f9
Feature flux controlnet img2img and inpaint pipeline ( #9408 )
...
* Implemented FLUX controlnet support to Img2Img pipeline
2024-09-17 09:43:54 -10:00
Yuxuan.Zhang
8336405e50
CogVideoX-5b-I2V support ( #9418 )
...
* draft Init
* draft
* vae encode image
* make style
* image latents preparation
* remove image encoder from conversion script
* fix minor bugs
* make pipeline work
* make style
* remove debug prints
* fix imports
* update example
* make fix-copies
* add fast tests
* fix import
* update vae
* update docs
* update image link
* apply suggestions from review
* apply suggestions from review
* add slow test
* make use of learned positional embeddings
* apply suggestions from review
* doc change
* Update convert_cogvideox_to_diffusers.py
* make style
* final changes
* make style
* fix tests
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-09-16 14:46:24 +05:30
Dhruv Nair
1e8cf2763d
[CI] Nightly Test Updates ( #9380 )
...
* update
* update
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-12 20:21:28 +05:30
Sayak Paul
adf1f911f0
[Tests] fix some fast gpu tests. ( #9379 )
...
fix some fast gpu tests.
2024-09-11 06:50:02 +05:30
Igor Filippov
a7361dccdc
[Pipeline] animatediff + vid2vid + controlnet ( #9337 )
...
* add animatediff + vid2vide + controlnet
* post tests fixes
* PR discussion fixes
* update docs
* change input video to links on HF + update an example
* make quality fix
* fix ip adapter test
* fix ip adapter test input
* update ip adapter test
2024-09-09 22:48:21 +05:30
YiYi Xu
8cdcdd9e32
add flux inpaint + img2img + controlnet to auto pipeline ( #9367 )
2024-09-06 07:14:48 -10:00
Dhruv Nair
d269cc8a4e
[CI] Quick fix for Cog Video Test ( #9373 )
...
update
2024-09-06 15:25:53 +05:30
Aryan
6dfa49963c
[core] Freenoise memory improvements ( #9262 )
...
* update
* implement prompt interpolation
* make style
* resnet memory optimizations
* more memory optimizations; todo: refactor
* update
* update animatediff controlnet with latest changes
* refactor chunked inference changes
* remove print statements
* update
* chunk -> split
* remove changes from incorrect conflict resolution
* remove changes from incorrect conflict resolution
* add explanation of SplitInferenceModule
* update docs
* Revert "update docs"
This reverts commit c55a50a271
2024-09-06 12:51:20 +05:30
Dhruv Nair
53051cf282
[CI] Update Single file Nightly Tests ( #9357 )
...
* update
* update
2024-09-05 14:33:44 +05:30
Vishnu V Jaddipal
249a9e48e8
Add Flux inpainting and Flux Img2Img ( #9135 )
...
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-09-04 10:31:43 -10:00
Fanli Lin
2ee3215949
[tests] make 2 tests device-agnostic ( #9347 )
...
* enabel on xpu
* fix style
2024-09-03 16:34:03 -10:00
Aryan
24053832b5
[tests] remove/speedup some low signal tests ( #9285 )
...
* remove 2 shapes from SDFunctionTesterMixin::test_vae_tiling
* combine freeu enable/disable test to reduce many inference runs
* remove low signal unet test for signature
* remove low signal embeddings test
* remove low signal progress bar test from PipelineTesterMixin
* combine ip-adapter single and multi tests to save many inferences
* fix broken tests
* Update tests/pipelines/test_pipelines_common.py
* Update tests/pipelines/test_pipelines_common.py
* add progress bar tests
2024-09-03 13:59:18 +05:30
Dhruv Nair
f6f16a0c11
[CI] More Fast GPU Test Fixes ( #9346 )
...
* update
* update
* update
* update
2024-09-03 13:22:38 +05:30
Dhruv Nair
007ad0e2aa
[CI] More fixes for Fast GPU Tests on main ( #9300 )
...
update
2024-09-02 17:51:48 +05:30
Aryan
0e6a8403f6
[core] Support VideoToVideo with CogVideoX ( #9333 )
...
* add vid2vid pipeline for cogvideox
* make fix-copies
* update docs
* fake context parallel cache, vae encode tiling
* add test for cog vid2vid
* use video link from HF docs repo
* add copied from comments; correctly rename test class
2024-09-02 16:54:58 +05:30
Aryan
cbc2ec8f44
AnimateDiff prompt travel ( #9231 )
...
* update
* implement prompt interpolation
* make style
* resnet memory optimizations
* more memory optimizations; todo: refactor
* update
* update animatediff controlnet with latest changes
* refactor chunked inference changes
* remove print statements
* undo memory optimization changes
* update docstrings
* fix tests
* fix pia tests
* apply suggestions from review
* add tests
* update comment
2024-08-28 14:48:12 +05:30
Sayak Paul
2d9ccf39b5
[Core] fuse_qkv_projection() to Flux ( #9185 )
...
* start fusing flux.
* test
* finish fusion
* fix-copues
2024-08-23 10:54:13 +05:30
zR
960c149c77
Cogvideox-5B Model adapter change ( #9203 )
...
* draft of embedding
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-22 16:03:29 -10:00
Aryan
0ec64fe9fc
[tests] fix broken xformers tests ( #9206 )
...
* fix xformers tests
* remove unnecessary modifications to cogvideox tests
* update
2024-08-22 15:17:47 +05:30
Sayak Paul
5090b09d48
[Flux LoRA] support parsing alpha from a flux lora state dict. ( #9236 )
...
* support parsing alpha from a flux lora state dict.
* conditional import.
* fix breaking changes.
* safeguard alpha.
* fix
2024-08-22 07:01:52 +05:30
YiYi Xu
c291617518
Flux followup ( #9074 )
...
* refactor rotary embeds
* adding jsmidt as co-author of this PR for https://github.com/huggingface/diffusers/pull/9133
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Joseph Smidt <josephsmidt@gmail.com >
2024-08-21 08:44:58 -10:00
satani99
9003d75f20
Add StableDiffusionXLControlNetPAGImg2ImgPipeline ( #8990 )
...
* Added pad controlnet sdxl img2img pipeline
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-08-21 07:24:22 -10:00
YiYi Xu
214372aa99
fix a regression in is_safetensors_compatible ( #9234 )
...
fix
2024-08-21 18:56:55 +05:30
Vinh H. Pham
867e0c919e
StableDiffusionLatentUpscalePipeline - positive/negative prompt embeds support ( #8947 )
...
* make latent upscaler accept prompt embeds
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-08-20 18:00:55 -10:00
Dhruv Nair
940b8e0358
[CI] Multiple Slow Test fixes. ( #9198 )
...
* update
* update
* update
* update
2024-08-19 13:31:09 +05:30
Dhruv Nair
b2add10d13
Update is_safetensors_compatible check ( #8991 )
...
* update
* update
* update
* update
* update
2024-08-19 11:35:22 +05:30
M Saqlain
ba4348d9a7
[Tests] Improve transformers model test suite coverage - Lumina ( #8987 )
...
* Added test suite for lumina
* Fixed failing tests
* Improved code quality
* Added function docstrings
* Improved formatting
2024-08-19 08:29:03 +05:30
Sayak Paul
f848febacd
feat: allow sharding for auraflow. ( #8853 )
2024-08-18 08:47:26 +05:30
Sayak Paul
39b87b14b5
feat: allow flux transformer to be sharded during inference ( #9159 )
...
* feat: support sharding for flux.
* tests
2024-08-16 10:00:51 +05:30
Aryan
a85b34e7fd
[refactor] CogVideoX followups + tiled decoding support ( #9150 )
...
* refactor context parallel cache; update torch compile time benchmark
* add tiling support
* make style
* remove num_frames % 8 == 0 requirement
* update default num_frames to original value
* add explanations + refactor
* update torch compile example
* update docs
* update
* clean up if-statements
* address review comments
* add test for vae tiling
* update docs
* update docs
* update docstrings
* add modeling test for cogvideox transformer
* make style
2024-08-14 03:53:21 +05:30
王奇勋
5ffbe14c32
[FLUX] Support ControlNet ( #9126 )
...
* cnt model
* cnt model
* cnt model
* fix Loader "Copied"
* format
* txt_ids for multiple images
* add test and format
* typo
* Update pipeline_flux_controlnet.py
* remove
* make quality
* fix copy
* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* Update src/diffusers/pipelines/flux/pipeline_flux_controlnet.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* Update src/diffusers/models/controlnet_flux.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* fix
* make copies
* test
* bs
---------
Co-authored-by: haofanwang <haofanwang.ai@gmail.com >
Co-authored-by: haofanwang <haofan@HaofandeMBP.lan >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-08-13 18:17:40 +05:30
林金鹏
cc0513091a
Support SD3 controlnet inpainting ( #9099 )
...
* add controlnet inpainting pipeline
* [SD3] add controlnet inpaint example
* update example and fix code style
* fix code style with ruff
* Update controlnet_sd3.md : add control inpaint pipeline
* Update docs/source/en/api/pipelines/controlnet_sd3.md
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
* Update docs/source/en/api/pipelines/controlnet_sd3.md
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
* Update docs/source/en/api/pipelines/controlnet_sd3.md
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
* Update src/diffusers/pipelines/controlnet_sd3/pipeline_stable_diffusion_3_controlnet_inpainting.py
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
* Update __init__.py : add sd3 control pipelines
* Update pipeline : add new param doc & check input reference.
* fix typo
* make style & make quality
* add unittest for sd3 controlnet inpaint
---------
Co-authored-by: 鹏徙 <linjinpeng.ljp@alibaba-inc.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
2024-08-13 17:30:46 +05:30
zR
2dad462d9b
Add CogVideoX text-to-video generation model ( #9082 )
...
* add CogVideoX
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-08-06 21:23:57 -10:00
Aryan
16a93f1a25
[core] FreeNoise ( #8948 )
...
* initial work draft for freenoise; needs massive cleanup
* fix freeinit bug
* add animatediff controlnet implementation
* revert attention changes
* add freenoise
* remove old helper functions
* add decode batch size param to all pipelines
* make style
* fix copied from comments
* make fix-copies
* make style
* copy animatediff controlnet implementation from #8972
* add experimental support for num_frames not perfectly fitting context length, ocntext stride
* make unet motion model lora work again based on #8995
* copy load video utils from #8972
* copied from AnimateDiff::prepare_latents
* address the case where last batch of frames does not match length of indices in prepare latents
* decode_batch_size->vae_batch_size; batch vae encode support in animatediff vid2vid
* revert sparsectrl and sdxl freenoise changes
* revert pia
* add freenoise tests
* make fix-copies
* improve docstrings
* add freenoise tests to animatediff controlnet
* update tests
* Update src/diffusers/models/unets/unet_motion_model.py
* add freenoise to animatediff pag
* address review comments
* make style
* update tests
* make fix-copies
* fix error message
* remove copied from comment
* fix imports in tests
* update
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-08-07 10:35:18 +05:30
Álvaro Somoza
39e1f7eaa4
[Kolors] Add PAG ( #8934 )
...
* txt2img pag added
* autopipe added, fixed case
* style
* apply suggestions
* added fast tests, added todo tests
* revert dummy objects for kolors
* fix pag dummies
* fix test imports
* update pag tests
* add kolor pag to docs
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-07 09:29:52 +05:30
Marc Sun
e4325606db
Fix loading sharded checkpoints when we have variants ( #9061 )
...
* Fix loading sharded checkpoint when we have variant
* add test
* remote print
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-06 13:38:44 -10:00
Ahn Donghoon (안동훈 / suno)
926daa30f9
add PAG support for Stable Diffusion 3 ( #8861 )
...
add pag sd3
---------
Co-authored-by: HyoungwonCho <jhw9811@korea.ac.kr >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: crepejung00 <jaewoojung00@naver.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-06 09:11:35 -10:00
Vinh H. Pham
87e50a2f1d
[Tests] Improve transformers model test suite coverage - Hunyuan DiT ( #8916 )
...
* add hunyuan model test
* apply suggestions
* reduce dims further
* reduce dims further
* run make style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-06 12:59:30 +05:30
Sayak Paul
52f1378e64
[Core] add QKV fusion to AuraFlow and PixArt Sigma ( #8952 )
...
* add fusion support to pixart
* add to auraflow.
* add tests
* apply review feedback.
* add back args and kwargs
* style
2024-08-05 14:09:37 -10:00
