Linoy Tsaban
413ca29b71
[Flux Dreambooth LoRA] - te bug fixes & updates ( #9139 )
...
* add requirements + fix link to bghira's guide
* text ecnoder training fixes
* text encoder training fixes
* text encoder training fixes
* text encoder training fixes
* style
* add tests
* fix encode_prompt call
* style
* unpack_latents test
* fix lora saving
* remove default val for max_sequenece_length in encode_prompt
* remove default val for max_sequenece_length in encode_prompt
* style
* testing
* style
* testing
* testing
* style
* fix sizing issue
* style
* revert scaling
* style
* style
* scaling test
* style
* scaling test
* remove model pred operation left from pre-conditioning
* remove model pred operation left from pre-conditioning
* fix trainable params
* remove te2 from casting
* transformer to accelerator
* remove prints
* empty commit
2024-08-12 11:58:03 +05:30
Dhruv Nair
10dc06c8d9
Update Video Loading/Export to use imageio ( #9094 )
...
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-12 10:19:53 +05:30
Dibbla!
3ece143308
Errata - fix typo ( #9100 )
2024-08-12 07:30:19 +05:30
Steven Liu
98930ee131
[docs] Resolve internal links to PEFT ( #9144 )
...
* resolve peft links
* fuse_lora
2024-08-10 06:37:46 +05:30
Daniel Socek
c1079f0887
Fix textual inversion SDXL and add support for 2nd text encoder ( #9010 )
...
* Fix textual inversion SDXL and add support for 2nd text encoder
Signed-off-by: Daniel Socek <daniel.socek@intel.com >
* Fix style/quality of text inv for sdxl
Signed-off-by: Daniel Socek <daniel.socek@intel.com >
---------
Signed-off-by: Daniel Socek <daniel.socek@intel.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-09 20:23:06 +05:30
Linoy Tsaban
65e30907b5
[Flux] Dreambooth LoRA training scripts ( #9086 )
...
* initial commit - dreambooth for flux
* update transformer to be FluxTransformer2DModel
* update training loop and validation inference
* fix sd3->flux docs
* add guidance handling, not sure if it makes sense(?)
* inital dreambooth lora commit
* fix text_ids in compute_text_embeddings
* fix imports of static methods
* fix pipeline loading in readme, remove auto1111 docs for now
* fix pipeline loading in readme, remove auto1111 docs for now, remove some irrelevant text_encoder_3 refs
* Update examples/dreambooth/train_dreambooth_flux.py
Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com >
* fix te2 loading and remove te2 refs from text encoder training
* fix tokenizer_2 initialization
* remove text_encoder training refs from lora script (for now)
* try with vae in bfloat16, fix model hook save
* fix tokenization
* fix static imports
* fix CLIP import
* remove text_encoder training refs (for now) from lora script
* fix minor bug in encode_prompt, add guidance def in lora script, ...
* fix unpack_latents args
* fix license in readme
* add "none" to weighting_scheme options for uniform sampling
* style
* adapt model saving - remove text encoder refs
* adapt model loading - remove text encoder refs
* initial commit for readme
* Update examples/dreambooth/train_dreambooth_lora_flux.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/dreambooth/train_dreambooth_lora_flux.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* fix vae casting
* remove precondition_outputs
* readme
* readme
* style
* readme
* readme
* update weighting scheme default & docs
* style
* add text_encoder training to lora script, change vae_scale_factor value in both
* style
* text encoder training fixes
* style
* update readme
* minor fixes
* fix te params
* fix te params
---------
Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-09 07:31:04 +05:30
Sayak Paul
cee7c1b0fb
Update README.md to include InstantID ( #8770 )
...
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-08-08 10:14:12 -07:00
Monjoy Narayan Choudhury
1fcb811a8e
Add Differential Diffusion to HunyuanDiT. ( #9040 )
...
* Add Differential Pipeline.
* Fix Styling Issue using ruff -fix
* Add details to Contributing.md
* Revert "Fix Styling Issue using ruff -fix"
This reverts commit d347de162dce7c3ff216aryan@huggingface.co >
2024-08-08 18:53:39 +05:30
David Steinberg
ae026db7aa
Fix a dead link ( #9116 )
...
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-08 18:46:50 +05:30
sayantan sadhu
8e3affc669
fix for lr scheduler in distributed training ( #9103 )
...
* fix for lr scheduler in distributed training
* Fixed the recalculation of the total training step section
* Fixed lint error
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-08 08:45:48 +05:30
Steven Liu
ba7e48455a
[docs] Organize model toctree ( #9118 )
...
* toctree
* fix
2024-08-08 08:31:58 +05:30
zR
2dad462d9b
Add CogVideoX text-to-video generation model ( #9082 )
...
* add CogVideoX
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-08-06 21:23:57 -10:00
Dhruv Nair
e3568d14ba
Freenoise change vae_batch_size to decode_chunk_size ( #9110 )
...
* update
* update
2024-08-07 12:47:18 +05:30
Aryan
f6df22447c
[feat] allow sparsectrl to be loaded from single file ( #9073 )
...
* allow sparsectrl to be loaded with single file
* update
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-08-07 11:12:30 +05:30
latentCall145
9b5180cb5f
Flux fp16 inference fix ( #9097 )
...
* clipping for fp16
* fix typo
* added fp16 inference to docs
* fix docs typo
* include link for fp16 investigation
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-07 10:54:20 +05:30
Aryan
16a93f1a25
[core] FreeNoise ( #8948 )
...
* initial work draft for freenoise; needs massive cleanup
* fix freeinit bug
* add animatediff controlnet implementation
* revert attention changes
* add freenoise
* remove old helper functions
* add decode batch size param to all pipelines
* make style
* fix copied from comments
* make fix-copies
* make style
* copy animatediff controlnet implementation from #8972
* add experimental support for num_frames not perfectly fitting context length, ocntext stride
* make unet motion model lora work again based on #8995
* copy load video utils from #8972
* copied from AnimateDiff::prepare_latents
* address the case where last batch of frames does not match length of indices in prepare latents
* decode_batch_size->vae_batch_size; batch vae encode support in animatediff vid2vid
* revert sparsectrl and sdxl freenoise changes
* revert pia
* add freenoise tests
* make fix-copies
* improve docstrings
* add freenoise tests to animatediff controlnet
* update tests
* Update src/diffusers/models/unets/unet_motion_model.py
* add freenoise to animatediff pag
* address review comments
* make style
* update tests
* make fix-copies
* fix error message
* remove copied from comment
* fix imports in tests
* update
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-08-07 10:35:18 +05:30
Sayak Paul
2d753b6fb5
fix train_dreambooth_lora_sd3.py loading hook ( #9107 )
2024-08-07 10:09:47 +05:30
Álvaro Somoza
39e1f7eaa4
[Kolors] Add PAG ( #8934 )
...
* txt2img pag added
* autopipe added, fixed case
* style
* apply suggestions
* added fast tests, added todo tests
* revert dummy objects for kolors
* fix pag dummies
* fix test imports
* update pag tests
* add kolor pag to docs
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-07 09:29:52 +05:30
Dhruv Nair
e1b603dc2e
[Single File] Add single file support for Flux Transformer ( #9083 )
...
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-07 08:49:57 +05:30
Marc Sun
e4325606db
Fix loading sharded checkpoints when we have variants ( #9061 )
...
* Fix loading sharded checkpoint when we have variant
* add test
* remote print
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-06 13:38:44 -10:00
Ahn Donghoon (안동훈 / suno)
926daa30f9
add PAG support for Stable Diffusion 3 ( #8861 )
...
add pag sd3
---------
Co-authored-by: HyoungwonCho <jhw9811@korea.ac.kr >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: crepejung00 <jaewoojung00@naver.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-06 09:11:35 -10:00
Dhruv Nair
325a5de3a9
[Docs] Add community projects section to docs ( #9013 )
...
* update
* update
* update
2024-08-06 08:59:39 -07:00
Dhruv Nair
4c6152c2fb
update
2024-08-06 12:00:14 +00:00
Vinh H. Pham
87e50a2f1d
[Tests] Improve transformers model test suite coverage - Hunyuan DiT ( #8916 )
...
* add hunyuan model test
* apply suggestions
* reduce dims further
* reduce dims further
* run make style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-06 12:59:30 +05:30
Aryan
a57a7af45c
[bug] remove unreachable norm_type=ada_norm_continuous from norm3 initialization conditions ( #9006 )
...
remove ada_norm_continuous from norm3 list
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-06 07:23:48 +05:30
Sayak Paul
52f1378e64
[Core] add QKV fusion to AuraFlow and PixArt Sigma ( #8952 )
...
* add fusion support to pixart
* add to auraflow.
* add tests
* apply review feedback.
* add back args and kwargs
* style
2024-08-05 14:09:37 -10:00
Tolga Cangöz
3dc97bd148
Update CLIPFeatureExtractor to CLIPImageProcessor and DPTFeatureExtractor to DPTImageProcessor ( #9002 )
...
* fix: update `CLIPFeatureExtractor` to `CLIPImageProcessor` in codebase
* `make style && make quality`
* Update `DPTFeatureExtractor` to `DPTImageProcessor` in codebase
* `make style`
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-05 09:20:29 -10:00
omahs
6d32b29239
Fix typos ( #9077 )
...
* fix typo
2024-08-05 09:00:08 -10:00
YiYi Xu
bc3c73ad0b
add sentencepiece as a soft dependency ( #9065 )
...
* add sentencepiece as soft dependency for kolors
* up
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-05 08:04:51 -10:00
Sayak Paul
5934873b8f
[Docs] add stable cascade unet doc. ( #9066 )
...
* add stable cascade unet doc.
* fix path
2024-08-05 21:28:48 +05:30
Aryan
b7058d142c
PAG variant for HunyuanDiT, PAG refactor ( #8936 )
...
* copy hunyuandit pipeline
* pag variant of hunyuan dit
* add tests
* update docs
* make style
* make fix-copies
* Update src/diffusers/pipelines/pag/pag_utils.py
* remove incorrect copied from
* remove pag hunyuan attn procs to resolve conflicts
* add pag attn procs again
* new implementation for pag_utils
* revert pag changes
* add pag refactor back; update pixart sigma
* update pixart pag tests
* apply suggestions from review
Co-Authored-By: yixu310@gmail.com
* make style
* update docs, fix tests
* fix tests
* fix test_components_function since list not accepted as valid __init__ param
* apply patch to fix broken tests
Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com >
* make style
* fix hunyuan tests
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-05 17:56:09 +05:30
Vinh H. Pham
e1d508ae92
[Tests] Improve transformers model test suite coverage - Latte ( #8919 )
...
* add LatteTransformer3DModel model test
* change patch_size to 1
* reduce req len
* reduce channel dims
* increase num_layers
* reduce dims further
* run make style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-05 17:13:03 +05:30
Sayak Paul
fc6a91e383
[FLUX] support LoRA ( #9057 )
...
* feat: lora support for Flux.
add tests
fix imports
major fixes.
* fix
fixes
final fixes?
* fix
* remove is_peft_available.
2024-08-05 10:24:05 +05:30
Aryan
2b76099610
[refactor] apply qk norm in attention processors ( #9071 )
...
* apply qk norm in attention processors
* revert attention processor
* qk-norm in only attention proc 2.0 and fused variant
2024-08-04 05:42:46 -10:00
psychedelicious
4f0d01d387
type get_attention_scores as optional in get_attention_scores ( #9075 )
...
`None` is valid for `get_attention_scores`, should be typed as such
2024-08-04 17:19:05 +05:30
asfiyab-nvidia
3dc10a535f
Update TensorRT txt2img and inpaint community pipelines ( #9037 )
...
* Update TensorRT txt2img and inpaint community pipelines
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
* update tensorrt install instructions
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
---------
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-04 16:00:40 +05:30
Sayak Paul
c370b90ff1
[Flux] minor documentation fixes for flux. ( #9048 )
...
* minor documentation fixes for flux.
* clipskip
* add gist
2024-08-04 15:53:01 +05:30
Philip Rideout
ebf3ab1477
Fix grammar mistake. ( #9072 )
2024-08-04 04:32:03 +05:30
Aryan
fbe29c6298
[refactor] create modeling blocks specific to AnimateDiff ( #8979 )
...
* animatediff specific transformer model
* make style
* make fix-copies
* move blocks to unet motion model
* make style
* remove dummy object
* fix incorrectly passed param causing test failures
* rename model and output class
* fix sparsectrl imports
* remove todo comments
* remove temporal double self attn param from controlnet sparsectrl
* add deprecated versions of blocks
* apply suggestions from review
* update
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-08-03 13:03:39 +05:30
Tolga Cangöz
7071b7461b
Errata: Fix typos & \s+$ ( #9008 )
...
* Fix typos
* chore: Fix typos
* chore: Update README.md for promptdiffusion example
* Trim trailing white spaces
* Fix a typo
* update number
* chore: update number
* Trim trailing white space
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-08-02 21:24:25 -07:00
Frank (Haofan) Wang
a054c78495
Update transformer_flux.py ( #9060 )
2024-08-03 08:58:32 +05:30
Dhruv Nair
b1f43d7189
Fix Nightly Deps ( #9036 )
...
update
2024-08-02 15:18:54 +05:30
Sayak Paul
0e460675e2
[Flux] allow tests to run ( #9050 )
...
* fix tests
* fix
* float64 skip
* remove sample_size.
* remove
* remove more
* default_sample_size.
* credit black forest for flux model.
* skip
* fix: tests
* remove OriginalModelMixin
* add transformer model test
* add: transformer model tests
2024-08-02 11:49:59 +05:30
Sayak Paul
7b98c4cc67
[Core] Add PAG support for PixArtSigma ( #8921 )
...
* feat: add pixart sigma pag.
* inits.
* fixes
* fix
* remove print.
* copy paste methods to the pixart pag mixin
* fix-copies
* add documentation.
* add tests.
* remove correction file.
* remove pag_applied_layers
* empty
2024-08-02 07:12:41 +05:30
Sayak Paul
27637a5402
Flux pipeline ( #9043 )
...
add flux!
Signed-off-by: Adrien <adrien@huggingface.co >
Co-authored-by: Adrien <adrien.69740@gmail.com >
Co-authored-by: Anatoly Belikov <abelikov@singularitynet.io >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-08-01 11:30:52 -10:00
Aryan
2ea22e1cc7
[docs] fix pia example ( #9015 )
...
fix pia example docstring
2024-08-02 02:47:40 +05:30
YiYi Xu
95a7832879
fix load sharded checkpoint from a subfolder (local path) ( #8913 )
...
fix
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-01 20:15:42 +05:30
Dhruv Nair
c646fbc124
Updates deps for pipeline test fetcher ( #9033 )
...
update
2024-08-01 13:22:13 +05:30
Aryan
05b706c003
PAG variant for AnimateDiff ( #8789 )
...
* add animatediff pag pipeline
* remove unnecessary print
* make fix-copies
* fix ip-adapter bug
* update docs
* add fast tests and fix bugs
* update
* update
* address review comments
* update ip adapter single test expected slice
* implement test_from_pipe_consistent_config; fix expected slice values
* LoraLoaderMixin->StableDiffusionLoraLoaderMixin; add latest freeinit test
2024-08-01 12:39:39 +05:30
Yoach Lacombe
ea1b4ea7ca
Fix Stable Audio repository id ( #9016 )
...
Fix Stable Audio repo id
2024-07-30 23:17:44 +05:30
