AI/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-01-27 17:22:53 +03:00
Code Activity
4,938 Commits 740 Branches 94 Tags
v0.32.2
Commit Graph

109 Commits

Author SHA1 Message Date
Aryan
526858c801 [LoRA] Support original format loras for HunyuanVideo (#10376) 
* update

* fix make copies

* update

* add relevant markers to the integration test suite.

* add copied.

* fox-copies

* temporarily add print.

* directly place on CUDA as CPU isn't that big on the CIO.

* fixes to fuse_lora, aryan was right.

* fixes

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2025-01-15 17:55:40 +05:30
Sayak Paul
b3d2dd36d7 [LoRA] fix: lora unloading when using expanded Flux LoRAs. (#10397) 
* fix: lora unloading when using expanded Flux LoRAs.

* fix argument name.

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>

* docs.

---------

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>
 2025-01-15 17:55:19 +05:30
maxs-kan
abfa922410 Fix Flux multiple Lora loading bug (#10388) 
* check for base_layer key in transformer state dict

* test_lora_expansion_works_for_absent_keys

* check

* Update tests/lora/test_lora_layers_flux.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* check

* test_lora_expansion_works_for_absent_keys/test_lora_expansion_works_for_extra_keys

* absent->extra

---------

Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2025-01-15 17:55:05 +05:30
Sayak Paul
6a7b01f60f [LoRA] feat: support unload_lora_weights() for Flux Control. (#10206) 
* feat: support unload_lora_weights() for Flux Control.

* tighten test

* minor

* updates

* meta device fixes.
 2025-01-15 17:54:56 +05:30
Aryan
4b557132ce [core] LTX Video 0.9.1 (#10330) 
* update

* make style

* update

* update

* update

* make style

* single file related changes

* update

* fix

* update single file urls and docs

* update

* fix
 2024-12-23 19:51:33 +05:30
Sayak Paul
851dfa30ae [Tests] Fix more tests sayak (#10359) 
* fixes to tests

* fixture

* fixes
 2024-12-23 19:11:21 +05:30
Sayak Paul
ea1ba0ba53 [LoRA] test fix (#10351) 
updates
 2024-12-23 15:45:45 +05:30
Sayak Paul
c34fc34563 [Tests] QoL improvements to the LoRA test suite (#10304) 
* misc lora test improvements.

* updates

* fixes to tests
 2024-12-23 13:59:55 +05:30
Sayak Paul
76e2727b5c [SANA LoRA] sana lora training tests and misc. (#10296) 
* sana lora training tests and misc.

* remove push to hub

* Update examples/dreambooth/train_dreambooth_lora_sana.py

Co-authored-by: Aryan <aryan@huggingface.co>

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-23 12:35:13 +05:30
Sayak Paul
bf6eaa8aec [Tests] add integration tests for lora expansion stuff in Flux. (#10318) 
add integration tests for lora expansion stuff in Flux.
 2024-12-20 16:14:58 +05:30
Sayak Paul
17128c42a4 [LoRA] feat: support loading regular Flux LoRAs into Flux Control, and Fill (#10259) 
* lora expansion with dummy zeros.

* updates

* fix working 🥳

* working.

* use torch.device meta for state dict expansion.

* tests

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>

* fixes

* fixes

* switch to debug

* fix

* Apply suggestions from code review

Co-authored-by: Aryan <aryan@huggingface.co>

* fix stuff

* docs

---------

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-20 14:30:32 +05:30
Aryan
d8825e7697 Fix failing lora tests after HunyuanVideo lora (#10307) 
fix
 2024-12-20 02:35:41 +05:30
Shenghai Yuan
1826a1e7d3 [LoRA] Support HunyuanVideo (#10254) 
* 1217

* 1217

* 1217

* update

* reverse

* add test

* update test

* make style

* update

* make style

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-19 16:22:20 +05:30
Aryan
f35a38725b [tests] remove nullop import checks from lora tests (#10273) 
remove nullop imports
 2024-12-19 01:19:08 +05:30
Sayak Paul
9408aa2dfc [LoRA] feat: lora support for SANA. (#10234) 
* feat: lora support for SANA.

* make fix-copies

* rename test class.

* attention_kwargs -> cross_attention_kwargs.

* Revert "attention_kwargs -> cross_attention_kwargs."

This reverts commit 23433bf9bc.

* exhaust 119 max line limit

* sana lora fine-tuning script.

* readme

* add a note about the supported models.

* Apply suggestions from code review

Co-authored-by: Aryan <aryan@huggingface.co>

* style

* docs for attention_kwargs.

* remove lora_scale from pag pipeline.

* copy fix

---------

Co-authored-by: Aryan <aryan@huggingface.co>
 2024-12-18 08:22:31 +05:30
Aryan
ac86393487 [LoRA] Support LTX Video (#10228) 
* add lora support for ltx

* add tests

* fix copied from comments

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-17 12:05:05 +05:30
Aryan
22c4f079b1 Test error raised when loading normal and expanding loras together in Flux (#10188) 
* add test for expanding lora and normal lora error

* Update tests/lora/test_lora_layers_flux.py

* fix things.

* Update src/diffusers/loaders/peft.py

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-15 21:46:21 +05:30
Sayak Paul
a6a18cff5e [LoRA] add a test to ensure set_adapters() and attn kwargs outs match (#10110) 
* add a test to ensure set_adapters() and attn kwargs outs match

* remove print

* fix

* Apply suggestions from code review

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

* assertFalse.

---------

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
 2024-12-12 12:52:50 +05:30
Aryan
49a9143479 Flux Control LoRA (#9999) 
* update


---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-12-10 09:08:13 -10:00
Sayak Paul
40fc389c44 [Tests] fix condition argument in xfail. (#10099) 
* fix condition argument in xfail.

* revert init changes.
 2024-12-05 10:13:45 +05:30
Sayak Paul
2e86a3f023 [Tests] skip nan lora tests on PyTorch 2.5.1 CPU. (#9975) 
* skip nan lora tests on PyTorch 2.5.1 CPU.

* cog

* use xfail

* correct xfail

* add condition

* tests
 2024-11-22 12:45:21 +05:30
raulmosa
3139d39fa7 Update handle single blocks on _convert_xlabs_flux_lora_to_diffusers (#9915) 
* Update handle single blocks on _convert_xlabs_flux_lora_to_diffusers to fix bug on updating keys and old_state_dict


---------

Co-authored-by: raul_ar <raul.moreno.salinas@autoretouch.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-20 12:53:20 -10:00
Sayak Paul
805aa93789 [LoRA] enable LoRA for Mochi-1 (#9943) 
* feat: add lora support to Mochi-1.
 2024-11-20 12:07:04 -10:00
Sayak Paul
7d0b9c4d4e [LoRA] feat: save_lora_adapter() (#9862) 
* feat: save_lora_adapter.
 2024-11-18 21:03:38 -10:00
SahilCarterr
08ac5cbc7f [Fix] Test of sd3 lora (#9843) 
* fix test

* fix test asser

* fix format

* Update test_lora_layers_sd3.py
 2024-11-05 11:05:20 -10:00
Sayak Paul
13e8fdecda [feat] add load_lora_adapter() for compatible models (#9712) 
* add first draft.

* fix

* updates.

* updates.

* updates

* updates

* updates.

* fix-copies

* lora constants.

* add tests

* Apply suggestions from code review

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

* docstrings.

---------

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
 2024-11-02 09:50:39 +05:30
Sayak Paul
1d1e1a2888 Some minor updates to the nightly and push workflows (#9759) 
* move lora integration tests to nightly./

* remove slow marker in the workflow where not needed.
 2024-10-24 23:49:09 +09:00
Sayak Paul
cef4f65cf7 [LoRA] log a warning when there are missing keys in the LoRA loading. (#9622) 
* log a warning when there are missing keys in the LoRA loading.

* handle missing keys and unexpected keys better.

* add tests

* fix-copies.

* updates

* tests

* concat warning.

* Add Differential Diffusion to Kolors (#9423)

* Added diff diff support for kolors img2img

* Fized relative imports

* Fized relative imports

* Added diff diff support for Kolors

* Fized import issues

* Added map

* Fized import issues

* Fixed naming issues

* Added diffdiff support for Kolors img2img pipeline

* Removed example docstrings

* Added map input

* Updated latents

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Updated `original_with_noise`

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Improved code quality

---------

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* FluxMultiControlNetModel (#9647)

* tests

* Update src/diffusers/loaders/lora_pipeline.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* fix

---------

Co-authored-by: M Saqlain <118016760+saqlain2204@users.noreply.github.com>
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-10-16 07:46:12 +05:30
SahilCarterr
22ed39f571 Added Lora Support to SD3 Img2Img Pipeline (#9659) 
* add lora
 2024-10-14 11:39:20 -10:00
Sayak Paul
86bcbc389e [Tests] increase transformers version in test_low_cpu_mem_usage_with_loading (#9662) 
increase transformers version in test_low_cpu_mem_usage_with_loading
 2024-10-13 22:39:38 +05:30
Sayak Paul
e16fd93d0a [LoRA] fix dora test to catch the warning properly. (#9627) 
fix dora test.
 2024-10-10 11:47:49 +05:30
Sayak Paul
31058cdaef [LoRA] allow loras to be loaded with low_cpu_mem_usage. (#9510) 
* allow loras to be loaded with low_cpu_mem_usage.

* add flux support but note https://github.com/huggingface/diffusers/pull/9510\#issuecomment-2378316687

* low_cpu_mem_usage.

* fix-copies

* fix-copies again

* tests

* _LOW_CPU_MEM_USAGE_DEFAULT_LORA

* _peft_version default.

* version checks.

* version check.

* version check.

* version check.

* require peft 0.13.1.

* explicitly specify low_cpu_mem_usage=False.

* docs.

* transformers version 4.45.2.

* update

* fix

* empty

* better name initialize_dummy_state_dict.

* doc todos.

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* style

* fix-copies

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-10-09 10:57:16 +05:30
Sayak Paul
02eeb8e77e [LoRA] Handle DoRA better (#9547) 
* handle dora.

* print test

* debug

* fix

* fix-copies

* update logits

* add warning in the test.

* make is_dora check consistent.

* fix-copies
 2024-10-08 21:47:44 +05:30
Sayak Paul
31010ecc45 [Chore] add a note on the versions in Flux LoRA integration tests (#9598) 
add a note on the versions.
 2024-10-07 17:43:48 +05:30
Sayak Paul
f9fd511466 [LoRA] support Kohya Flux LoRAs that have text encoders as well (#9542) 
* support kohya flux loras that have tes.
 2024-09-30 07:59:39 -10:00
Sayak Paul
81cf3b2f15 [Tests] [LoRA] clean up the serialization stuff. (#9512) 
* clean up the serialization stuff.

* better
 2024-09-27 07:57:09 -10:00
Sayak Paul
2daedc0ad3 [LoRA] make set_adapters() method more robust. (#9535) 
* make set_adapters() method more robust.

* remove patch

* better and concise code.

* Update src/diffusers/loaders/lora_base.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-09-27 07:32:43 +05:30
YiYi Xu
bac8a2412d a few fix for SingleFile tests (#9522) 
* update sd15 repo

* update more
 2024-09-24 13:36:53 -10:00
Aryan
e5d0a328d6 [refactor] LoRA tests (#9481) 
* refactor scheduler class usage

* reorder to make tests more readable

* remove pipeline specific checks and skip tests directly

* rewrite denoiser conditions cleaner

* bump tolerance for cog test
 2024-09-21 07:10:36 +05:30
Aryan
2b443a5d62 [training] CogVideoX Lora (#9302) 
* cogvideox lora training draft

* update

* update

* update

* update

* update

* make fix-copies

* update

* update

* apply suggestions from review

* apply suggestions from reveiw

* fix typo

* Update examples/cogvideo/train_cogvideox_lora.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* fix lora alpha

* use correct lora scaling for final test pipeline

* Update examples/cogvideo/train_cogvideox_lora.py

Co-authored-by: YiYi Xu <yixu310@gmail.com>

* apply suggestions from review; prodigy optimizer

YiYi Xu <yixu310@gmail.com>

* add tests

* make style

* add README

* update

* update

* make style

* fix

* update

* add test skeleton

* revert lora utils changes

* add cleaner modifications to lora testing utils

* update lora tests

* deepspeed stuff

* add requirements.txt

* deepspeed refactor

* add lora stuff to img2vid pipeline to fix tests

* fight tests

* add co-authors

Co-Authored-By: Fu-Yun Wang <1697256461@qq.com>

Co-Authored-By: zR <2448370773@qq.com>

* fight lora runner tests

* import Dummy optim and scheduler only wheh required

* update docs

* add coauthors

Co-Authored-By: Fu-Yun Wang <1697256461@qq.com>

* remove option to train text encoder

Co-Authored-By: bghira <bghira@users.github.com>

* update tests

* fight more tests

* update

* fix vid2vid

* fix typo

* remove lora tests; todo in follow-up PR

* undo img2vid changes

* remove text encoder related changes in lora loader mixin

* Revert "remove text encoder related changes in lora loader mixin"

This reverts commit f8a8444487.

* update

* round 1 of fighting tests

* round 2 of fighting tests

* fix copied from comment

* fix typo in lora test

* update styling

Co-Authored-By: YiYi Xu <yixu310@gmail.com>

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: zR <2448370773@qq.com>
Co-authored-by: Fu-Yun Wang <1697256461@qq.com>
Co-authored-by: bghira <bghira@users.github.com>
 2024-09-19 14:37:57 +05:30
Sayak Paul
d13b0d63c0 [Flux] add lora integration tests. (#9353) 
* add lora integration tests.

* internal note

* add a skip marker.
 2024-09-19 09:21:28 +05:30
Dhruv Nair
1e8cf2763d [CI] Nightly Test Updates (#9380) 
* update

* update

* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-09-12 20:21:28 +05:30
Dhruv Nair
f6f16a0c11 [CI] More Fast GPU Test Fixes (#9346) 
* update

* update

* update

* update
 2024-09-03 13:22:38 +05:30
Sayak Paul
5090b09d48 [Flux LoRA] support parsing alpha from a flux lora state dict. (#9236) 
* support parsing alpha from a flux lora state dict.

* conditional import.

* fix breaking changes.

* safeguard alpha.

* fix
 2024-08-22 07:01:52 +05:30
Dhruv Nair
940b8e0358 [CI] Multiple Slow Test fixes. (#9198) 
* update

* update

* update

* update
 2024-08-19 13:31:09 +05:30
Sayak Paul
fc6a91e383 [FLUX] support LoRA (#9057) 
* feat: lora support for Flux.

add tests

fix imports

major fixes.

* fix

fixes

final fixes?

* fix

* remove is_peft_available.
 2024-08-05 10:24:05 +05:30
Sayak Paul
d87fe95f90 [Chore] add LoraLoaderMixin to the inits (#8981) 
* introduce  to promote reusability.

* up

* add more tests

* up

* remove comments.

* fix fuse_nan test

* clarify the scope of fuse_lora and unfuse_lora

* remove space

* rewrite fuse_lora a bit.

* feedback

* copy over load_lora_into_text_encoder.

* address dhruv's feedback.

* fix-copies

* fix issubclass.

* num_fused_loras

* fix

* fix

* remove mapping

* up

* fix

* style

* fix-copies

* change to SD3TransformerLoRALoadersMixin

* Apply suggestions from code review

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* up

* handle wuerstchen

* up

* move lora to lora_pipeline.py

* up

* fix-copies

* fix documentation.

* comment set_adapters().

* fix-copies

* fix set_adapters() at the model level.

* fix?

* fix

* loraloadermixin.

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-07-26 08:59:33 +05:30
YiYi Xu
62863bb1ea Revert "[LoRA] introduce LoraBaseMixin to promote reusability." (#8976) 
Revert "[LoRA] introduce LoraBaseMixin to promote reusability. (#8774)"

This reverts commit 527430d0a4.
 2024-07-25 09:10:35 -10:00
Sayak Paul
527430d0a4 [LoRA] introduce LoraBaseMixin to promote reusability. (#8774) 
* introduce  to promote reusability.

* up

* add more tests

* up

* remove comments.

* fix fuse_nan test

* clarify the scope of fuse_lora and unfuse_lora

* remove space

* rewrite fuse_lora a bit.

* feedback

* copy over load_lora_into_text_encoder.

* address dhruv's feedback.

* fix-copies

* fix issubclass.

* num_fused_loras

* fix

* fix

* remove mapping

* up

* fix

* style

* fix-copies

* change to SD3TransformerLoRALoadersMixin

* Apply suggestions from code review

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* up

* handle wuerstchen

* up

* move lora to lora_pipeline.py

* up

* fix-copies

* fix documentation.

* comment set_adapters().

* fix-copies

* fix set_adapters() at the model level.

* fix?

* fix

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-07-25 21:40:58 +05:30
Tolga Cangöz
57084dacc5 Remove unnecessary lines (#8569) 
* Remove unused line


---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-07-08 10:42:02 -10:00