Steven Liu
c934720629
[docs] Model cards ( #11112 )
...
* initial
* update
* hunyuanvideo
* ltx
* fix
* wan
* gen guide
* feedback
* feedback
* pipeline-level quant config
* feedback
* ltx
2025-06-02 16:55:14 -07:00
Linoy Tsaban
e30d3bf544
[LoRA] add LoRA support to HiDream and fine-tuning script ( #11281 )
...
* initial commit
* initial commit
* initial commit
* initial commit
* initial commit
* initial commit
* Update examples/dreambooth/train_dreambooth_lora_hidream.py
Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com >
* move prompt embeds, pooled embeds outside
* Update examples/dreambooth/train_dreambooth_lora_hidream.py
Co-authored-by: hlky <hlky@hlky.ac >
* Update examples/dreambooth/train_dreambooth_lora_hidream.py
Co-authored-by: hlky <hlky@hlky.ac >
* fix import
* fix import and tokenizer 4, text encoder 4 loading
* te
* prompt embeds
* fix naming
* shapes
* initial commit to add HiDreamImageLoraLoaderMixin
* fix init
* add tests
* loader
* fix model input
* add code example to readme
* fix default max length of text encoders
* prints
* nullify training cond in unpatchify for temp fix to incompatible shaping of transformer output during training
* smol fix
* unpatchify
* unpatchify
* fix validation
* flip pred and loss
* fix shift!!!
* revert unpatchify changes (for now)
* smol fix
* Apply style fixes
* workaround moe training
* workaround moe training
* remove prints
* to reduce some memory, keep vae in `weight_dtype` same as we have for flux (as it's the same vae)
bbd0c161b5/examples/dreambooth/train_dreambooth_lora_flux.py (L1207)spsayakpaul@gmail.com >
* Update examples/dreambooth/train_dreambooth_lora_hidream.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* vae offload change
* fix text encoder offloading
* Apply style fixes
* cleaner to_kwargs
* fix module name in copied from
* add requirements
* fix offloading
* fix offloading
* fix offloading
* update transformers version in reqs
* try AutoTokenizer
* try AutoTokenizer
* Apply style fixes
* empty commit
* Delete tests/lora/test_lora_layers_hidream.py
* change tokenizer_4 to load with AutoTokenizer as well
* make text_encoder_four and tokenizer_four configurable
* save model card
* save model card
* revert T5
* fix test
* remove non diffusers lumina2 conversion
---------
Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com >
Co-authored-by: hlky <hlky@hlky.ac >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-22 11:44:02 +03:00
Sayak Paul
efc9d68b15
[chore] fix lora docs utils ( #11338 )
...
fix lora docs utils
2025-04-17 09:25:53 +05:30
Hameer Abbasi
9352a5ca56
[LoRA] Add LoRA support to AuraFlow ( #10216 )
...
* Add AuraFlowLoraLoaderMixin
* Add comments, remove qkv fusion
* Add Tests
* Add AuraFlowLoraLoaderMixin to documentation
* Add Suggested changes
* Change attention_kwargs->joint_attention_kwargs
* Rebasing derp.
* fix
* fix
* Quality fixes.
* make style
* `make fix-copies`
* `ruff check --fix`
* Attept 1 to fix tests.
* Attept 2 to fix tests.
* Attept 3 to fix tests.
* Address review comments.
* Rebasing derp.
* Get more tests passing by copying from Flux. Address review comments.
* `joint_attention_kwargs`->`attention_kwargs`
* Add `lora_scale` property for te LoRAs.
* Make test better.
* Remove useless property.
* Skip TE-only tests for AuraFlow.
* Support LoRA for non-CLIP TEs.
* Restore LoRA tests.
* Undo adding LoRA support for non-CLIP TEs.
* Undo support for TE in AuraFlow LoRA.
* `make fix-copies`
* Sync with upstream changes.
* Remove unneeded stuff.
* Mirror `Lumina2`.
* Skip for MPS.
* Address review comments.
* Remove duplicated code.
* Remove unnecessary code.
* Remove repeated docs.
* Propagate attention.
* Fix TE target modules.
* MPS fix for LoRA tests.
* Unrelated TE LoRA tests fix.
* Fix AuraFlow LoRA tests by applying to the right denoiser layers.
Co-authored-by: AstraliteHeart <81396681+AstraliteHeart@users.noreply.github.com >
* Apply style fixes
* empty commit
* Fix the repo consistency issues.
* Remove unrelated changes.
* Style.
* Fix `test_lora_fuse_nan`.
* fix quality issues.
* `pytest.xfail` -> `ValueError`.
* Add back `skip_mps`.
* Apply style fixes
* `make fix-copies`
---------
Co-authored-by: Warlord-K <warlordk28@gmail.com >
Co-authored-by: hlky <hlky@hlky.ac >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: AstraliteHeart <81396681+AstraliteHeart@users.noreply.github.com >
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-15 10:41:28 +05:30
Sayak Paul
f10d3c6d04
[LoRA] add LoRA support to Lumina2 and fine-tuning script ( #10818 )
...
* feat: lora support for Lumina2.
* fix-copies.
* updates
* updates
* docs.
* fix
* add: training script.
* tests
* updates
* updates
* major updates.
* updates
* fixes
* docs.
* updates
* updates
2025-02-20 09:41:51 +05:30
Sayak Paul
924f880d4d
[docs] add missing entries to the lora docs. ( #10819 )
...
add missing entries to the lora docs.
2025-02-18 09:10:18 -08:00
Daniel Regado
3191248472
[WIP] SD3.5 IP-Adapter Pipeline Integration ( #9987 )
...
* Added support for single IPAdapter on SD3.5 pipeline
---------
Co-authored-by: hlky <hlky@hlky.ac >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-19 14:48:18 -10:00
Sayak Paul
ea893a9ae7
[Docs] add rest of the lora loader mixins to the docs. ( #10230 )
...
add rest of the lora loader mixins to the docs.
2024-12-16 08:50:27 -08:00
Aryan
ba06124e4a
Remove CogVideoX mentions from single file docs; Test updates ( #9444 )
...
* remove mentions from single file
* update tests
* update
2024-09-17 10:05:45 -10:00
Yuxuan.Zhang
8336405e50
CogVideoX-5b-I2V support ( #9418 )
...
* draft Init
* draft
* vae encode image
* make style
* image latents preparation
* remove image encoder from conversion script
* fix minor bugs
* make pipeline work
* make style
* remove debug prints
* fix imports
* update example
* make fix-copies
* add fast tests
* fix import
* update vae
* update docs
* update image link
* apply suggestions from review
* apply suggestions from review
* add slow test
* make use of learned positional embeddings
* apply suggestions from review
* doc change
* Update convert_cogvideox_to_diffusers.py
* make style
* final changes
* make style
* fix tests
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-09-16 14:46:24 +05:30
zR
2dad462d9b
Add CogVideoX text-to-video generation model ( #9082 )
...
* add CogVideoX
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-08-06 21:23:57 -10:00
Dhruv Nair
4c6152c2fb
update
2024-08-06 12:00:14 +00:00
Sayak Paul
d87fe95f90
[Chore] add LoraLoaderMixin to the inits ( #8981 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
* rewrite fuse_lora a bit.
* feedback
* copy over load_lora_into_text_encoder.
* address dhruv's feedback.
* fix-copies
* fix issubclass.
* num_fused_loras
* fix
* fix
* remove mapping
* up
* fix
* style
* fix-copies
* change to SD3TransformerLoRALoadersMixin
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* up
* handle wuerstchen
* up
* move lora to lora_pipeline.py
* up
* fix-copies
* fix documentation.
* comment set_adapters().
* fix-copies
* fix set_adapters() at the model level.
* fix?
* fix
* loraloadermixin.
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-07-26 08:59:33 +05:30
YiYi Xu
62863bb1ea
Revert "[LoRA] introduce LoraBaseMixin to promote reusability." ( #8976 )
...
Revert "[LoRA] introduce LoraBaseMixin to promote reusability. (#8774 )"
This reverts commit 527430d0a4
2024-07-25 09:10:35 -10:00
Sayak Paul
527430d0a4
[LoRA] introduce LoraBaseMixin to promote reusability. ( #8774 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
* rewrite fuse_lora a bit.
* feedback
* copy over load_lora_into_text_encoder.
* address dhruv's feedback.
* fix-copies
* fix issubclass.
* num_fused_loras
* fix
* fix
* remove mapping
* up
* fix
* style
* fix-copies
* change to SD3TransformerLoRALoadersMixin
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* up
* handle wuerstchen
* up
* move lora to lora_pipeline.py
* up
* fix-copies
* fix documentation.
* comment set_adapters().
* fix-copies
* fix set_adapters() at the model level.
* fix?
* fix
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-07-25 21:40:58 +05:30
Sayak Paul
984d340534
Revert "[LoRA] introduce LoraBaseMixin to promote reusability." ( #8773 )
...
Revert "[LoRA] introduce `LoraBaseMixin` to promote reusability. (#8670 )"
This reverts commit a2071a1837
2024-07-03 07:05:01 +05:30
Sayak Paul
a2071a1837
[LoRA] introduce LoraBaseMixin to promote reusability. ( #8670 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
2024-07-03 07:04:37 +05:30
Dhruv Nair
b1a2c0d577
Expand Single File support in SD3 Pipeline ( #8517 )
...
* update
* update
2024-06-13 18:29:19 +05:30
Steven Liu
151a56b80e
[docs] Single file usage ( #8412 )
...
* single file usage
* edit
2024-06-06 12:40:34 -07:00
Dhruv Nair
cef4a51223
Clean up from_single_file docs ( #8268 )
...
* update
* update
2024-05-24 17:43:51 +05:30
Dhruv Nair
cb0f3b49cb
[Refactor] Better align from_single_file logic with from_pretrained ( #7496 )
...
* refactor unet single file loading a bit.
* retrieve the unet from create_diffusers_unet_model_from_ldm
* update
* update
* updae
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* tests
* update
* update
* update
* Update docs/source/en/api/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/loaders/single_file.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-05-09 19:00:19 +05:30
Steven Liu
1f22c98820
[docs] IP-Adapter image embedding ( #7226 )
...
* update
* fix parameter name
* feedback
* add no mask version
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-08 08:49:58 -08:00
Steven Liu
9efe1e52c3
[docs] IP-Adapter ( #6897 )
...
* use cases
* first draft
* fix image links
* lcm-lora
* feedback
* review
* feedback
* feedback
2024-02-14 13:23:37 -08:00
Sayak Paul
30e5e81d58
change to 2024 in the license ( #6902 )
...
change to 2024
2024-02-08 08:19:31 -10:00
Dhruv Nair
fee93c81eb
[Refactor] Update from single file ( #6428 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update'
* update
* update
* update
* update
* update
* update
* up
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* up
* update
* update
* update
* update
* update'
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* clean
* update
* update
* clean up
* clean up
* update
* clean
* clean
* update
* updaet
* clean up
* fix docs
* update
* update
* Revert "update"
This reverts commit dbfb8f1ea9
2024-01-23 14:42:03 +05:30
Steven Liu
3c79dd9dbe
[docs] PEFT adapter API ( #6499 )
...
follow up
2024-01-09 08:09:15 -08:00
Sayak Paul
585f941366
[Core] introduce PeftAdapterMixin module. ( #6416 )
...
* introduce integrations module.
* remove duplicate methods.
* better imports.
* move to loaders.py
* remove peftadaptermixin from modelmixin.
* add: peftadaptermixin selectively.
* add: entry to _toctree
* Empty-Commit
2024-01-05 18:18:28 +05:30
Steven Liu
8ccc76ab37
[docs] IP-Adapter API doc ( #6140 )
...
add ip-adapter
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-14 09:19:37 -08:00
Steven Liu
7457aa67cb
[docs] Loader APIs ( #5813 )
...
* first draft
* remove old loader doc
* start adding lora code examples
* finish
* add link to loralinearlayer
* feedback
* fix
2023-11-20 10:53:13 -08:00
