Dhruv Nair
f7cb595428
[Single File] Fix loading Flux Dev finetunes with Comfy Prefix ( #10545 )
...
* update
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2025-01-13 21:25:07 +05:30
hlky
c3478a42b9
Fix Nightly AudioLDM2PipelineFastTests ( #10556 )
...
* Fix Nightly AudioLDM2PipelineFastTests
* add phonemizer to setup extras test
* fix
* make style
2025-01-13 13:54:06 +00:00
hlky
50c81df4e7
Fix StableDiffusionInstructPix2PixPipelineSingleFileSlowTests ( #10557 )
2025-01-13 13:47:10 +00:00
Sayak Paul
edb8c1bce6
[Flux] Improve true cfg condition ( #10539 )
...
* improve flux true cfg condition
* add test
2025-01-12 18:33:34 +05:30
Sayak Paul
36acdd7517
[Tests] skip tests properly with unittest.skip() ( #10527 )
...
* skip tests properly.
* more
* more
2025-01-11 08:46:22 +05:30
Junyu Chen
e7db062e10
[DC-AE] support tiling for DC-AE ( #10510 )
...
* autoencoder_dc tiling
* add tiling and slicing support in SANA pipelines
* create variables for padding length because the line becomes too long
* add tiling and slicing support in pag SANA pipelines
* revert changes to tile size
* make style
* add vae tiling test
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2025-01-11 07:15:26 +05:30
Sayak Paul
9f06a0d1a4
[CI] Match remaining assertions from big runner ( #10521 )
...
* print
* remove print.
* print
* update slice.
* empty
2025-01-10 16:37:36 +05:30
Sayak Paul
a6f043a80f
[LoRA] allow big CUDA tests to run properly for LoRA (and others) ( #9845 )
...
* allow big lora tests to run on the CI.
* print
* print.
* print
* print
* print
* print
* more
* print
* remove print.
* remove print
* directly place on cuda.
* remove pipeline.
* remove
* fix
* fix
* spaces
* quality
* updates
* directly place flux controlnet pipeline on cuda.
* torch_device instead of cuda.
* style
* device placement.
* fixes
* add big gpu marker for mochi; rename test correctly
* address feedback
* fix
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2025-01-10 12:50:24 +05:30
Sayak Paul
daf9d0f119
[chore] remove prints from tests. ( #10505 )
...
remove prints from tests.
2025-01-09 14:19:43 +05:30
hlky
b13cdbb294
UNet2DModel mid_block_type ( #10469 )
2025-01-08 10:50:29 -10:00
AstraliteHeart
cb342b745a
Add AuraFlow GGUF support ( #10463 )
...
* Add support for loading AuraFlow models from GGUF
https://huggingface.co/city96/AuraFlow-v0.3-gguf
* Update AuraFlow documentation for GGUF, add GGUF tests and model detection.
* Address code review comments.
* Remove unused config.
---------
Co-authored-by: hlky <hlky@hlky.ac >
2025-01-08 13:23:12 +05:30
Aryan
71ad16b463
Add _no_split_modules to some models ( #10308 )
...
* set supports gradient checkpointing to true where necessary; add missing no split modules
* fix cogvideox tests
* update
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2025-01-08 06:34:19 +05:30
Aryan
811560b1d7
[LoRA] Support original format loras for HunyuanVideo ( #10376 )
...
* update
* fix make copies
* update
* add relevant markers to the integration test suite.
* add copied.
* fox-copies
* temporarily add print.
* directly place on CUDA as CPU isn't that big on the CIO.
* fixes to fuse_lora, aryan was right.
* fixes
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2025-01-07 13:18:57 +05:30
hlky
8f2253c58c
Add torch_xla and from_single_file to instruct-pix2pix ( #10444 )
...
* Add torch_xla and from_single_file to instruct-pix2pix
* StableDiffusionInstructPix2PixPipelineSingleFileSlowTests
* StableDiffusionInstructPix2PixPipelineSingleFileSlowTests
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2025-01-06 10:11:16 -10:00
Sayak Paul
d9d94e12f3
[LoRA] fix: lora unloading when using expanded Flux LoRAs. ( #10397 )
...
* fix: lora unloading when using expanded Flux LoRAs.
* fix argument name.
Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com >
* docs.
---------
Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com >
2025-01-06 08:35:05 -10:00
Sayak Paul
b5726358cf
[Tests] add slow and nightly markers to sd3 lora integation. ( #10458 )
...
add slow and nightly markers to sd3 lora integation.
2025-01-06 07:29:04 +05:30
Daniel Regado
68bd6934b1
IP-Adapter support for StableDiffusion3ControlNetPipeline ( #10363 )
...
* IP-Adapter support for `StableDiffusion3ControlNetPipeline`
* Update src/diffusers/pipelines/controlnet_sd3/pipeline_stable_diffusion_3_controlnet.py
Co-authored-by: hlky <hlky@hlky.ac >
---------
Co-authored-by: hlky <hlky@hlky.ac >
2025-01-02 10:02:32 -10:00
maxs-kan
44640c8358
Fix Flux multiple Lora loading bug ( #10388 )
...
* check for base_layer key in transformer state dict
* test_lora_expansion_works_for_absent_keys
* check
* Update tests/lora/test_lora_layers_flux.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* check
* test_lora_expansion_works_for_absent_keys/test_lora_expansion_works_for_extra_keys
* absent->extra
---------
Co-authored-by: hlky <hlky@hlky.ac >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2025-01-02 08:34:48 -10:00
Sayak Paul
1b202c5730
[LoRA] feat: support unload_lora_weights() for Flux Control. ( #10206 )
...
* feat: support unload_lora_weights() for Flux Control.
* tighten test
* minor
* updates
* meta device fixes.
2024-12-25 17:27:16 +05:30
Aryan
cd991d1e1a
Fix TorchAO related bugs; revert device_map changes ( #10371 )
...
* Revert "Add support for sharded models when TorchAO quantization is enabled (#10256 )"
This reverts commit 41ba8c0bf6spsayakpaul@gmail.com >
* Update tests/quantization/torchao/test_torchao.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* improve tests
* fix
* update correct slices
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-25 15:37:49 +05:30
Fanli Lin
023b0e0d55
[tests] fix AssertionError: Torch not compiled with CUDA enabled ( #10356 )
...
fix bug on xpu
2024-12-24 15:28:50 +00:00
Aryan
4b557132ce
[core] LTX Video 0.9.1 ( #10330 )
...
* update
* make style
* update
* update
* update
* make style
* single file related changes
* update
* fix
* update single file urls and docs
* update
* fix
2024-12-23 19:51:33 +05:30
Sayak Paul
851dfa30ae
[Tests] Fix more tests sayak ( #10359 )
...
* fixes to tests
* fixture
* fixes
2024-12-23 19:11:21 +05:30
Sayak Paul
ea1ba0ba53
[LoRA] test fix ( #10351 )
...
updates
2024-12-23 15:45:45 +05:30
Sayak Paul
c34fc34563
[Tests] QoL improvements to the LoRA test suite ( #10304 )
...
* misc lora test improvements.
* updates
* fixes to tests
2024-12-23 13:59:55 +05:30
Sayak Paul
76e2727b5c
[SANA LoRA] sana lora training tests and misc. ( #10296 )
...
* sana lora training tests and misc.
* remove push to hub
* Update examples/dreambooth/train_dreambooth_lora_sana.py
Co-authored-by: Aryan <aryan@huggingface.co >
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-23 12:35:13 +05:30
Aryan
02c777c065
[tests] Refactor TorchAO serialization fast tests ( #10271 )
...
refactor
2024-12-23 11:04:57 +05:30
Aryan
ffc0eaab6d
Bump minimum TorchAO version to 0.7.0 ( #10293 )
...
* bump min torchao version to 0.7.0
* update
2024-12-23 11:03:04 +05:30
Junsong Chen
b58868e6f4
[Sana bug] bug fix for 2K model config ( #10340 )
...
* fix the Positinoal Embedding bug in 2K model;
* Change the default model to the BF16 one for more stable training and output
* make style
* substract buffer size
* add compute_module_persistent_sizes
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-12-23 08:56:25 +05:30
hlky
be2070991f
Support Flux IP Adapter ( #10261 )
...
* Flux IP-Adapter
* test cfg
* make style
* temp remove copied from
* fix test
* fix test
* v2
* fix
* make style
* temp remove copied from
* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Move encoder_hid_proj to inside FluxTransformer2DModel
* merge
* separate encode_prompt, add copied from, image_encoder offload
* make
* fix test
* fix
* Update src/diffusers/pipelines/flux/pipeline_flux.py
* test_flux_prompt_embeds change not needed
* true_cfg -> true_cfg_scale
* fix merge conflict
* test_flux_ip_adapter_inference
* add fast test
* FluxIPAdapterMixin not test mixin
* Update pipeline_flux.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-21 17:49:58 +00:00
hlky
bf9a641f1a
Fix EMAModel test_from_pretrained ( #10325 )
2024-12-21 14:10:44 +00:00
Sayak Paul
bf6eaa8aec
[Tests] add integration tests for lora expansion stuff in Flux. ( #10318 )
...
add integration tests for lora expansion stuff in Flux.
2024-12-20 16:14:58 +05:30
Sayak Paul
17128c42a4
[LoRA] feat: support loading regular Flux LoRAs into Flux Control, and Fill ( #10259 )
...
* lora expansion with dummy zeros.
* updates
* fix working 🥳
* working.
* use torch.device meta for state dict expansion.
* tests
Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com >
* fixes
* fixes
* switch to debug
* fix
* Apply suggestions from code review
Co-authored-by: Aryan <aryan@huggingface.co >
* fix stuff
* docs
---------
Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-20 14:30:32 +05:30
Aryan
41ba8c0bf6
Add support for sharded models when TorchAO quantization is enabled ( #10256 )
...
* add sharded + device_map check
2024-12-19 15:42:20 -10:00
Daniel Regado
3191248472
[WIP] SD3.5 IP-Adapter Pipeline Integration ( #9987 )
...
* Added support for single IPAdapter on SD3.5 pipeline
---------
Co-authored-by: hlky <hlky@hlky.ac >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-12-19 14:48:18 -10:00
dg845
648d968cfc
Enable Gradient Checkpointing for UNet2DModel (New) ( #7201 )
...
* Port UNet2DModel gradient checkpointing code from #6718 .
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Vincent Neemie <92559302+VincentNeemie@users.noreply.github.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-19 14:45:45 -10:00
djm
b756ec6e80
unet's sample_size attribute is to accept tuple(h, w) in StableDiffusionPipeline ( #10181 )
2024-12-19 22:24:18 +00:00
Aryan
d8825e7697
Fix failing lora tests after HunyuanVideo lora ( #10307 )
...
fix
2024-12-20 02:35:41 +05:30
Shenghai Yuan
1826a1e7d3
[LoRA] Support HunyuanVideo ( #10254 )
...
* 1217
* 1217
* 1217
* update
* reverse
* add test
* update test
* make style
* update
* make style
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-19 16:22:20 +05:30
hlky
0ed09a17bb
Check correct model type is passed to from_pretrained ( #10189 )
...
* Check correct model type is passed to `from_pretrained`
* Flax, skip scheduler
* test_wrong_model
* Fix for scheduler
* Update tests/pipelines/test_pipelines.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* EnumMeta
* Flax
* scheduler in expected types
* make
* type object 'CLIPTokenizer' has no attribute '_PipelineFastTests__name'
* support union
* fix typing in kandinsky
* make
* add LCMScheduler
* 'LCMScheduler' object has no attribute 'sigmas'
* tests for wrong scheduler
* make
* update
* warning
* tests
* Update src/diffusers/pipelines/pipeline_utils.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* import FlaxSchedulerMixin
* skip scheduler
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-12-19 09:24:52 +00:00
Aryan
f781b8c30c
Hunyuan VAE tiling fixes and transformer docs ( #10295 )
...
* update
* udpate
* fix test
2024-12-19 10:28:10 +05:30
Aryan
f35a38725b
[tests] remove nullop import checks from lora tests ( #10273 )
...
remove nullop imports
2024-12-19 01:19:08 +05:30
Aryan
f66bd3261c
Rename Mochi integration test correctly ( #10220 )
...
rename integration test
2024-12-18 22:41:23 +05:30
Andrés Romero
83709d5a06
Flux Control(Depth/Canny) + Inpaint ( #10192 )
...
* flux_control_inpaint - failing test_flux_different_prompts
* removing test_flux_different_prompts?
* fix style
* fix from PR comments
* fix style
* reducing guidance_scale in demo
* Update src/diffusers/pipelines/flux/pipeline_flux_control_inpaint.py
Co-authored-by: hlky <hlky@hlky.ac >
* make
* prepare_latents is not copied from
* update docs
* typos
---------
Co-authored-by: affromero <ubuntu@ip-172-31-17-146.ec2.internal >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-18 09:14:16 +00:00
Sayak Paul
9408aa2dfc
[LoRA] feat: lora support for SANA. ( #10234 )
...
* feat: lora support for SANA.
* make fix-copies
* rename test class.
* attention_kwargs -> cross_attention_kwargs.
* Revert "attention_kwargs -> cross_attention_kwargs."
This reverts commit 23433bf9bcaryan@huggingface.co >
* style
* docs for attention_kwargs.
* remove lora_scale from pag pipeline.
* copy fix
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-12-18 08:22:31 +05:30
Aryan
1524781b88
[tests] Remove/rename unsupported quantization torchao type ( #10263 )
...
update
2024-12-17 21:43:15 +05:30
Dhruv Nair
e24941b2a7
[Single File] Add GGUF support ( #9964 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update src/diffusers/quantizers/gguf/utils.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update docs/source/en/quantization/gguf.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* update
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-17 16:09:37 +05:30
Aryan
ac86393487
[LoRA] Support LTX Video ( #10228 )
...
* add lora support for ltx
* add tests
* fix copied from comments
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-12-17 12:05:05 +05:30
Aryan
9f00c617a0
[core] TorchAO Quantizer ( #10009 )
...
* torchao quantizer
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-12-16 13:35:40 -10:00
Aryan
aace1f412b
[core] Hunyuan Video ( #10136 )
...
* copy transformer
* copy vae
* copy pipeline
* make fix-copies
* refactor; make original code work with diffusers; test latents for comparison generated with this commit
* move rope into pipeline; remove flash attention; refactor
* begin conversion script
* make style
* refactor attention
* refactor
* refactor final layer
* their mlp -> our feedforward
* make style
* add docs
* refactor layer names
* refactor modulation
* cleanup
* refactor norms
* refactor activations
* refactor single blocks attention
* refactor attention processor
* make style
* cleanup a bit
* refactor double transformer block attention
* update mochi attn proc
* use diffusers attention implementation in all modules; checkpoint for all values matching original
* remove helper functions in vae
* refactor upsample
* refactor causal conv
* refactor resnet
* refactor
* refactor
* refactor
* grad checkpointing
* autoencoder test
* fix scaling factor
* refactor clip
* refactor llama text encoding
* add coauthor
Co-Authored-By: "Gregory D. Hunkins" <greg@ollano.com >
* refactor rope; diff: 0.14990234375; reason and fix: create rope grid on cpu and move to device
Note: The following line diverges from original behaviour. We create the grid on the device, whereas
original implementation creates it on CPU and then moves it to device. This results in numerical
differences in layerwise debugging outputs, but visually it is the same.
* use diffusers timesteps embedding; diff: 0.10205078125
* rename
* convert
* update
* add tests for transformer
* add pipeline tests; text encoder 2 is not optional
* fix attention implementation for torch
* add example
* update docs
* update docs
* apply suggestions from review
* refactor vae
* update
* Apply suggestions from code review
Co-authored-by: hlky <hlky@hlky.ac >
* Update src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video.py
Co-authored-by: hlky <hlky@hlky.ac >
* Update src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video.py
Co-authored-by: hlky <hlky@hlky.ac >
* make fix-copies
* update
---------
Co-authored-by: "Gregory D. Hunkins" <greg@ollano.com >
Co-authored-by: hlky <hlky@hlky.ac >
2024-12-16 13:56:18 +05:30
