Leo Jiang
94643fac8a
[bugfix] bugfix for npu free memory ( #9640 )
...
* Improve NPU performance
* Improve NPU performance
* Improve NPU performance
* Improve NPU performance
* [bugfix] bugfix for npu free memory
* [bugfix] bugfix for npu free memory
* [bugfix] bugfix for npu free memory
---------
Co-authored-by: 蒋硕 <jiangshuo9@h-partners.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-25 23:35:19 +09:00
Zhiyang Shen
435f6b7e47
[Docs] fix docstring typo in SD3 pipeline ( #9765 )
...
* fix docstring typo in SD3 pipeline
* fix docstring typo in SD3 pipeline
2024-10-25 16:33:35 +05:30
Sayak Paul
1d1e1a2888
Some minor updates to the nightly and push workflows ( #9759 )
...
* move lora integration tests to nightly./
* remove slow marker in the workflow where not needed.
2024-10-24 23:49:09 +09:00
Rachit Shah
24c7d578ba
config attribute not foud error for FluxImagetoImage Pipeline for multi controlnet solved ( #9586 )
...
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-23 10:33:29 -10:00
Linoy Tsaban
bfa0aa4ff2
[SD3-5 dreambooth lora] update model cards ( #9749 )
...
* improve readme
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-23 23:16:53 +03:00
Álvaro Somoza
ab1b7b2080
[Official callbacks] SDXL Controlnet CFG Cutoff ( #9311 )
...
* initial proposal
* style
2024-10-23 13:21:56 -03:00
Fanli Lin
9366c8f84b
fix bug in require_accelerate_version_greater ( #9746 )
...
fix bug
2024-10-23 10:01:33 +05:30
Sayak Paul
e45c25d03a
post-release 0.31.0 ( #9742 )
...
* post-release
* style
2024-10-22 20:42:30 +05:30
Dhruv Nair
76c00c7236
is_safetensors_compatible fix ( #9741 )
...
update
2024-10-22 19:35:03 +05:30
Dhruv Nair
0d9d98fe5f
Fix typos ( #9739 )
...
* update
* update
* update
* update
* update
* update
2024-10-22 16:12:28 +05:30
Sayak Paul
60ffa84253
[bitsandbbytes] follow-ups ( #9730 )
...
* bnb follow ups.
* add a warning when dtypes mismatch.
* fx-copies
* clear cache.
* check_if_quantized_param
* add a check on shape.
* updates
* docs
* improve readability.
* resources.
* fix
2024-10-22 16:00:05 +05:30
Álvaro Somoza
0f079b932d
[Fix] Using sharded checkpoints with gated repositories ( #9737 )
...
fix
2024-10-22 01:33:52 -03:00
Yu Zheng
b0ffe92230
Update sd3 controlnet example ( #9735 )
...
* use make_image_grid in diffusers.utils
* use checkpoint on the Hub
2024-10-22 09:02:16 +05:30
Tolga Cangöz
1b64772b79
Fix schedule_shifted_power usage in 🪆 Matryoshka Diffusion Models ( #9723 )
...
* [matryoshka.py] Add schedule_shifted_power attribute and update get_schedule_shifted method
2024-10-21 14:23:50 -10:00
YiYi Xu
2d280f173f
fix singlestep dpm tests ( #9716 )
...
fix
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-21 13:27:01 -10:00
G.O.D
63a0c9e5f7
[bugfix] reduce float value error when adding noise ( #9004 )
...
* Update train_controlnet.py
reduce float value error for bfloat16
* Update train_controlnet_sdxl.py
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-10-21 13:26:05 -10:00
YiYi Xu
e2d037bbf1
minor doc/test update ( #9734 )
...
* update some docs and tests!
---------
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
2024-10-21 13:06:13 -10:00
timdalxx
bcd61fd349
[docs] add docstrings in pipline_stable_diffusion.py ( #9590 )
...
* fix the issue on flux dreambooth lora training
* update : origin main code
* docs: update pipeline_stable_diffusion docstring
* docs: update pipeline_stable_diffusion docstring
* Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* fix: style
* fix: style
* fix: copies
* make fix-copies
* remove extra newline
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-21 09:39:20 -07:00
Sayak Paul
d27ecc5960
[Docs] docs to xlabs controlnets. ( #9688 )
...
* docs to xlabs controlnets.
Co-authored-by: Anzhella Pankratova <son0shad@gmail.com >
* Apply suggestions from code review
Co-authored-by: Anzhella Pankratova <54744846+Anghellia@users.noreply.github.com >
---------
Co-authored-by: Anzhella Pankratova <son0shad@gmail.com >
Co-authored-by: Anzhella Pankratova <54744846+Anghellia@users.noreply.github.com >
2024-10-21 09:38:22 -07:00
Chenyu Li
6b915672f4
Fix typo in cogvideo pipeline ( #9722 )
...
Fix type in cogvideo pipeline
2024-10-21 21:39:39 +05:30
Sayak Paul
b821f006d0
[Quantization] Add quantization support for bitsandbytes ( #9213 )
...
* quantization config.
* fix-copies
* fix
* modules_to_not_convert
* add bitsandbytes utilities.
* make progress.
* fixes
* quality
* up
* up
rotary embedding refactor 2: update comments, fix dtype for use_real=False (#9312 )
fix notes and dtype
up
up
* minor
* up
* up
* fix
* provide credits where due.
* make configurations work.
* fixes
* fix
* update_missing_keys
* fix
* fix
* make it work.
* fix
* provide credits to transformers.
* empty commit
* handle to() better.
* tests
* change to bnb from bitsandbytes
* fix tests
fix slow quality tests
SD3 remark
fix
complete int4 tests
add a readme to the test files.
add model cpu offload tests
warning test
* better safeguard.
* change merging status
* courtesy to transformers.
* move upper.
* better
* make the unused kwargs warning friendlier.
* harmonize changes with https://github.com/huggingface/transformers/pull/33122
* style
* trainin tests
* feedback part i.
* Add Flux inpainting and Flux Img2Img (#9135 )
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Update `UNet2DConditionModel`'s error messages (#9230 )
* refactor
[CI] Update Single file Nightly Tests (#9357 )
* update
* update
feedback.
improve README for flux dreambooth lora (#9290 )
* improve readme
* improve readme
* improve readme
* improve readme
fix one uncaught deprecation warning for accessing vae_latent_channels in VaeImagePreprocessor (#9372 )
deprecation warning vae_latent_channels
add mixed int8 tests and more tests to nf4.
[core] Freenoise memory improvements (#9262 )
* update
* implement prompt interpolation
* make style
* resnet memory optimizations
* more memory optimizations; todo: refactor
* update
* update animatediff controlnet with latest changes
* refactor chunked inference changes
* remove print statements
* update
* chunk -> split
* remove changes from incorrect conflict resolution
* remove changes from incorrect conflict resolution
* add explanation of SplitInferenceModule
* update docs
* Revert "update docs"
This reverts commit c55a50a271#9135 )"
This reverts commit 5799954dd459462357+stevhliu@users.noreply.github.com >
* contribution guide.
* changes
* empty
* fix tests
* harmonize with https://github.com/huggingface/transformers/pull/33546 .
* numpy_cosine_distance
* config_dict modification.
* remove if config comment.
* note for load_state_dict changes.
* float8 check.
* quantizer.
* raise an error for non-True low_cpu_mem_usage values when using quant.
* low_cpu_mem_usage shenanigans when using fp32 modules.
* don't re-assign _pre_quantization_type.
* make comments clear.
* remove comments.
* handle mixed types better when moving to cpu.
* add tests to check if we're throwing warning rightly.
* better check.
* fix 8bit test_quality.
* handle dtype more robustly.
* better message when keep_in_fp32_modules.
* handle dtype casting.
* fix dtype checks in pipeline.
* fix warning message.
* Update src/diffusers/models/modeling_utils.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* mitigate the confusing cpu warning
---------
Co-authored-by: Vishnu V Jaddipal <95531133+Gothos@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-21 10:11:57 +05:30
Aryan
24281f8036
make deps_table_update to fix CI tests (#9720 )
...
* update
* dummy change to trigger CI; will revert
* no deps peft
* np deps
* todo
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-21 09:58:26 +05:30
Sayak Paul
2a1d2f6218
[Docker] pin torch versions in the dockerfiles. ( #9721 )
...
* pin torch versions in the dockerfiles.
* more
2024-10-20 10:44:09 +05:30
Aryan
56d6d21bae
[CI] pin max torch version to fix CI errors ( #9709 )
...
* pin max torch version
* update
* Update setup.py
2024-10-20 01:50:56 +05:30
hlky
89565e9171
Add prompt scheduling callback to community scripts ( #9718 )
2024-10-19 14:22:22 -03:00
bonlime
5d3e7bdaaa
Fix bug in Textual Inversion Unloading ( #9304 )
...
* Update textual_inversion.py
* add unload test
* add comment
* fix style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Your Name <you@example.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-19 02:37:32 -10:00
Linoy Tsaban
2541d141d5
[advanced flux lora script] minor updates to readme ( #9705 )
...
* fix arg naming
* fix arg naming
* fix arg naming
* fix arg naming
2024-10-18 15:35:44 +03:00
Aryan
5704376d03
[refactor] DiffusionPipeline.download ( #9557 )
...
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-10-17 12:38:06 -10:00
Linoy Tsaban
9a7f824645
[Flux] Add advanced training script + support textual inversion inference ( #9434 )
...
* add ostris trainer to README & add cache latents of vae
* add ostris trainer to README & add cache latents of vae
* style
* readme
* add test for latent caching
* add ostris noise scheduler
9ee1ef2a0a/toolkit/samplers/custom_flowmatch_sampler.py (L95)https://github.com/huggingface/diffusers/pull/9160
* revert changes to canonical script to keep it separate from https://github.com/huggingface/diffusers/pull/9160
* style
* remove redundant line and change code block placement to align with logic
* add initializer_token arg
* add transformer frac for range support from pure textual inversion to the orig pivotal tuning
* support pure textual inversion - wip
* adjustments to support pure textual inversion and transformer optimization in only part of the epochs
* fix logic when using initializer token
* fix pure_textual_inversion_condition
* fix ti/pivotal loading of last validation run
* remove embeddings loading for ti in final training run (to avoid adding huggingface hub dependency)
* support pivotal for t5
* adapt pivotal for T5 encoder
* adapt pivotal for T5 encoder and support in flux pipeline
* t5 pivotal support + support fo pivotal for clip only or both
* fix param chaining
* fix param chaining
* README first draft
* readme
* readme
* readme
* style
* fix import
* style
* add fix from https://github.com/huggingface/diffusers/pull/9419
* add to readme, change function names
* te lr changes
* readme
* change concept tokens logic
* fix indices
* change arg name
* style
* dummy test
* revert dummy test
* reorder pivoting
* add warning in case the token abstraction is not the instance prompt
* experimental - wip - specific block training
* fix documentation and token abstraction processing
* remove transformer block specification feature (for now)
* style
* fix copies
* fix indexing issue when --initializer_concept has different amounts
* add if TextualInversionLoaderMixin to all flux pipelines
* style
* fix import
* fix imports
* address review comments - remove necessary prints & comments, use pin_memory=True, use free_memory utils, unify warning and prints
* style
* logger info fix
* make lora target modules configurable and change the default
* make lora target modules configurable and change the default
* style
* make lora target modules configurable and change the default, add notes to readme
* style
* add tests
* style
* fix repo id
* add updated requirements for advanced flux
* fix indices of t5 pivotal tuning embeddings
* fix path in test
* remove `pin_memory`
* fix filename of embedding
* fix filename of embedding
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-17 12:22:11 +03:00
Aryan
d9029f2c59
[tests] fix name and unskip CogI2V integration test ( #9683 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-16 16:28:19 +05:30
Aryan
d204e53291
[core] improve VAE encode/decode framewise batching ( #9684 )
...
* update
* apply suggestions from review
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-16 16:25:41 +05:30
Aryan
8cabd4a0db
[pipeline] CogVideoX-Fun Control ( #9671 )
...
* cogvideox-fun control
* make style
* make fix-copies
* karras schedulers
* Update src/diffusers/pipelines/cogvideo/pipeline_cogvideox_fun_control.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/api/pipelines/cogvideox.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* apply suggestions from review
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-16 16:21:09 +05:30
Jongho Choi
5783286d2b
[peft] simple update when unscale ( #9689 )
...
Update peft_utils.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-16 16:10:19 +05:30
Linoy Tsaban
ee4ab23892
[SD3 dreambooth-lora training] small updates + bug fixes ( #9682 )
...
* add latent caching + smol updates
* update license
* replace with free_memory
* add --upcast_before_saving to allow saving transformer weights in lower precision
* fix models to accumulate
* fix mixed precision issue as proposed in https://github.com/huggingface/diffusers/pull/9565
* smol update to readme
* style
* fix caching latents
* style
* add tests for latent caching
* style
* fix latent caching
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-16 11:13:37 +03:00
Sayak Paul
cef4f65cf7
[LoRA] log a warning when there are missing keys in the LoRA loading. ( #9622 )
...
* log a warning when there are missing keys in the LoRA loading.
* handle missing keys and unexpected keys better.
* add tests
* fix-copies.
* updates
* tests
* concat warning.
* Add Differential Diffusion to Kolors (#9423 )
* Added diff diff support for kolors img2img
* Fized relative imports
* Fized relative imports
* Added diff diff support for Kolors
* Fized import issues
* Added map
* Fized import issues
* Fixed naming issues
* Added diffdiff support for Kolors img2img pipeline
* Removed example docstrings
* Added map input
* Updated latents
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* Updated `original_with_noise`
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* Improved code quality
---------
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
* FluxMultiControlNetModel (#9647 )
* tests
* Update src/diffusers/loaders/lora_pipeline.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* fix
---------
Co-authored-by: M Saqlain <118016760+saqlain2204@users.noreply.github.com >
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com >
Co-authored-by: hlky <hlky@hlky.ac >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-16 07:46:12 +05:30
Charchit Sharma
29a2c5d1ca
Resolves [BUG] 'GatheredParameters' object is not callable ( #9614 )
...
* gatherparams bug
* calling context lib object
* fix
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-10-16 06:44:10 +05:30
glide-the
0d935df67d
Docs: CogVideoX ( #9578 )
...
* CogVideoX docs
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-15 14:41:56 -10:00
YiYi Xu
3e9a28a8a1
[authored by @Anghellia) Add support of Xlabs Controlnets #9638 ( #9687 )
...
* Add support of Xlabs Controlnets
---------
Co-authored-by: Anzhella Pankratova <son0shad@gmail.com >
2024-10-15 12:10:45 -10:00
Aryan
2ffbb88f1c
[training] CogVideoX-I2V LoRA ( #9482 )
...
* update
* update
* update
* update
* update
* add coauthor
Co-Authored-By: yuan-shenghai <963658029@qq.com >
* add coauthor
Co-Authored-By: Shenghai Yuan <140951558+SHYuanBest@users.noreply.github.com >
* update
Co-Authored-By: yuan-shenghai <963658029@qq.com >
* update
---------
Co-authored-by: yuan-shenghai <963658029@qq.com >
Co-authored-by: Shenghai Yuan <140951558+SHYuanBest@users.noreply.github.com >
2024-10-16 02:07:07 +05:30
Ahnjj_DEV
d40da7b68a
Fix some documentation in ./src/diffusers/models/adapter.py ( #9591 )
...
* Fix some documentation in ./src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/adapter.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* run make style
* make style & fix
* make style : 0.1.5 version ruff
* revert changes to examples
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-10-15 10:27:39 -07:00
wony617
a3e8d3f7de
[docs] refactoring docstrings in models/embeddings_flax.py ( #9592 )
...
* [docs] refactoring docstrings in `models/embeddings_flax.py`
* Update src/diffusers/models/embeddings_flax.py
* make style
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-10-15 19:15:14 +05:30
wony617
fff4be8e23
[docs] refactoring docstrings in community/hd_painter.py ( #9593 )
...
* [docs] refactoring docstrings in community/hd_painter.py
* Update examples/community/hd_painter.py
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
* make style
---------
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-10-15 18:50:12 +05:30
Jiwook Han
355bb641e3
[doc] Fix some docstrings in src/diffusers/training_utils.py ( #9606 )
...
* refac: docstrings in training_utils.py
* fix: manual edits
* run make style
* add docstring at cast_training_params
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-10-15 18:47:52 +05:30
Charchit Sharma
92d2baf643
refactor image_processor.py file ( #9608 )
...
* refactor image_processor file
* changes as requested
* +1 edits
* quality fix
* indent issue
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-15 17:20:33 +05:30
0x名無し
dccf39f01e
Dreambooth lora flux bug 3dtensor to 2dtensor ( #9653 )
...
* fixed issue #9350 , Tensor is deprecated
* ran make style
2024-10-15 17:18:13 +05:30
Sayak Paul
99d87474fd
[Chore] fix import of EntryNotFoundError. ( #9676 )
...
fix import of EntryNotFoundError.
2024-10-15 14:07:08 +05:30
Robin
79b118e863
[Fix] when run load pretain with local_files_only, local variable 'cached_folder' referenced before assignment ( #9376 )
...
Fix local variable 'cached_folder' referenced before assignment in hub_utils.py
Fix when use `local_files_only=True` with `subfolder`, local variable 'cached_folder' referenced before assignment issue.
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-14 20:49:36 -10:00
hlky
9d0616189e
Slight performance improvement to Euler, EDMEuler, FlowMatchHeun, KDPM2Ancestral ( #9616 )
...
* Slight performance improvement to Euler
* Slight performance improvement to EDMEuler
* Slight performance improvement to FlowMatchHeun
* Slight performance improvement to KDPM2Ancestral
* Update KDPM2AncestralDiscreteSchedulerTest
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-14 19:34:25 -10:00
hlky
5f0df17703
Refactor SchedulerOutput and add pred_original_sample in DPMSolverSDE, Heun, KDPM2Ancestral and KDPM2 ( #9650 )
...
Refactor SchedulerOutput and add pred_original_sample
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-14 18:11:01 -10:00
hlky
957e5cabff
Convert list/tuple of HunyuanDiT2DControlNetModel to HunyuanDiT2DMultiControlNetModel ( #9651 )
...
Convert list/tuple of HunyuanDiT2DControlNetModel to HunyuanDiT2DMultiControlNetModel
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-10-14 18:09:30 -10:00
