YiYi Xu
9e9ed353a2
fix loading sharded checkpoints from subfolder ( #8798 )
...
* fix load sharded checkpoints from subfolder{
* style
* os.path.join
* add a small test
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
2024-07-06 11:32:04 -10:00
Mathis Koroglu
3e0d128da7
Motion Model / Adapter versatility ( #8301 )
...
* Motion Model / Adapter versatility
- allow to use a different number of layers per block
- allow to use a different number of transformer per layers per block
- allow a different number of motion attention head per block
- use dropout argument in get_down/up_block in 3d blocks
* Motion Model added arguments renamed & refactoring
* Add test for asymmetric UNetMotionModel
2024-06-27 11:11:29 +05:30
Tolga Cangöz
c375903db5
Errata - Fix typos & improve contributing page ( #8572 )
...
* Fix typos & improve contributing page
* `make style && make quality`
* fix typos
* Fix typo
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-06-24 14:13:03 +05:30
YiYi Xu
c71c19c5e6
a few fix for shard checkpoints ( #8656 )
...
fix
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-06-21 12:50:58 +05:30
Marc Sun
96399c3ec6
Fix sharding when no device_map is passed ( #8531 )
...
* Fix sharding when no device_map is passed
* style
* add tests
* align
* add docstring
* format
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-06-18 05:47:23 -10:00
Sayak Paul
7d887118b9
[Core] support saving and loading of sharded checkpoints ( #7830 )
...
* feat: support saving a model in sharded checkpoints.
* feat: make loading of sharded checkpoints work.
* add tests
* cleanse the loading logic a bit more.
* more resilience while loading from the Hub.
* parallelize shard downloads by using snapshot_download()/
* default to a shard size.
* more fix
* Empty-Commit
* debug
* fix
* uality
* more debugging
* fix more
* initial comments from Benjamin
* move certain methods to loading_utils
* add test to check if the correct number of shards are present.
* add a test to check if loading of sharded checkpoints from the Hub is okay
* clarify the unit when passed as an int.
* use hf_hub for sharding.
* remove unnecessary code
* remove unnecessary function
* lucain's comments.
* fixes
* address high-level comments.
* fix test
* subfolder shenanigans./
* Update src/diffusers/utils/hub_utils.py
Co-authored-by: Lucain <lucainp@gmail.com >
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com >
* remove _huggingface_hub_version as not needed.
* address more feedback.
* add a test for local_files_only=True/
* need hf hub to be at least 0.23.2
* style
* final comment.
* clean up subfolder.
* deal with suffixes in code.
* _add_variant default.
* use weights_name_pattern
* remove add_suffix_keyword
* clean up downloading of sharded ckpts.
* don't return something special when using index.json
* fix more
* don't use bare except
* remove comments and catch the errors better
* fix a couple of things when using is_file()
* empty
---------
Co-authored-by: Lucain <lucainp@gmail.com >
2024-06-07 14:49:10 +05:30
Sayak Paul
a0542c1917
[LoRA] Remove legacy LoRA code and related adjustments ( #8316 )
...
* remove legacy code from load_attn_procs.
* finish first draft
* fix more.
* fix more
* add test
* add serialization support.
* fix-copies
* require peft backend for lora tests
* style
* fix test
* fix loading.
* empty
* address benjamin's feedback.
2024-06-05 08:15:30 +04:00
Sayak Paul
ba1bfac20b
[Core] Refactor IPAdapterPlusImageProjection a bit ( #7994 )
...
* use IPAdapterPlusImageProjectionBlock in IPAdapterPlusImageProjection
* reposition IPAdapterPlusImageProjection
* refactor complete?
* fix heads param retrieval.
* update test dict creation method.
2024-05-29 06:30:47 +05:30
Dhruv Nair
baab065679
Remove unnecessary single file tests for SD Cascade UNet ( #7996 )
...
update
2024-05-22 12:29:59 +05:30
Dhruv Nair
cb0f3b49cb
[Refactor] Better align from_single_file logic with from_pretrained ( #7496 )
...
* refactor unet single file loading a bit.
* retrieve the unet from create_diffusers_unet_model_from_ldm
* update
* update
* updae
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* tests
* update
* update
* update
* Update docs/source/en/api/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/loaders/single_file.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-05-09 19:00:19 +05:30
Sayak Paul
3fd31eef51
[Core] introduce _no_split_modules to ModelMixin ( #6396 )
...
* introduce _no_split_modules.
* unnecessary spaces.
* remove unnecessary kwargs and style
* fix: accelerate imports.
* change to _determine_device_map
* add the blocks that have residual connections.
* add: CrossAttnUpBlock2D
* add: testin
* style
* line-spaces
* quality
* add disk offload test without safetensors.
* checking disk offloading percentages.
* change model split
* add: utility for checking multi-gpu requirement.
* model parallelism test
* splits.
* splits.
* splits
* splits.
* splits.
* splits.
* offload folder to test_disk_offload_with_safetensors
* add _no_split_modules
* fix-copies
2024-04-30 08:46:51 +05:30
Sayak Paul
b833d0fc80
[Tests] mark UNetControlNetXSModelTests::test_forward_no_control to be flaky ( #7771 )
...
decorate UNetControlNetXSModelTests::test_forward_no_control with is_flaky
2024-04-25 07:29:04 +05:30
Fabio Rigano
b5c8b555d7
Move IP Adapter Face ID to core ( #7186 )
...
* Switch to peft and multi proj layers
* Move Face ID loading and inference to core
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-04-18 14:13:27 -10:00
UmerHA
fda1531d8a
Fixing implementation of ControlNet-XS ( #6772 )
...
* CheckIn - created DownSubBlocks
* Added extra channels, implemented subblock fwd
* Fixed connection sizes
* checkin
* Removed iter, next in forward
* Models for SD21 & SDXL run through
* Added back pipelines, cleared up connections
* Cleaned up connection creation
* added debug logs
* updated logs
* logs: added input loading
* Update umer_debug_logger.py
* log: Loading hint
* Update umer_debug_logger.py
* added logs
* Changed debug logging
* debug: added more logs
* Fixed num_norm_groups
* Debug: Logging all of SDXL input
* Update umer_debug_logger.py
* debug: updated logs
* checkim
* Readded tests
* Removed debug logs
* Fixed Slow Tests
* Added value ckecks | Updated model_cpu_offload_seq
* accelerate-offloading works ; fast tests work
* Made unet & addon explicit in controlnet
* Updated slow tests
* Added dtype/device to ControlNetXS
* Filled in test model paths
* Added image_encoder/feature_extractor to XL pipe
* Fixed fast tests
* Added comments and docstrings
* Fixed copies
* Added docs ; Updates slow tests
* Moved changes to UNetMidBlock2DCrossAttn
* tiny cleanups
* Removed stray prints
* Removed ip adapters + freeU
- Removed ip adapters + freeU as they don't make sense for ControlNet-XS
- Fixed imports of UNet components
* Fixed test_save_load_float16
* Make style, quality, fix-copies
* Changed loading/saving API for ControlNetXS
- Changed loading/saving API for ControlNetXS
- other small fixes
* Removed ControlNet-XS from research examples
* Make style, quality, fix-copies
* Small fixes
- deleted ControlNetXSModel.init_original
- added time_embedding_mix to StableDiffusionControlNetXSPipeline .from_pretrained / StableDiffusionXLControlNetXSPipeline.from_pretrained
- fixed copy hints
* checkin May 11 '23
* CheckIn Mar 12 '24
* Fixed tests for SD
* Added tests for UNetControlNetXSModel
* Fixed SDXL tests
* cleanup
* Delete Pipfile
* CheckIn Mar 20
Started replacing sub blocks by `ControlNetXSCrossAttnDownBlock2D` and `ControlNetXSCrossAttnUplock2D`
* check-in Mar 23
* checkin 24 Mar
* Created init for UNetCnxs and CnxsAddon
* CheckIn
* Made from_modules, from_unet and no_control work
* make style,quality,fix-copies & small changes
* Fixed freezing
* Added gradient ckpt'ing; fixed tests
* Fix slow tests(+compile) ; clear naming confusion
* Don't create UNet in init ; removed class_emb
* Incorporated review feedback
- Deleted get_base_pipeline / get_controlnet_addon for pipes
- Pipes inherit from StableDiffusionXLPipeline
- Made module dicts for cnxs-addon's down/mid/up classes
- Added support for qkv fusion and freeU
* Make style, quality, fix-copies
* Implemented review feedback
* Removed compatibility check for vae/ctrl embedding
* make style, quality, fix-copies
* Delete Pipfile
* Integrated review feedback
- Importing ControlNetConditioningEmbedding now
- get_down/mid/up_block_addon now outside class
- renamed `do_control` to `apply_control`
* Reduced size of test tensors
For this, added `norm_num_groups` as parameter everywhere
* Renamed cnxs-`Addon` to cnxs-`Adapter`
- `ControlNetXSAddon` -> `ControlNetXSAdapter`
- `ControlNetXSAddonDownBlockComponents` -> `DownBlockControlNetXSAdapter`, and similarly for mid/up
- `get_mid_block_addon` -> `get_mid_block_adapter`, and similarly for mid/up
* Fixed save_pretrained/from_pretrained bug
* Removed redundant code
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-04-16 21:56:20 +05:30
Sayak Paul
1c60e094de
[Tests] reduce block sizes of UNet and VAE tests ( #7560 )
...
* reduce block sizes for unet1d.
* reduce blocks for unet_2d.
* reduce block size for unet_motion
* increase channels.
* correctly increase channels.
* reduce number of layers in unet2dconditionmodel tests.
* reduce block sizes for unet2dconditionmodel tests
* reduce block sizes for unet3dconditionmodel.
* fix: test_feed_forward_chunking
* fix: test_forward_with_norm_groups
* skip spatiotemporal tests on MPS.
* reduce block size in AutoencoderKL.
* reduce block sizes for vqmodel.
* further reduce block size.
* make style.
* Empty-Commit
* reduce sizes for ConsistencyDecoderVAETests
* further reduction.
* further block reductions in AutoencoderKL and AssymetricAutoencoderKL.
* massively reduce the block size in unet2dcontionmodel.
* reduce sizes for unet3d
* fix tests in unet3d.
* reduce blocks further in motion unet.
* fix: output shape
* add attention_head_dim to the test configuration.
* remove unexpected keyword arg
* up a bit.
* groups.
* up again
* fix
2024-04-05 10:08:32 +05:30
M. Tolga Cangöz
e97a633b63
Update access of configuration attributes ( #7343 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-18 08:53:29 -10:00
Dhruv Nair
4974b84564
Update Cascade Tests ( #7324 )
...
* update
* update
* update
2024-03-14 20:51:22 +05:30
Dhruv Nair
ed224f94ba
Add single file support for Stable Cascade ( #7274 )
...
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-13 08:37:31 +05:30
Aryan
bb1b76d3bf
IPAdapterTesterMixin ( #6862 )
...
* begin IPAdapterTesterMixin
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-02-22 14:25:33 -10:00
Sayak Paul
30e5e81d58
change to 2024 in the license ( #6902 )
...
change to 2024
2024-02-08 08:19:31 -10:00
Sayak Paul
ec9840a5db
[Refactor] harmonize the module structure for models in tests ( #6738 )
...
* harmonize the module structure for models in tests
* make the folders modules.
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-02-01 14:23:39 +05:30
