AI/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-01-27 17:22:53 +03:00
Code Activity
4,743 Commits 740 Branches 94 Tags
074e12358bc17e7dbe111ea4f62f05dbae8a49d5
Commit Graph

4743 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
SkyCol
074e12358b Add prompt about wandb in examples/dreambooth/readme. (#10014) 
Add files via upload
 2024-11-25 18:42:06 +05:30
Sayak Paul
047bf49291 [Docs] add: missing pipelines from the spec. (#10005) 
add: missing pipelines from the spec.
 2024-11-25 00:27:59 -10:00
Linoy Tsaban
c4b5d2ff6b [SD3 dreambooth lora] smol fix to checkpoint saving (#9993) 
* smol change to fix checkpoint saving & resuming (as done in train_dreambooth_sd3.py)

* style

* modify comment to explain reasoning behind hidden size check
 2024-11-24 18:51:06 +02:00
Aryan
7ac6e286ee Flux Fill, Canny, Depth, Redux (#9985) 
* update

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-23 01:41:25 -10:00
hlky
b5fd6f13f5 ControlNet from_single_file when already converted (#9978) 
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-11-22 17:52:52 +05:30
Fanli Lin
64b3e0f539 make pipelines tests device-agnostic (part1) (#9399) 
* enable on xpu

* add 1 more

* add one more

* enable more

* add 1 more

* add more

* enable 1

* enable more cases

* enable

* enable

* update comment

* one more

* enable 1

* add more cases

* enable xpu

* add one more caswe

* add more cases

* add 1

* add more

* add more cases

* add case

* enable

* add more

* add more

* add more

* enbale more

* add more

* update code

* update test marker

* add skip back

* update comment

* remove single files

* remove

* style

* add

* revert

* reformat

* update decorator

* update

* update

* update

* Update tests/pipelines/deepfloyd_if/test_if.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update src/diffusers/utils/testing_utils.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update tests/pipelines/animatediff/test_animatediff_controlnet.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update tests/pipelines/animatediff/test_animatediff.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update tests/pipelines/animatediff/test_animatediff_controlnet.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* update float16

* no unitest.skipt

* update

* apply style check

* reapply format

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-11-22 15:32:54 +05:30
Sayak Paul
2e86a3f023 [Tests] skip nan lora tests on PyTorch 2.5.1 CPU. (#9975) 
* skip nan lora tests on PyTorch 2.5.1 CPU.

* cog

* use xfail

* correct xfail

* add condition

* tests
 2024-11-22 12:45:21 +05:30
Aryan
cd6ca9df29 Fix prepare latent image ids and vae sample generators for flux (#9981) 
* fix

* update expected slice
 2024-11-21 13:02:31 +05:30
YiYi Xu
e564abe292 fix controlnet module refactor (#9968) 
* fix
 2024-11-20 13:11:39 -10:00
raulmosa
3139d39fa7 Update handle single blocks on _convert_xlabs_flux_lora_to_diffusers (#9915) 
* Update handle single blocks on _convert_xlabs_flux_lora_to_diffusers to fix bug on updating keys and old_state_dict


---------

Co-authored-by: raul_ar <raul.moreno.salinas@autoretouch.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-20 12:53:20 -10:00
linjiapro
12358622e5 Improve control net block index for sd3 (#9758) 
* improve control net index

---------

Co-authored-by: YiYi Xu <yixu310@gmail.com>
 2024-11-20 12:45:18 -10:00
Sayak Paul
805aa93789 [LoRA] enable LoRA for Mochi-1 (#9943) 
* feat: add lora support to Mochi-1.
 2024-11-20 12:07:04 -10:00
Dhruv Nair
f6f7afa1d7 Flux latents fix (#9929) 
* update

* update

* update

* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-20 17:30:17 +05:30
hlky
637e2302ac Fix beta and exponential sigmas + add tests (#9954) 
* Fix beta and exponential sigmas + add tests

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-20 01:20:34 -10:00
Bagheera
99c0483b67 add skip_layers argument to SD3 transformer model class (#9880) 
* add skip_layers argument to SD3 transformer model class

* add unit test for skip_layers in stable diffusion 3

* sd3: pipeline should support skip layer guidance

* up

---------

Co-authored-by: bghira <bghira@users.github.com>
Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-11-19 15:22:54 -05:00
Parag Ekbote
cc7d88f247 Move IP Adapter Scripts to research project (#9960) 
* Move files to research-projects.

* docs: add IP Adapter training instructions

* Delete venv

* Update examples/ip_adapter/tutorial_train_sdxl.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Cherry-picked commits and re-moved files
to research_projects.

* make style.

* Update toctree and delete ip_adapter.

* Nit Fix

* Fix nit.

* Fix nit.

* Create training script for single GPU and set
model format to .safetensors

* Add sample inference script and restore _toctree

* Restore toctree.yaml

* fix spacing.

* Update toctree.yaml

---------

Co-authored-by: AMohamedAakhil <a.aakhilmohamed@gmail.com>
Co-authored-by: BootesVoid <78485654+AMohamedAakhil@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-19 10:37:22 -08:00
Dhruv Nair
ea40933f36 [CI] Unpin torch<2.5 in CI (#9961) 
* update

* update
 2024-11-19 18:50:46 +05:30
Aryan
0583a8d12a Make CogVideoX RoPE implementation consistent (#9963) 
* update cogvideox rope implementation

* apply suggestions from review
 2024-11-19 17:40:38 +05:30
Sayak Paul
7d0b9c4d4e [LoRA] feat: save_lora_adapter() (#9862) 
* feat: save_lora_adapter.
 2024-11-18 21:03:38 -10:00
Linoy Tsaban
acf479bded [advanced flux training] bug fix + reduce memory cost as in #9829 (#9838) 
* memory improvement as done here: https://github.com/huggingface/diffusers/pull/9829

* fix bug

* fix bug

* style

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-19 08:43:36 +05:30
Parag Ekbote
03bf77c4af Notebooks for Community Scripts-2 (#9952) 
4 Notebooks for Community Scripts and minor
script improvements.
 2024-11-18 12:58:57 -08:00
Yuxuan.Zhang
3b2830618d CogVideoX 1.5 (#9877) 
* CogVideoX1_1PatchEmbed test

* 1360 * 768

* refactor

* make style

* update docs

* add modeling tests for cogvideox 1.5

* update

* make fix-copies

* add ofs embed(for convert)

* add ofs embed(for convert)

* more resolution for cogvideox1.5-5b-i2v

* use even number of latent frames only

* update pipeline implementations

* make style

* set patch_size_t as None by default

* #skip frames 0

* refactor

* make style

* update docs

* fix ofs_embed

* update docs

* invert_scale_latents

* update

* fix

* Update docs/source/en/api/pipelines/cogvideox.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/cogvideox.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/cogvideox.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/api/pipelines/cogvideox.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update src/diffusers/models/transformers/cogvideox_transformer_3d.py

* update conversion script

* remove copied from

* fix test

* Update docs/source/en/api/pipelines/cogvideox.md

* Update docs/source/en/api/pipelines/cogvideox.md

* Update docs/source/en/api/pipelines/cogvideox.md

* Update docs/source/en/api/pipelines/cogvideox.md

---------

Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-11-19 00:56:34 +05:30
Grant Sherrick
c3c94fe71b Add server example (#9918) 
* Add server example.

* Minor updates to README.

* Add fixes after local testing.

* Apply suggestions from code review

Updates to README from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* More doc updates.

* Maybe this will work to build the docs correctly?

* Fix style issues.

* Fix toc.

* Minor reformatting.

* Move docs to proper loc.

* Fix missing tick.

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Sync docs changes back to README.

* Very minor update to docs to add space.

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2024-11-18 09:26:13 -08:00
Parag Ekbote
365a938884 Fixed Nits in Docs and Example Script (#9940) 
Fixed nits in docs and example script.

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-18 09:03:22 -08:00
ちくわぶ
345907f32d Add all AttnProcessor classes in AttentionProcessor type (#9909) 
Add all AttnProcessor in `AttentionProcessor` type
 2024-11-18 16:18:12 +09:00
_
07d0fbf3ec Correct pipeline_output.py to the type Mochi (#9945) 
Correct pipeline_output.py
 2024-11-18 08:40:06 +09:00
Heavenn
1d2204d3a0 Modify apply_overlay for inpainting with padding_mask_crop (Inpainting area: "Only Masked") (#8793) 
* Modify apply_overlay for inpainting

* style

---------

Co-authored-by: root <root@debian>
Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-11-17 12:14:13 +09:00
高佳宝
d38c50c8dd Update ip_adapter.py (#8882) 
update comments of load_ip_adapter function
 2024-11-17 06:54:13 +09:00
Parag Ekbote
e255920719 Move Wuerstchen Dreambooth to research_projects (#9935) 
update file paths to research_projects folder.

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-16 18:56:16 +05:30
Pakkapon Phongthawee
40ab1c03f3 add depth controlnet sd3 pre-trained checkpoints to docs (#9937) 2024-11-16 18:36:01 +05:30
Sam
5c94937dc7 Update pipeline_flux_img2img.py (#9928) 
* Update pipeline_flux_img2img.py

Added FromSingleFileMixin to this pipeline loader like the other FLUX pipelines.

* Update pipeline_flux_img2img.py

typo

* modified:   src/diffusers/pipelines/flux/pipeline_flux_img2img.py
 2024-11-14 17:58:14 -03:00
Benjamin Paine
d74483c47a Fix Progress Bar Updates in SD 1.5 PAG Img2Img pipeline (#9925) 
fix progress bar updates in SD 1.5 PAG Img2Img pipeline
 2024-11-14 16:40:20 -03:00
Parag Ekbote
1dbd26fa23 Notebooks for Community Scripts Examples (#9905) 
* Add Notebooks on Community Scripts
 2024-11-12 14:08:48 -10:00
Eliseu Silva
dac623b59f Feature IP Adapter Xformers Attention Processor (#9881) 
* Feature IP Adapter Xformers Attention Processor: this fix error loading incorrect attention processor when setting Xformers attn after load ip adapter scale, issues: #8863 #8872
 2024-11-08 15:40:51 -10:00
Sayak Paul
8d6dc2be5d Revert "[Flux] reduce explicit device transfers and typecasting in flux." (#9896) 
Revert "[Flux] reduce explicit device transfers and typecasting in flux. (#9817)"

This reverts commit 5588725e8e.
 2024-11-08 13:35:38 -10:00
Sayak Paul
d720b2132e [Advanced LoRA v1.5] fix: gradient unscaling problem (#7018) 
fix: gradient unscaling problem

Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com>
 2024-11-08 19:31:43 -04:00
SahilCarterr
9cc96a64f1 [FIX] Fix TypeError in DreamBooth SDXL when use_dora is False (#9879) 
* fix use_dora

* fix style and quality

* fix use_dora with peft version

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-08 19:09:24 -04:00
Michael Tkachuk
5b972fbd6a Enabling gradient checkpointing in eval() mode (#9878) 
* refactored
 2024-11-08 09:03:26 -10:00
SahilCarterr
0be52c07d6 [fix] Replaced shutil.copy with shutil.copyfile (#9885) 
fix shutil.copy
 2024-11-08 08:32:32 -10:00
Dhruv Nair
1b392544c7 Improve downloads of sharded variants (#9869) 
* update

* update

* update

* update

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-08 17:49:00 +05:30
Sayak Paul
5588725e8e [Flux] reduce explicit device transfers and typecasting in flux. (#9817) 
reduce explicit device transfers and typecasting in flux.
 2024-11-06 22:33:39 -04:00
Sayak Paul
ded3db164b [Core] introduce controlnet module (#8768) 
* move vae flax module.

* controlnet module.

* prepare for PR.

* revert a commit

* gracefully deprecate controlnet deps.

* fix

* fix doc path

* fix-copies

* fix path

* style

* style

* conflicts

* fix

* fix-copies

* sparsectrl.

* updates

* fix

* updates

* updates

* updates

* fix

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2024-11-06 22:08:55 -04:00
SahilCarterr
76b7d86a9a Updated _encode_prompt_with_clip and encode_prompt in train_dreamboth_sd3 (#9800) 
* updated encode prompt and clip encod prompt


---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-05 15:08:50 -10:00
Sookwan Han
e2b3c248d8 Add new community pipeline for 'Adaptive Mask Inpainting', introduced in [ECCV2024] ComA (#9228) 
* Add new community pipeline for 'Adaptive Mask Inpainting', introduced in [ECCV2024] Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Models
 2024-11-05 15:05:58 -10:00
Vahid Askari
a03bf4a531 Fix: Remove duplicated comma in distributed_inference.md (#9868) 
Fix: Remove duplicated comma

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-05 23:37:11 +01:00
SahilCarterr
08ac5cbc7f [Fix] Test of sd3 lora (#9843) 
* fix test

* fix test asser

* fix format

* Update test_lora_layers_sd3.py
 2024-11-05 11:05:20 -10:00
Aryan
3f329a426a [core] Mochi T2V (#9769) 
* update

* udpate

* update transformer

* make style

* fix

* add conversion script

* update

* fix

* update

* fix

* update

* fixes

* make style

* update

* update

* update

* init

* update

* update

* add

* up

* up

* up

* update

* mochi transformer

* remove original implementation

* make style

* update inits

* update conversion script

* docs

* Update src/diffusers/pipelines/mochi/pipeline_mochi.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update src/diffusers/pipelines/mochi/pipeline_mochi.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* fix docs

* pipeline fixes

* make style

* invert sigmas in scheduler; fix pipeline

* fix pipeline num_frames

* flip proj and gate in swiglu

* make style

* fix

* make style

* fix tests

* latent mean and std fix

* update

* cherry-pick 1069d210e1

* remove additional sigma already handled by flow match scheduler

* fix

* remove hardcoded value

* replace conv1x1 with linear

* Update src/diffusers/pipelines/mochi/pipeline_mochi.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* framewise decoding and conv_cache

* make style

* Apply suggestions from code review

* mochi vae encoder changes

* rebase correctly

* Update scripts/convert_mochi_to_diffusers.py

* fix tests

* fixes

* make style

* update

* make style

* update

* add framewise and tiled encoding

* make style

* make original vae implementation behaviour the default; note: framewise encoding does not work

* remove framewise encoding implementation due to presence of attn layers

* fight test 1

* fight test 2

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2024-11-05 20:33:41 +05:30
RogerSinghChugh
a3cc641f78 Refac training utils.py (#9815) 
* Refac training utils.py

* quality

---------

Co-authored-by: sayakpaul <spsayakpaul@gmail.com>
 2024-11-04 09:40:44 -08:00
Sayak Paul
13e8fdecda [feat] add load_lora_adapter() for compatible models (#9712) 
* add first draft.

* fix

* updates.

* updates.

* updates

* updates

* updates.

* fix-copies

* lora constants.

* add tests

* Apply suggestions from code review

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

* docstrings.

---------

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
 2024-11-02 09:50:39 +05:30
Dorsa Rohani
c10f875ff0 Add Diffusion Policy for Reinforcement Learning (#9824) 
* enable cpu ability

* model creation + comprehensive testing

* training + tests

* all tests working

* remove unneeded files + clarify docs

* update train tests

* update readme.md

* remove data from gitignore

* undo cpu enabled option

* Update README.md

* update readme

* code quality fixes

* diffusion policy example

* update readme

* add pretrained model weights + doc

* add comment

* add documentation

* add docstrings

* update comments

* update readme

* fix code quality

* Update examples/reinforcement_learning/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/reinforcement_learning/diffusion_policy.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* suggestions + safe globals for weights_only=True

* suggestions + safe weights loading

* fix code quality

* reformat file

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2024-11-02 09:18:44 +05:30