zR
2dad462d9b
Add CogVideoX text-to-video generation model ( #9082 )
...
* add CogVideoX
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-08-06 21:23:57 -10:00
latentCall145
9b5180cb5f
Flux fp16 inference fix ( #9097 )
...
* clipping for fp16
* fix typo
* added fp16 inference to docs
* fix docs typo
* include link for fp16 investigation
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-07 10:54:20 +05:30
Γlvaro Somoza
39e1f7eaa4
[Kolors] Add PAG ( #8934 )
...
* txt2img pag added
* autopipe added, fixed case
* style
* apply suggestions
* added fast tests, added todo tests
* revert dummy objects for kolors
* fix pag dummies
* fix test imports
* update pag tests
* add kolor pag to docs
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-07 09:29:52 +05:30
Dhruv Nair
e1b603dc2e
[Single File] Add single file support for Flux Transformer ( #9083 )
...
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-07 08:49:57 +05:30
Ahn Donghoon (μλν / suno)
926daa30f9
add PAG support for Stable Diffusion 3 ( #8861 )
...
add pag sd3
---------
Co-authored-by: HyoungwonCho <jhw9811@korea.ac.kr >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: crepejung00 <jaewoojung00@naver.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-06 09:11:35 -10:00
Dhruv Nair
325a5de3a9
[Docs] Add community projects section to docs ( #9013 )
...
* update
* update
* update
2024-08-06 08:59:39 -07:00
Dhruv Nair
4c6152c2fb
update
2024-08-06 12:00:14 +00:00
Tolga CangΓΆz
3dc97bd148
Update CLIPFeatureExtractor to CLIPImageProcessor and DPTFeatureExtractor to DPTImageProcessor ( #9002 )
...
* fix: update `CLIPFeatureExtractor` to `CLIPImageProcessor` in codebase
* `make style && make quality`
* Update `DPTFeatureExtractor` to `DPTImageProcessor` in codebase
* `make style`
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-05 09:20:29 -10:00
omahs
6d32b29239
Fix typos ( #9077 )
...
* fix typo
2024-08-05 09:00:08 -10:00
Sayak Paul
5934873b8f
[Docs] add stable cascade unet doc. ( #9066 )
...
* add stable cascade unet doc.
* fix path
2024-08-05 21:28:48 +05:30
Aryan
b7058d142c
PAG variant for HunyuanDiT, PAG refactor ( #8936 )
...
* copy hunyuandit pipeline
* pag variant of hunyuan dit
* add tests
* update docs
* make style
* make fix-copies
* Update src/diffusers/pipelines/pag/pag_utils.py
* remove incorrect copied from
* remove pag hunyuan attn procs to resolve conflicts
* add pag attn procs again
* new implementation for pag_utils
* revert pag changes
* add pag refactor back; update pixart sigma
* update pixart pag tests
* apply suggestions from review
Co-Authored-By: yixu310@gmail.com
* make style
* update docs, fix tests
* fix tests
* fix test_components_function since list not accepted as valid __init__ param
* apply patch to fix broken tests
Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com >
* make style
* fix hunyuan tests
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-05 17:56:09 +05:30
Sayak Paul
c370b90ff1
[Flux] minor documentation fixes for flux. ( #9048 )
...
* minor documentation fixes for flux.
* clipskip
* add gist
2024-08-04 15:53:01 +05:30
Philip Rideout
ebf3ab1477
Fix grammar mistake. ( #9072 )
2024-08-04 04:32:03 +05:30
Tolga CangΓΆz
7071b7461b
Errata: Fix typos & \s+$ ( #9008 )
...
* Fix typos
* chore: Fix typos
* chore: Update README.md for promptdiffusion example
* Trim trailing white spaces
* Fix a typo
* update number
* chore: update number
* Trim trailing white space
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-08-02 21:24:25 -07:00
Sayak Paul
7b98c4cc67
[Core] Add PAG support for PixArtSigma ( #8921 )
...
* feat: add pixart sigma pag.
* inits.
* fixes
* fix
* remove print.
* copy paste methods to the pixart pag mixin
* fix-copies
* add documentation.
* add tests.
* remove correction file.
* remove pag_applied_layers
* empty
2024-08-02 07:12:41 +05:30
Sayak Paul
27637a5402
Flux pipeline ( #9043 )
...
add flux!
Signed-off-by: Adrien <adrien@huggingface.co >
Co-authored-by: Adrien <adrien.69740@gmail.com >
Co-authored-by: Anatoly Belikov <abelikov@singularitynet.io >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-08-01 11:30:52 -10:00
Aryan
05b706c003
PAG variant for AnimateDiff ( #8789 )
...
* add animatediff pag pipeline
* remove unnecessary print
* make fix-copies
* fix ip-adapter bug
* update docs
* add fast tests and fix bugs
* update
* update
* address review comments
* update ip adapter single test expected slice
* implement test_from_pipe_consistent_config; fix expected slice values
* LoraLoaderMixin->StableDiffusionLoraLoaderMixin; add latest freeinit test
2024-08-01 12:39:39 +05:30
Aryan
e5b94b4c57
[core] Move community AnimateDiff ControlNet to core ( #8972 )
...
* add animatediff controlnet to core
* make style; remove unused method
* fix copied from comment
* add tests
* changes to make tests work
* add utility function to load videos
* update docs
* update pipeline example
* make style
* update docs with example
* address review comments
* add latest freeinit test from #8969
* LoraLoaderMixin -> StableDiffusionLoraLoaderMixin
* fix docs
* Update src/diffusers/utils/loading_utils.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* fix: variable out of scope
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-07-30 17:10:37 +05:30
Yoach Lacombe
69e72b1dd1
Stable Audio integration ( #8716 )
...
* WIP modeling code and pipeline
* add custom attention processor + custom activation + add to init
* correct ProjectionModel forward
* add stable audio to __initèè
* add autoencoder and update pipeline and modeling code
* add half Rope
* add partial rotary v2
* add temporary modfis to scheduler
* add EDM DPM Solver
* remove TODOs
* clean GLU
* remove att.group_norm to attn processor
* revert back src/diffusers/schedulers/scheduling_dpmsolver_multistep.py
* refactor GLU -> SwiGLU
* remove redundant args
* add channel multiples in autoencoder docstrings
* changes in docsrtings and copyright headers
* clean pipeline
* further cleaning
* remove peft and lora and fromoriginalmodel
* Delete src/diffusers/pipelines/stable_audio/diffusers.code-workspace
* make style
* dummy models
* fix copied from
* add fast oobleck tests
* add brownian tree
* oobleck autoencoder slow tests
* remove TODO
* fast stable audio pipeline tests
* add slow tests
* make style
* add first version of docs
* wrap is_torchsde_available to the scheduler
* fix slow test
* test with input waveform
* add input waveform
* remove some todos
* create stableaudio gaussian projection + make style
* add pipeline to toctree
* fix copied from
* make quality
* refactor timestep_features->time_proj
* refactor joint_attention_kwargs->cross_attention_kwargs
* remove forward_chunk
* move StableAudioDitModel to transformers folder
* correct convert + remove partial rotary embed
* apply suggestions from yiyixuxu -> removing attn.kv_heads
* remove temb
* remove cross_attention_kwargs
* further removal of cross_attention_kwargs
* remove text encoder autocast to fp16
* continue removing autocast
* make style
* refactor how text and audio are embedded
* add paper
* update example code
* make style
* unify projection model forward + fix device placement
* make style
* remove fuse qkv
* apply suggestions from review
* Update src/diffusers/pipelines/stable_audio/pipeline_stable_audio.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* make style
* smaller models in fast tests
* pass sequential offloading fast tests
* add docs for vae and autoencoder
* make style and update example
* remove useless import
* add cosine scheduler
* dummy classes
* cosine scheduler docs
* better description of scheduler
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-07-30 15:29:06 +05:30
Sayak Paul
00d8d46e23
[Docs] credit where it's due for Lumina and Latte. ( #9000 )
...
credit where it's due for Lumina and Latte.
2024-07-29 10:02:03 -07:00
Γlvaro Somoza
73acebb8cf
[Kolors] Add IP Adapter ( #8901 )
...
* initial draft
* apply suggestions
* fix failing test
* added ipa to img2img
* add docs
* apply suggestions
2024-07-26 14:25:44 -04:00
Aryan
5c53ca5ed8
[core] AnimateDiff SparseCtrl ( #8897 )
...
* initial sparse control model draft
* remove unnecessary implementation
* copy animatediff pipeline
* remove deprecated callbacks
* update
* update pipeline implementation progress
* make style
* make fix-copies
* update progress
* add partially working pipeline
* remove debug prints
* add model docs
* dummy objects
* improve motion lora conversion script
* fix bugs
* update docstrings
* remove unnecessary model params; docs
* address review comment
* add copied from to zero_module
* copy animatediff test
* add fast tests
* update docs
* update
* update pipeline docs
* fix expected slice values
* fix license
* remove get_down_block usage
* remove temporal_double_self_attention from get_down_block
* update
* update docs with org and documentation images
* make from_unet work in sparsecontrolnetmodel
* add latest freeinit test from #8969
* make fix-copies
* LoraLoaderMixin -> StableDiffsuionLoraLoaderMixin
2024-07-26 17:46:05 +05:30
RandomGamingDev
2afb2e0aac
Added accelerator based gradient accumulation for basic_example ( #8966 )
...
added accelerator based gradient accumulation for basic_example
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-07-26 09:35:52 +05:30
Sayak Paul
d87fe95f90
[Chore] add LoraLoaderMixin to the inits ( #8981 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
* rewrite fuse_lora a bit.
* feedback
* copy over load_lora_into_text_encoder.
* address dhruv's feedback.
* fix-copies
* fix issubclass.
* num_fused_loras
* fix
* fix
* remove mapping
* up
* fix
* style
* fix-copies
* change to SD3TransformerLoRALoadersMixin
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* up
* handle wuerstchen
* up
* move lora to lora_pipeline.py
* up
* fix-copies
* fix documentation.
* comment set_adapters().
* fix-copies
* fix set_adapters() at the model level.
* fix?
* fix
* loraloadermixin.
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-07-26 08:59:33 +05:30
YiYi Xu
62863bb1ea
Revert "[LoRA] introduce LoraBaseMixin to promote reusability." ( #8976 )
...
Revert "[LoRA] introduce LoraBaseMixin to promote reusability. (#8774 )"
This reverts commit 527430d0a4
2024-07-25 09:10:35 -10:00
Sayak Paul
527430d0a4
[LoRA] introduce LoraBaseMixin to promote reusability. ( #8774 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
* rewrite fuse_lora a bit.
* feedback
* copy over load_lora_into_text_encoder.
* address dhruv's feedback.
* fix-copies
* fix issubclass.
* num_fused_loras
* fix
* fix
* remove mapping
* up
* fix
* style
* fix-copies
* change to SD3TransformerLoRALoadersMixin
* Apply suggestions from code review
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* up
* handle wuerstchen
* up
* move lora to lora_pipeline.py
* up
* fix-copies
* fix documentation.
* comment set_adapters().
* fix-copies
* fix set_adapters() at the model level.
* fix?
* fix
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-07-25 21:40:58 +05:30
RandomGamingDev
cdd12bde17
Added Code for Gradient Accumulation to work for basic_training ( #8961 )
...
added line allowing gradient accumulation to work for basic_training example
2024-07-25 08:40:53 +05:30
Jiwook Han
5802c2e3f2
Reflect few contributions on ethical_guidelines.md that were not reflected on #8294 ( #8914 )
...
fix_ethical_guidelines.md
2024-07-22 08:48:23 -07:00
Sayak Paul
f4af03b350
[Docs] small fixes to pag guide. ( #8920 )
...
small fixes to pag guide.
2024-07-22 08:35:01 -07:00
Seongsu Park
267bf65707
π [i18n-KO] Translated docs to Korean (added 7 docs and etc) ( #8804 )
...
* remove unused docs
* add ko-18n docs
* docs typo, edit etc
* reorder list, add `in translation` in toctree
* fix minor translation
* fix docs minor tone, etc
2024-07-22 08:08:44 -07:00
Aryan
12625c1c9c
[docs] pipeline docs for latte ( #8844 )
...
* add pipeline docs for latte
* add inference time to latte docs
* apply review suggestions
2024-07-18 09:27:48 +05:30
NguyΓͺΜn CΓ΄ng TuΜ Anh
d704b3bf8c
add PAG support sd15 controlnet ( #8820 )
...
* add pag support sd15 controlnet
* fix quality import
* remove unecessary import
* remove if state
* fix tests
* remove useless function
* add sd1.5 controlnet pag docs
---------
Co-authored-by: anhnct8 <anhnct8@fpt.com >
2024-07-12 15:42:56 +05:30
Sayak Paul
973a62d408
[Docs] add AuraFlow docs ( #8851 )
...
* add pipeline documentation.
* add api spec for pipeline
* model documentation
* model spec
2024-07-12 09:52:18 +02:00
Dhruv Nair
11d18f3217
Add single file loading support for AnimateDiff ( #8819 )
...
* update
* update
* update
* update
2024-07-12 09:51:57 +05:30
Γlvaro Somoza
87b9db644b
[Core] Add Kolors ( #8812 )
...
* initial draft
2024-07-11 06:09:17 -10:00
Xin Ma
b8cf84a3f9
Latte: Latent Diffusion Transformer for Video Generation ( #8404 )
...
* add Latte to diffusers
* remove print
* remove print
* remove print
* remove unuse codes
* remove layer_norm_latte and add a flag
* remove layer_norm_latte and add a flag
* update latte_pipeline
* update latte_pipeline
* remove unuse squeeze
* add norm_hidden_states.ndim == 2: # for Latte
* fixed test latte pipeline bugs
* fixed test latte pipeline bugs
* delete sh
* add doc for latte
* add licensing
* Move Transformer3DModelOutput to modeling_outputs
* give a default value to sample_size
* remove the einops dependency
* change norm2 for latte
* modify pipeline of latte
* update test for Latte
* modify some codes for latte
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* modify for Latte pipeline
* video_length -> num_frames; update prepare_latents copied from
* make fix-copies
* make style
* typo: videe -> video
* update
* modify for Latte pipeline
* modify latte pipeline
* modify latte pipeline
* modify latte pipeline
* modify latte pipeline
* modify for Latte pipeline
* Delete .vscode directory
* make style
* make fix-copies
* add latte transformer 3d to docs _toctree.yml
* update example
* reduce frames for test
* fixed bug of _text_preprocessing
* set num frame to 1 for testing
* remove unuse print
* add text = self._clean_caption(text) again
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: Aryan <aryan@huggingface.co >
2024-07-11 15:06:22 +05:30
PommesPeter
98388670d2
[Alpha-VLLM Team] Add Lumina-T2X to diffusers ( #8652 )
...
---------
Co-authored-by: zhuole1025 <zhuole1025@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-07-07 17:12:09 -10:00
Sayak Paul
984d340534
Revert "[LoRA] introduce LoraBaseMixin to promote reusability." ( #8773 )
...
Revert "[LoRA] introduce `LoraBaseMixin` to promote reusability. (#8670 )"
This reverts commit a2071a1837
2024-07-03 07:05:01 +05:30
Sayak Paul
a2071a1837
[LoRA] introduce LoraBaseMixin to promote reusability. ( #8670 )
...
* introduce to promote reusability.
* up
* add more tests
* up
* remove comments.
* fix fuse_nan test
* clarify the scope of fuse_lora and unfuse_lora
* remove space
2024-07-03 07:04:37 +05:30
Jiwook Han
dd4b731e68
Reflect few contributions on philosophy.md that were not reflected on #8294 ( #8690 )
...
* Update philosophy.md
Some contributions were not reflected previously, so I am resubmitting them.
* Update docs/source/ko/conceptual/philosophy.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/ko/conceptual/philosophy.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-07-02 08:43:56 -07:00
Dhruv Nair
31b211bfe3
Fix mistake in Single File Docs page ( #8765 )
...
update
2024-07-02 12:45:49 +05:30
Dhruv Nair
0368483b61
Remove legacy single file model loading mixins ( #8754 )
...
update
2024-07-01 07:20:19 -10:00
YiYi Xu
ddb9d8548c
[doc] add a tip about using SDXL refiner with hunyuan-dit and pixart ( #8735 )
...
* up
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-07-01 06:30:09 -10:00
Shauray Singh
8690e8b9d6
add PAG support for SD architecture ( #8725 )
...
* add pag to sd pipelines
2024-06-29 09:26:11 -10:00
Sayak Paul
10b4e354b6
[Chore] remove deprecation from transformer2d regarding the output class. ( #8698 )
...
* remove deprecation from transformer2d regarding the output class.
* up
* deprecate more
2024-06-26 07:35:36 -10:00
XCL
fa2abfdb03
[Tencent Hunyuan Team] Add Hunyuan-DiT ControlNet Inference ( #8694 )
...
* add controlnet support
---------
Co-authored-by: xingchaoliu <xingchaoliu@tencent.com >
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-06-26 00:43:03 -10:00
YiYi Xu
1d3ef67b09
[doc] add more about from_pipe API for PAG doc ( #8701 )
...
* add more about from_pipe API
* Update docs/source/en/using-diffusers/pag.md
* Update docs/source/en/using-diffusers/pag.md
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-06-25 22:26:12 -10:00
Sayak Paul
e8284281c1
add docs on model sharding ( #8658 )
...
* add docs on model sharding
* add entry to _toctree.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* simplify wording
* add a note on transformer library handling
* move device placement section
* Update docs/source/en/training/distributed_inference.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-06-26 07:35:11 +05:30
Γlvaro Somoza
14d224d4e6
[Docs] SD3 T5 Token limit doc ( #8654 )
...
* doc for max_sequence_length
* better position and changed note to tip
* apply suggestions
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-06-25 14:41:27 -04:00
YiYi Xu
540399f540
add PAG support ( #7944 )
...
* first draft
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com >
Co-authored-by: Ahn Donghoon (μλν / suno) <suno.vivid@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-06-25 08:40:02 -10:00
