zR
06f36713ae
Cogvideox-5B Model adapter change ( #9203 )
...
* draft of embedding
---------
Co-authored-by: Aryan <aryan@huggingface.co >
2024-08-23 15:17:20 -10:00
zR
2dad462d9b
Add CogVideoX text-to-video generation model ( #9082 )
...
* add CogVideoX
---------
Co-authored-by: Aryan <aryan@huggingface.co >
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: Aryan <contact.aryanvs@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-08-06 21:23:57 -10:00
Tolga Cangöz
7071b7461b
Errata: Fix typos & \s+$ ( #9008 )
...
* Fix typos
* chore: Fix typos
* chore: Update README.md for promptdiffusion example
* Trim trailing white spaces
* Fix a typo
* update number
* chore: update number
* Trim trailing white space
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update README.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-08-02 21:24:25 -07:00
Sayak Paul
27637a5402
Flux pipeline ( #9043 )
...
add flux!
Signed-off-by: Adrien <adrien@huggingface.co >
Co-authored-by: Adrien <adrien.69740@gmail.com >
Co-authored-by: Anatoly Belikov <abelikov@singularitynet.io >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-08-01 11:30:52 -10:00
Yoach Lacombe
69e72b1dd1
Stable Audio integration ( #8716 )
...
* WIP modeling code and pipeline
* add custom attention processor + custom activation + add to init
* correct ProjectionModel forward
* add stable audio to __initèè
* add autoencoder and update pipeline and modeling code
* add half Rope
* add partial rotary v2
* add temporary modfis to scheduler
* add EDM DPM Solver
* remove TODOs
* clean GLU
* remove att.group_norm to attn processor
* revert back src/diffusers/schedulers/scheduling_dpmsolver_multistep.py
* refactor GLU -> SwiGLU
* remove redundant args
* add channel multiples in autoencoder docstrings
* changes in docsrtings and copyright headers
* clean pipeline
* further cleaning
* remove peft and lora and fromoriginalmodel
* Delete src/diffusers/pipelines/stable_audio/diffusers.code-workspace
* make style
* dummy models
* fix copied from
* add fast oobleck tests
* add brownian tree
* oobleck autoencoder slow tests
* remove TODO
* fast stable audio pipeline tests
* add slow tests
* make style
* add first version of docs
* wrap is_torchsde_available to the scheduler
* fix slow test
* test with input waveform
* add input waveform
* remove some todos
* create stableaudio gaussian projection + make style
* add pipeline to toctree
* fix copied from
* make quality
* refactor timestep_features->time_proj
* refactor joint_attention_kwargs->cross_attention_kwargs
* remove forward_chunk
* move StableAudioDitModel to transformers folder
* correct convert + remove partial rotary embed
* apply suggestions from yiyixuxu -> removing attn.kv_heads
* remove temb
* remove cross_attention_kwargs
* further removal of cross_attention_kwargs
* remove text encoder autocast to fp16
* continue removing autocast
* make style
* refactor how text and audio are embedded
* add paper
* update example code
* make style
* unify projection model forward + fix device placement
* make style
* remove fuse qkv
* apply suggestions from review
* Update src/diffusers/pipelines/stable_audio/pipeline_stable_audio.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* make style
* smaller models in fast tests
* pass sequential offloading fast tests
* add docs for vae and autoencoder
* make style and update example
* remove useless import
* add cosine scheduler
* dummy classes
* cosine scheduler docs
* better description of scheduler
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-07-30 15:29:06 +05:30
Aryan
5c53ca5ed8
[core] AnimateDiff SparseCtrl ( #8897 )
...
* initial sparse control model draft
* remove unnecessary implementation
* copy animatediff pipeline
* remove deprecated callbacks
* update
* update pipeline implementation progress
* make style
* make fix-copies
* update progress
* add partially working pipeline
* remove debug prints
* add model docs
* dummy objects
* improve motion lora conversion script
* fix bugs
* update docstrings
* remove unnecessary model params; docs
* address review comment
* add copied from to zero_module
* copy animatediff test
* add fast tests
* update docs
* update
* update pipeline docs
* fix expected slice values
* fix license
* remove get_down_block usage
* remove temporal_double_self_attention from get_down_block
* update
* update docs with org and documentation images
* make from_unet work in sparsecontrolnetmodel
* add latest freeinit test from #8969
* make fix-copies
* LoraLoaderMixin -> StableDiffsuionLoraLoaderMixin
2024-07-26 17:46:05 +05:30
Dhruv Nair
5fbb4d32d5
[CI] Slow Test Updates ( #8870 )
...
* update
* update
* update
2024-07-25 16:00:43 +05:30
Lucain
56e772ab7e
Use model_info.id instead of model_info.modelId ( #8912 )
...
Mention model_info.id instead of model_info.modelId
2024-07-20 20:01:21 +05:30
Sayak Paul
2261510bbc
[Core] Add AuraFlow ( #8796 )
...
* add lavender flow transformer
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-07-11 08:50:19 -10:00
PommesPeter
98388670d2
[Alpha-VLLM Team] Add Lumina-T2X to diffusers ( #8652 )
...
---------
Co-authored-by: zhuole1025 <zhuole1025@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-07-07 17:12:09 -10:00
XCL
6b6b4bcffe
[Tencent Hunyuan Team] Add checkpoint conversion scripts and changed controlnet ( #8783 )
...
* add conversion files; changed controlnet for hunyuandit
* style
---------
Co-authored-by: xingchaoliu <xingchaoliu@tencent.com >
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-07-03 07:45:18 -10:00
YiYi Xu
715a7da1b2
add sd3 conversion script ( #8702 )
...
add conversion script
2024-06-25 14:24:58 -10:00
Mark Van Aken
be4afa0bb4
#7535 Update FloatTensor type hints to Tensor ( #7883 )
...
* find & replace all FloatTensors to Tensor
* apply formatting
* Update torch.FloatTensor to torch.Tensor in the remaining files
* formatting
* Fix the rest of the places where FloatTensor is used as well as in documentation
* formatting
* Update new file from FloatTensor to Tensor
2024-05-10 09:53:31 -10:00
Tolga Cangöz
c1c42698c9
Remove dead code and fix f-string issue ( #7720 )
...
* Remove dead code
* PylancereportGeneralTypeIssues: Strings nested within an f-string cannot use the same quote character as the f-string prior to Python 3.12.
* Remove dead code
2024-05-08 13:15:28 -10:00
Aryan
818f760732
[Pipeline] AnimateDiff SDXL ( #6721 )
...
* update conversion script to handle motion adapter sdxl checkpoint
* add animatediff xl
* handle addition_embed_type
* fix output
* update
* add imports
* make fix-copies
* add decode latents
* update docstrings
* add animatediff sdxl to docs
* remove unnecessary lines
* update example
* add test
* revert conv_in conv_out kernel param
* remove unused param addition_embed_type_num_heads
* latest IPAdapter impl
* make fix-copies
* fix return
* add IPAdapterTesterMixin to tests
* fix return
* revert based on suggestion
* add freeinit
* fix test_to_dtype test
* use StableDiffusionMixin instead of different helper methods
* fix progress bar iterations
* apply suggestions from review
* hardcode flip_sin_to_cos and freq_shift
* make fix-copies
* fix ip adapter implementation
* fix last failing test
* make style
* Update docs/source/en/api/pipelines/animatediff.md
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* remove todo
* fix doc-builder errors
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-05-08 21:27:14 +05:30
Dhruv Nair
eb96ff0d59
Safetensor loading in AnimateDiff conversion scripts ( #7764 )
...
* update
* update
2024-04-29 17:36:50 +05:30
Junsong Chen
39215aa30e
PixArt-Sigma Implementation ( #7654 )
...
* support PixArt-DMD
---------
Co-authored-by: jschen <chenjunsong4@h-partners.com >
Co-authored-by: badayvedat <badayvedat@gmail.com >
Co-authored-by: Vedat Baday <54285744+badayvedat@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-04-23 22:33:08 -10:00
Sayak Paul
e25e525fde
[LoRA test suite] refactor the test suite and cleanse it ( #7316 )
...
* cleanse and refactor lora testing suite.
* more cleanup.
* make check_if_lora_correctly_set a utility function
* fix: typo
* retrigger ci
* style
2024-03-20 17:13:52 +05:30
M. Tolga Cangöz
e97a633b63
Update access of configuration attributes ( #7343 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-18 08:53:29 -10:00
Sayak Paul
46ab56a468
add: support for notifying maintainers about the nightly test status ( #7117 )
...
* add: support for notifying maintainers about the nightly test status
* add: a tempoerary workflow for validation.
* cancel in progress.
* runs-on
* clean up
* add: peft dep
* change device.
* multiple edits.
* remove temp workflow.
2024-03-13 16:48:11 +05:30
Dhruv Nair
30132aba30
Update Stable Cascade Conversion Scripts ( #7271 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-13 12:35:44 +05:30
Kashif Rasul
40aa47b998
[Pipiline] Wuerstchen v3 aka Stable Cascasde pipeline ( #6487 )
...
* initial diffNext v3
* move to v3 folder
* imports
* dry up the unets
* no switch_level
* fix init
* add switch_level tp config
* Fixed some things
* Added pooled text embeddings
* Initial work on adding image encoder
* changes from @dome272
* Stuff for the image encoder processing and variable naming in decoder
* fix arg name
* inference fixes
* inference fixes
* default TimestepBlock without conds
* c_skip=0 by default
* fix bfloat16 to cpu
* use config
* undo temp change
* fix gen_c_embeddings args
* change text encoding
* text encoding
* undo print
* undo .gitignore change
* Allow WuerstchenV3PriorPipeline to use the base DDPM & DDIM schedulers
* use WuerstchenV3Unet in both pipelines
* fix imports
* initial failing tests
* cleanup
* use scheduler.timesterps
* some fixes to the tests, still not fully working
* fix tests
* fix prior tests
* add dropout to the model_kwargs
* more tests passing
* update expected_slice
* initial rename
* rename tests
* rename class names
* make fix-copies
* initial docs
* autodocs
* typos
* fix arg docs
* add text_encoder info
* combined pipeline has optional image arg
* fix documentation
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* use self.config
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* c_in -> in_channels
* removed kwargs from unet's forward
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* remove older callback api
* removed kwargs and fixed decoder guidance > 1
* decoder takes emeds
* check and use image_embeds
* fixed all but one decoder test
* fix decoder tests
* update callback api
* fix some more combined tests
* push combined pipeline
* initial docs
* fix doc_string
* update combined api
* no test_callback_inputs test for combined pipeline
* add optional components
* fix ordering of components
* fix combined tests
* update convert script
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* fix imports
* move effnet out of deniosing loop
* prompt_embeds_pooled only when doing guidance
* Fix repeat shape
* move StableCascadeUnet to models/unets/
* more descriptive names
* converted when numpy()
* StableCascadePriorPipelineOutput docs
* rename StableCascadeUNet
* add slow tests
* fix slow tests
* update
* update
* updated model_path
* add args for weights
* set push_to_hub to false
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
Co-authored-by: Pablo Pernias <pablo@pernias.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: 99991 <99991@users.noreply.github.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-03-06 15:07:25 +05:30
Junsong Chen
f55873b783
Fix PixArt 256px inference ( #6789 )
...
* feat 256px diffusers inference bug
* change the max_length of T5 to pipeline config file
* fix bug in convert_pixart_alpha_to_diffusers.py
* Update scripts/convert_pixart_alpha_to_diffusers.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* remove multi_scale_train parser
* Update src/diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* styling
* change `model_token_max_length` to call argument.
* Refactoring
* add: max_sequence_length to the docstring.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-03-03 10:31:21 +05:30
Dhruv Nair
8f2d13c684
Fix setting fp16 dtype in AnimateDiff convert script. ( #7127 )
...
* update
* update
2024-02-29 22:47:39 +05:30
Dhruv Nair
d603ccb614
Small change to download in dance diffusion convert script ( #7070 )
...
* update
* make style
2024-02-26 12:05:19 +05:30
Sayak Paul
371f765908
[Diffusers -> Original SD conversion] fix things ( #6933 )
...
* fix: bias loading bug
* fixes for SDXL
* apply changes to the conversion script to match single_file_utils.py
* do transpose to match the single file loading logic.
2024-02-12 17:30:22 +05:30
Sayak Paul
30e5e81d58
change to 2024 in the license ( #6902 )
...
change to 2024
2024-02-08 08:19:31 -10:00
Patryk Bartkowiak
3ac2357794
changed positional parameters to named parameters like in docs ( #6905 )
...
Co-authored-by: Patryk Bartkowiak <patryk.bartkowiak@tcl.com >
Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com >
2024-02-08 21:39:03 +05:30
Sayak Paul
1835510524
Remove torch_dtype in to() to end deprecation ( #6886 )
...
* remove torch_dtype from to()
* remove torch_dtype from usage scripts.
* remove old lora backend
* Revert "remove old lora backend"
This reverts commit adcddf6ba4
2024-02-08 09:38:57 +05:30
Sayak Paul
04cd6adf8c
[Feat] add I2VGenXL for image-to-video generation ( #6665 )
...
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-01-31 10:38:51 -10:00
Sayak Paul
09b7bfce91
[Core] move transformer scripts to transformers modules ( #6747 )
...
* move transformer scripts to transformers modules
* move transformer model test
* move prior transformer test to directory
* fix doc path
* correct doc path
* add: __init__.py
2024-01-29 22:28:28 +05:30
Sayak Paul
1f0705adcf
[Big refactor] move unets to unets module 🦋 ( #6630 )
...
* move unets to module 🦋
* parameterize unet-level import.
* fix flax unet2dcondition model import
* models __init__
* mildly depcrecating models.unet_2d_blocks in favor of models.unets.unet_2d_blocks.
* noqa
* correct depcrecation behaviour
* inherit from the actual classes.
* Empty-Commit
* backwards compatibility for unet_2d.py
* backward compatibility for unet_2d_condition
* bc for unet_1d
* bc for unet_1d_blocks
2024-01-23 08:57:58 +05:30
Sayak Paul
cb4b3f0b78
[OmegaConf] replace it with yaml ( #6488 )
...
* remove omegaconf from convert_from_ckpt.
* remove from single_file.
* change to string based ubscription.
* style
* okay
* fix: vae_param
* no . indexing.
* style
* style
* turn getattrs into explicit if/else
* style
* propagate changes to ldm_uncond.
* propagate to gligen
* propagate to if.
* fix: quotes.
* propagate to audioldm.
* propagate to audioldm2
* propagate to musicldm.
* propagate to vq_diffusion
* propagate to zero123.
* remove omegaconf from diffusers codebase.
2024-01-15 20:02:10 +05:30
apolinário
0b63ad5ad5
Create convert_diffusers_sdxl_lora_to_webui.py ( #6395 )
...
* Create convert_diffusers_sdxl_lora_to_webui.py
* Move some conversion logic to utils
* fix logging import
* Add usage example
---------
Co-authored-by: multimodalart <joaopaulo.passos+multimodal@gmail.com >
2023-12-30 08:15:11 -06:00
Dhruv Nair
fb02316db8
Add AnimateDiff conversion scripts ( #6340 )
...
* add scripts
* update
2023-12-26 22:40:00 +05:30
Will Berman
4039815276
open muse ( #5437 )
...
amused
rename
Update docs/source/en/api/pipelines/amused.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
AdaLayerNormContinuous default values
custom micro conditioning
micro conditioning docs
put lookup from codebook in constructor
fix conversion script
remove manual fused flash attn kernel
add training script
temp remove training script
add dummy gradient checkpointing func
clarify temperatures is an instance variable by setting it
remove additional SkipFF block args
hardcode norm args
rename tests folder
fix paths and samples
fix tests
add training script
training readme
lora saving and loading
non-lora saving/loading
some readme fixes
guards
Update docs/source/en/api/pipelines/amused.md
Co-authored-by: Suraj Patil <surajp815@gmail.com >
Update examples/amused/README.md
Co-authored-by: Suraj Patil <surajp815@gmail.com >
Update examples/amused/train_amused.py
Co-authored-by: Suraj Patil <surajp815@gmail.com >
vae upcasting
add fp16 integration tests
use tuple for micro cond
copyrights
remove casts
delegate to torch.nn.LayerNorm
move temperature to pipeline call
upsampling/downsampling changes
2023-12-21 11:40:55 -08:00
d8ahazard
6976cab7ca
Fix possible re-conversion issues after extracting from safetensors ( #6097 )
...
* Fix possible re-conversion issues after extracting from diffusers
Properly rename specific vae keys.
* Whoops
2023-12-18 11:51:20 +01:00
Sayak Paul
56b3b21693
[Refactor autoencoders] feat: introduce autoencoders module ( #6129 )
...
* feat: introduce autoencoders module
* more changes for styling and copy fixing
* path changes in the docs.
* fix: import structure in init.
* fix controlnetxs import
2023-12-18 12:42:15 +05:30
Suraj Patil
63f767ef15
Add SVD ( #5895 )
...
* begin model
* finish blocks
* add_embedding
* addition_time_embed_dim
* use TimestepEmbedding
* fix temporal res block
* fix time_pos_embed
* fix add_embedding
* add conversion script
* fix model
* up
* add new resnet blocks
* make forward work
* return sample in original shape
* fix temb shape in TemporalResnetBlock
* add spatio temporal transformers
* add vae blocks
* fix blocks
* update
* update
* fix shapes in Alphablender and add time activation in res blcok
* use new blocks
* style
* fix temb shape
* fix SpatioTemporalResBlock
* reuse TemporalBasicTransformerBlock
* fix TemporalBasicTransformerBlock
* use TransformerSpatioTemporalModel
* fix TransformerSpatioTemporalModel
* fix time_context dim
* clean up
* make temb optional
* add blocks
* rename model
* update conversion script
* remove UNetMidBlockSpatioTemporal
* add in init
* remove unused arg
* remove unused arg
* remove more unsed args
* up
* up
* check for None
* update vae
* update up/mid blocks for decoder
* begin pipeline
* adapt scheduler
* add guidance scalings
* fix norm eps in temporal transformers
* add temporal autoencoder
* make pipeline run
* fix frame decodig
* decode in float32
* decode n frames at a time
* pass decoding_t to decode_latents
* fix decode_latents
* vae encode/decode in fp32
* fix dtype in TransformerSpatioTemporalModel
* type image_latents same as image_embeddings
* allow using differnt eps in temporal block for video decoder
* fix default values in vae
* pass num frames in decode
* switch spatial to temporal for mixing in VAE
* fix num frames during split decoding
* cast alpha to sample dtype
* fix attention in MidBlockTemporalDecoder
* fix typo
* fix guidance_scales dtype
* fix missing activation in TemporalDecoder
* skip_post_quant_conv
* add vae conversion
* style
* take guidance scale as input
* up
* allow passing PIL to export_video
* accept fps as arg
* add pipeline and vae in init
* remove hack
* use AutoencoderKLTemporalDecoder
* don't scale image latents
* add unet tests
* clean up unet
* clean TransformerSpatioTemporalModel
* add slow svd test
* clean up
* make temb optional in Decoder mid block
* fix norm eps in TransformerSpatioTemporalModel
* clean up temp decoder
* clean up
* clean up
* use c_noise values for timesteps
* use math for log
* update
* fix copies
* doc
* upcast vae
* update forward pass for gradient checkpointing
* make added_time_ids is tensor
* up
* fix upcasting
* remove post quant conv
* add _resize_with_antialiasing
* fix _compute_padding
* cleanup model
* more cleanup
* more cleanup
* more cleanup
* remove freeu
* remove attn slice
* small clean
* up
* up
* remove extra step kwargs
* remove eta
* remove dropout
* remove callback
* remove merge factor args
* clean
* clean up
* move to dedicated folder
* remove attention_head_dim
* docstr and small fix
* update unet doc strings
* rename decoding_t
* correct linting
* store c_skip and c_out
* cleanup
* clean TemporalResnetBlock
* more cleanup
* clean up vae
* clean up
* begin doc
* more cleanup
* up
* up
* doc
* Improve
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* Apply suggestions from code review
* Default chunk size to None
* add example
* Better
* Apply suggestions from code review
* update doc
* Update src/diffusers/pipelines/stable_diffusion_video/pipeline_stable_diffusion_video.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* style
* Get torch compile working
* up
* rename
* fix doc
* add chunking
* torch compile
* torch compile
* add modelling outputs
* torch compile
* Improve chunking
* Apply suggestions from code review
* Update docs/source/en/using-diffusers/svd.md
* Close diff tag
* remove slicing
* resnet docstr
* add docstr in resnet
* rename
* Apply suggestions from code review
* update tests
* Fix output type latents
* fix more
* fix more
* Update docs/source/en/using-diffusers/svd.md
* fix more
* add pipeline tests
* remove unused arg
* clean up
* make sure get_scaling receives tensors
* fix euler scheduler
* fix get_scalings
* simply euler for now
* remove old test file
* use randn_tensor to create noise
* fix device for rand tensor
* increase expected_max_difference
* fix test_inference_batch_single_identical
* actually fix test_inference_batch_single_identical
* disable test_save_load_float16
* skip test_float16_inference
* skip test_inference_batch_single_identical
* fix test_xformers_attention_forwardGenerator_pass
* Apply suggestions from code review
* update StableVideoDiffusionPipelineSlowTests
* update image
* add diffusers example
* fix more
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
2023-11-29 19:13:36 +01:00
Patrick von Platen
b978334d71
[@cene555][Kandinsky 3.0] Add Kandinsky 3.0 ( #5913 )
...
* finalize
* finalize
* finalize
* add slow test
* add slow test
* add slow test
* Fix more
* add slow test
* fix more
* fix more
* fix more
* fix more
* fix more
* fix more
* fix more
* fix more
* fix more
* Better
* Fix more
* Fix more
* add slow test
* Add auto pipelines
* add slow test
* Add all
* add slow test
* add slow test
* add slow test
* add slow test
* add slow test
* Apply suggestions from code review
* add slow test
* add slow test
2023-11-24 17:46:00 +01:00
Kashif Rasul
6b04d61cf6
[Styling] stylify using ruff ( #5841 )
...
* ruff format
* not need to use doc-builder's black styling as the doc is styled in ruff
* make fix-copies
* comment
* use run_ruff
2023-11-20 11:48:34 +01:00
Lucain
c896b841e4
Set usedforsecurity=False in hashlib methods (FIPS compliance) ( #5790 )
...
* Set usedforsecurity=False in hashlib methods (FIPS compliance)
* update version dependency
* bump hfh version
* bump hfh version
2023-11-17 14:56:58 +01:00
Will Berman
2fd46405cd
consistency decoder ( #5694 )
...
* consistency decoder
* rename
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update src/diffusers/pipelines/consistency_models/pipeline_consistency_models.py
* uP
* Apply suggestions from code review
* uP
* uP
* uP
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-11-09 12:21:41 +01:00
Sayak Paul
d61889fc17
[Feat] PixArt-Alpha ( #5642 )
...
* init pixart alpha pipeline
* fix: import
* script
* script
* script
* add: vae to the pipeline
* add: vae_scale_factor
* add: checkpoint_path
* clean conversion script a bit.
* size embeddings.
* fix: size embedding
* update scrip
* support for interpolation of position embedding.
* support for conditioning.
* ..
* ..
* ..
* final layer
* final layer
* align if encode_prompt
* support for caption embedding
* refactor
* refactor
* refactor
* start cross attention
* start cross attention
* cross_attention_dim
* cross
* cross
* support for resolution and aspect_ratio
* support for caption projection
* refactor patch embeddings
* batch_size
* up
* commit
* commit
* commit.
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze.
* squeeze.
* fix final block./
* fix final block./
* fix final block./
* clean
* fix: interpolation scale.
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* make --checkpoint_path non-required.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* remove num_tokens
* timesteps -> timestep
* timesteps -> timestep
* timesteps -> timestep
* timesteps -> timestep
* timesteps -> timestep
* timesteps -> timestep
* debug
* debug
* update conversion script.
* update conversion script.
* update conversion script.
* debug
* debug
* debug
* clean
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* deug
* debug
* debug
* debug
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* clean
* fix
* fix
* boom
* boom
* some changes
* boom
* save
* up
* remove i
* fix more tests
* DPMSolverMultistepScheduler
* fix
* offloading
* fix conversion script
* fix conversion script
* remove print
* remove support for negative prompt embeds.
* typo.
* remove extra kwargs
* bring conversion script to where it was
* fix
* trying mu luck
* trying my luck again
* again
* again
* again
* clean up
* up
* up
* update example
* support for 512
* remove spacing
* finalize docs.
* test debug
* fix: assertion values.
* debug
* debug
* debug
* fix: repeat
* remove prints.
* Apply suggestions from code review
* Apply suggestions from code review
* Correct more
* Apply suggestions from code review
* Change all
* Clean more
* fix more
* Fix more
* Fix more
* Correct more
* address patrick's comments.
* remove unneeded args
* clean up pipeline.
* sty;e
* make the use of additional conditions better conditioned.
* None better
* dtype
* height and width validation
* add a note about size brackets.
* fix
* spit out slow test outputs.
* fix?
* fix optional test
* fix more
* remove unneeded comment
* debug
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-11-06 08:40:04 +01:00
dg845
cd1b8d7ca8
[WIP] Refactor UniDiffuser Pipeline and Tests ( #4948 )
...
* Add VAE slicing and tiling methods.
* Switch to using VaeImageProcessing for preprocessing and postprocessing of images.
* Rename the VaeImageProcessor to vae_image_processor to avoid a name clash with the CLIPImageProcessor (image_processor).
* Remove the postprocess() function because we're using a VaeImageProcessor instead.
* Remove UniDiffuserPipeline.decode_image_latents because we're using VaeImageProcessor instead.
* Refactor generating text from text latents into a decode_text_latents method.
* Add enable_full_determinism() to UniDiffuser tests.
* make style
* Add PipelineLatentTesterMixin to UniDiffuserPipelineFastTests.
* Remove enable_model_cpu_offload since it is now part of DiffusionPipeline.
* Rename the VaeImageProcessor instance to self.image_processor for consistency with other pipelines and rename the CLIPImageProcessor instance to clip_image_processor to avoid a name clash.
* Update UniDiffuser conversion script.
* Make safe_serialization configurable in UniDiffuser conversion script.
* Rename image_processor to clip_image_processor in UniDiffuser tests.
* Add PipelineKarrasSchedulerTesterMixin to UniDiffuserPipelineFastTests.
* Add initial test for compiling the UniDiffuser model (not tested yet).
* Update encode_prompt and _encode_prompt to match that of StableDiffusionPipeline.
* Turn off standard classifier-free guidance for now.
* make style
* make fix-copies
* apply suggestions from review
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-10-02 18:24:55 +02:00
Ayush Mangal
157c9011d8
Add BLIP Diffusion ( #4388 )
...
* Add BLIP Diffusion skeleton
* Add other model components
* Add BLIP2, need to change it for now
* Fix pipeline imports
* Load pretrained ViT
* Make qformer fwd pass same
* Replicate fwd passes
* Fix device bug
* Add accelerate functions
* Remove extra functions from Blip2
* Minor bug
* Integrate initial review changes
* Refactoring
* Refactoring
* Refactor
* Add controlnet
* Refactor
* Update conversion script
* Add image processor
* Shift postprocessing to ImageProcessor
* Refactor
* Fix device
* Add fast tests
* Update conversion script
* Fix checkpoint conversion script
* Integrate review changes
* Integrate reivew changes
* Remove unused functions from test
* Reuse HF image processor in Cond image
* Create new BlipImageProcessor based on transfomers
* Fix image preprocessor
* Minor
* Minor
* Add canny preprocessing
* Fix controlnet preprocessing
* Fix blip diffusion test
* Add controlnet test
* Add initial doc strings
* Integrate review changes
* Refactor
* Update examples
* Remove DDIM comments
* Add copied from for prepare_latents
* Add type anotations
* Add docstrings
* Do black formatting
* Add batch support
* Make tests pass
* Make controlnet tests pass
* Black formatting
* Fix progress bar
* Fix some licensing comments
* Fix imports
* Refactor controlnet
* Make tests faster
* Edit examples
* Black formatting/Ruff
* Add doc
* Minor
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Move controlnet pipeline
* Make tests faster
* Fix imports
* Fix formatting
* Fix make errors
* Fix make errors
* Minor
* Add suggested doc changes
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Edit docs
* Fix 16 bit loading
* Update examples
* Edit toctree
* Update docs/source/en/api/pipelines/blip_diffusion.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Minor
* Add tips
* Edit examples
* Update model paths
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-21 17:05:35 +01:00
김태민
5b78141fd3
[FIX BUG] add config_files parser #5114 ( #5115 )
...
* add config_files parser #5114
* add config_files parser_fix #5114
2023-09-20 16:17:47 +02:00
dg845
4c8a05f115
Fix Consistency Models UNet2DMidBlock2D Attention GroupNorm Bug ( #4863 )
...
* Add attn_groups argument to UNet2DMidBlock2D to control theinternal Attention block's GroupNorm.
* Add docstring for attn_norm_num_groups in UNet2DModel.
* Since the test UNet config uses resnet_time_scale_shift == 'scale_shift', also set attn_norm_num_groups to 32.
* Add test for attn_norm_num_groups to UNet2DModelTests.
* Fix expected slices for slow tests.
* Also fix tolerances for slow tests.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-15 11:27:51 +01:00
Vladimir Mandic
ef29b24fda
allow loading of sd models from safetensors without online lookups using local config files ( #5019 )
...
finish config_files implementation
2023-09-14 12:30:15 +02:00
Kashif Rasul
16a056a7b5
Wuerstchen fixes ( #4942 )
...
* fix arguments and make example code work
* change arguments in combined test
* Add default timesteps
* style
* fixed test
* fix broken test
* formatting
* fix docstrings
* fix num_images_per_prompt
* fix doc styles
* please dont change this
* fix tests
* rename to DEFAULT_STAGE_C_TIMESTEPS
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
2023-09-11 15:47:53 +02:00
