Mark Van Aken
be4afa0bb4
#7535 Update FloatTensor type hints to Tensor ( #7883 )
...
* find & replace all FloatTensors to Tensor
* apply formatting
* Update torch.FloatTensor to torch.Tensor in the remaining files
* formatting
* Fix the rest of the places where FloatTensor is used as well as in documentation
* formatting
* Update new file from FloatTensor to Tensor
2024-05-10 09:53:31 -10:00
Tolga Cangöz
c1c42698c9
Remove dead code and fix f-string issue ( #7720 )
...
* Remove dead code
* PylancereportGeneralTypeIssues: Strings nested within an f-string cannot use the same quote character as the f-string prior to Python 3.12.
* Remove dead code
2024-05-08 13:15:28 -10:00
Aryan
818f760732
[Pipeline] AnimateDiff SDXL ( #6721 )
...
* update conversion script to handle motion adapter sdxl checkpoint
* add animatediff xl
* handle addition_embed_type
* fix output
* update
* add imports
* make fix-copies
* add decode latents
* update docstrings
* add animatediff sdxl to docs
* remove unnecessary lines
* update example
* add test
* revert conv_in conv_out kernel param
* remove unused param addition_embed_type_num_heads
* latest IPAdapter impl
* make fix-copies
* fix return
* add IPAdapterTesterMixin to tests
* fix return
* revert based on suggestion
* add freeinit
* fix test_to_dtype test
* use StableDiffusionMixin instead of different helper methods
* fix progress bar iterations
* apply suggestions from review
* hardcode flip_sin_to_cos and freq_shift
* make fix-copies
* fix ip adapter implementation
* fix last failing test
* make style
* Update docs/source/en/api/pipelines/animatediff.md
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* remove todo
* fix doc-builder errors
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-05-08 21:27:14 +05:30
Dhruv Nair
eb96ff0d59
Safetensor loading in AnimateDiff conversion scripts ( #7764 )
...
* update
* update
2024-04-29 17:36:50 +05:30
Junsong Chen
39215aa30e
PixArt-Sigma Implementation ( #7654 )
...
* support PixArt-DMD
---------
Co-authored-by: jschen <chenjunsong4@h-partners.com >
Co-authored-by: badayvedat <badayvedat@gmail.com >
Co-authored-by: Vedat Baday <54285744+badayvedat@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2024-04-23 22:33:08 -10:00
Sayak Paul
e25e525fde
[LoRA test suite] refactor the test suite and cleanse it ( #7316 )
...
* cleanse and refactor lora testing suite.
* more cleanup.
* make check_if_lora_correctly_set a utility function
* fix: typo
* retrigger ci
* style
2024-03-20 17:13:52 +05:30
M. Tolga Cangöz
e97a633b63
Update access of configuration attributes ( #7343 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-18 08:53:29 -10:00
Sayak Paul
46ab56a468
add: support for notifying maintainers about the nightly test status ( #7117 )
...
* add: support for notifying maintainers about the nightly test status
* add: a tempoerary workflow for validation.
* cancel in progress.
* runs-on
* clean up
* add: peft dep
* change device.
* multiple edits.
* remove temp workflow.
2024-03-13 16:48:11 +05:30
Dhruv Nair
30132aba30
Update Stable Cascade Conversion Scripts ( #7271 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-03-13 12:35:44 +05:30
Kashif Rasul
40aa47b998
[Pipiline] Wuerstchen v3 aka Stable Cascasde pipeline ( #6487 )
...
* initial diffNext v3
* move to v3 folder
* imports
* dry up the unets
* no switch_level
* fix init
* add switch_level tp config
* Fixed some things
* Added pooled text embeddings
* Initial work on adding image encoder
* changes from @dome272
* Stuff for the image encoder processing and variable naming in decoder
* fix arg name
* inference fixes
* inference fixes
* default TimestepBlock without conds
* c_skip=0 by default
* fix bfloat16 to cpu
* use config
* undo temp change
* fix gen_c_embeddings args
* change text encoding
* text encoding
* undo print
* undo .gitignore change
* Allow WuerstchenV3PriorPipeline to use the base DDPM & DDIM schedulers
* use WuerstchenV3Unet in both pipelines
* fix imports
* initial failing tests
* cleanup
* use scheduler.timesterps
* some fixes to the tests, still not fully working
* fix tests
* fix prior tests
* add dropout to the model_kwargs
* more tests passing
* update expected_slice
* initial rename
* rename tests
* rename class names
* make fix-copies
* initial docs
* autodocs
* typos
* fix arg docs
* add text_encoder info
* combined pipeline has optional image arg
* fix documentation
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* use self.config
* Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* c_in -> in_channels
* removed kwargs from unet's forward
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* remove older callback api
* removed kwargs and fixed decoder guidance > 1
* decoder takes emeds
* check and use image_embeds
* fixed all but one decoder test
* fix decoder tests
* update callback api
* fix some more combined tests
* push combined pipeline
* initial docs
* fix doc_string
* update combined api
* no test_callback_inputs test for combined pipeline
* add optional components
* fix ordering of components
* fix combined tests
* update convert script
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* fix imports
* move effnet out of deniosing loop
* prompt_embeds_pooled only when doing guidance
* Fix repeat shape
* move StableCascadeUnet to models/unets/
* more descriptive names
* converted when numpy()
* StableCascadePriorPipelineOutput docs
* rename StableCascadeUNet
* add slow tests
* fix slow tests
* update
* update
* updated model_path
* add args for weights
* set push_to_hub to false
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
Co-authored-by: Pablo Pernias <pablo@pernias.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: 99991 <99991@users.noreply.github.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-03-06 15:07:25 +05:30
Junsong Chen
f55873b783
Fix PixArt 256px inference ( #6789 )
...
* feat 256px diffusers inference bug
* change the max_length of T5 to pipeline config file
* fix bug in convert_pixart_alpha_to_diffusers.py
* Update scripts/convert_pixart_alpha_to_diffusers.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* remove multi_scale_train parser
* Update src/diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* styling
* change `model_token_max_length` to call argument.
* Refactoring
* add: max_sequence_length to the docstring.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-03-03 10:31:21 +05:30
Dhruv Nair
8f2d13c684
Fix setting fp16 dtype in AnimateDiff convert script. ( #7127 )
...
* update
* update
2024-02-29 22:47:39 +05:30
Dhruv Nair
d603ccb614
Small change to download in dance diffusion convert script ( #7070 )
...
* update
* make style
2024-02-26 12:05:19 +05:30
Sayak Paul
371f765908
[Diffusers -> Original SD conversion] fix things ( #6933 )
...
* fix: bias loading bug
* fixes for SDXL
* apply changes to the conversion script to match single_file_utils.py
* do transpose to match the single file loading logic.
2024-02-12 17:30:22 +05:30
Sayak Paul
30e5e81d58
change to 2024 in the license ( #6902 )
...
change to 2024
2024-02-08 08:19:31 -10:00
Patryk Bartkowiak
3ac2357794
changed positional parameters to named parameters like in docs ( #6905 )
...
Co-authored-by: Patryk Bartkowiak <patryk.bartkowiak@tcl.com >
Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com >
2024-02-08 21:39:03 +05:30
Sayak Paul
1835510524
Remove torch_dtype in to() to end deprecation ( #6886 )
...
* remove torch_dtype from to()
* remove torch_dtype from usage scripts.
* remove old lora backend
* Revert "remove old lora backend"
This reverts commit adcddf6ba4
2024-02-08 09:38:57 +05:30
Sayak Paul
04cd6adf8c
[Feat] add I2VGenXL for image-to-video generation ( #6665 )
...
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-01-31 10:38:51 -10:00
Sayak Paul
09b7bfce91
[Core] move transformer scripts to transformers modules ( #6747 )
...
* move transformer scripts to transformers modules
* move transformer model test
* move prior transformer test to directory
* fix doc path
* correct doc path
* add: __init__.py
2024-01-29 22:28:28 +05:30
Sayak Paul
1f0705adcf
[Big refactor] move unets to unets module 🦋 ( #6630 )
...
* move unets to module 🦋
* parameterize unet-level import.
* fix flax unet2dcondition model import
* models __init__
* mildly depcrecating models.unet_2d_blocks in favor of models.unets.unet_2d_blocks.
* noqa
* correct depcrecation behaviour
* inherit from the actual classes.
* Empty-Commit
* backwards compatibility for unet_2d.py
* backward compatibility for unet_2d_condition
* bc for unet_1d
* bc for unet_1d_blocks
2024-01-23 08:57:58 +05:30
Sayak Paul
cb4b3f0b78
[OmegaConf] replace it with yaml ( #6488 )
...
* remove omegaconf from convert_from_ckpt.
* remove from single_file.
* change to string based ubscription.
* style
* okay
* fix: vae_param
* no . indexing.
* style
* style
* turn getattrs into explicit if/else
* style
* propagate changes to ldm_uncond.
* propagate to gligen
* propagate to if.
* fix: quotes.
* propagate to audioldm.
* propagate to audioldm2
* propagate to musicldm.
* propagate to vq_diffusion
* propagate to zero123.
* remove omegaconf from diffusers codebase.
2024-01-15 20:02:10 +05:30
apolinário
0b63ad5ad5
Create convert_diffusers_sdxl_lora_to_webui.py ( #6395 )
...
* Create convert_diffusers_sdxl_lora_to_webui.py
* Move some conversion logic to utils
* fix logging import
* Add usage example
---------
Co-authored-by: multimodalart <joaopaulo.passos+multimodal@gmail.com >
2023-12-30 08:15:11 -06:00
Dhruv Nair
fb02316db8
Add AnimateDiff conversion scripts ( #6340 )
...
* add scripts
* update
2023-12-26 22:40:00 +05:30
Will Berman
4039815276
open muse ( #5437 )
...
amused
rename
Update docs/source/en/api/pipelines/amused.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
AdaLayerNormContinuous default values
custom micro conditioning
micro conditioning docs
put lookup from codebook in constructor
fix conversion script
remove manual fused flash attn kernel
add training script
temp remove training script
add dummy gradient checkpointing func
clarify temperatures is an instance variable by setting it
remove additional SkipFF block args
hardcode norm args
rename tests folder
fix paths and samples
fix tests
add training script
training readme
lora saving and loading
non-lora saving/loading
some readme fixes
guards
Update docs/source/en/api/pipelines/amused.md
Co-authored-by: Suraj Patil <surajp815@gmail.com >
Update examples/amused/README.md
Co-authored-by: Suraj Patil <surajp815@gmail.com >
Update examples/amused/train_amused.py
Co-authored-by: Suraj Patil <surajp815@gmail.com >
vae upcasting
add fp16 integration tests
use tuple for micro cond
copyrights
remove casts
delegate to torch.nn.LayerNorm
move temperature to pipeline call
upsampling/downsampling changes
2023-12-21 11:40:55 -08:00
d8ahazard
6976cab7ca
Fix possible re-conversion issues after extracting from safetensors ( #6097 )
...
* Fix possible re-conversion issues after extracting from diffusers
Properly rename specific vae keys.
* Whoops
2023-12-18 11:51:20 +01:00
Sayak Paul
56b3b21693
[Refactor autoencoders] feat: introduce autoencoders module ( #6129 )
...
* feat: introduce autoencoders module
* more changes for styling and copy fixing
* path changes in the docs.
* fix: import structure in init.
* fix controlnetxs import
2023-12-18 12:42:15 +05:30
Suraj Patil
63f767ef15
Add SVD ( #5895 )
...
* begin model
* finish blocks
* add_embedding
* addition_time_embed_dim
* use TimestepEmbedding
* fix temporal res block
* fix time_pos_embed
* fix add_embedding
* add conversion script
* fix model
* up
* add new resnet blocks
* make forward work
* return sample in original shape
* fix temb shape in TemporalResnetBlock
* add spatio temporal transformers
* add vae blocks
* fix blocks
* update
* update
* fix shapes in Alphablender and add time activation in res blcok
* use new blocks
* style
* fix temb shape
* fix SpatioTemporalResBlock
* reuse TemporalBasicTransformerBlock
* fix TemporalBasicTransformerBlock
* use TransformerSpatioTemporalModel
* fix TransformerSpatioTemporalModel
* fix time_context dim
* clean up
* make temb optional
* add blocks
* rename model
* update conversion script
* remove UNetMidBlockSpatioTemporal
* add in init
* remove unused arg
* remove unused arg
* remove more unsed args
* up
* up
* check for None
* update vae
* update up/mid blocks for decoder
* begin pipeline
* adapt scheduler
* add guidance scalings
* fix norm eps in temporal transformers
* add temporal autoencoder
* make pipeline run
* fix frame decodig
* decode in float32
* decode n frames at a time
* pass decoding_t to decode_latents
* fix decode_latents
* vae encode/decode in fp32
* fix dtype in TransformerSpatioTemporalModel
* type image_latents same as image_embeddings
* allow using differnt eps in temporal block for video decoder
* fix default values in vae
* pass num frames in decode
* switch spatial to temporal for mixing in VAE
* fix num frames during split decoding
* cast alpha to sample dtype
* fix attention in MidBlockTemporalDecoder
* fix typo
* fix guidance_scales dtype
* fix missing activation in TemporalDecoder
* skip_post_quant_conv
* add vae conversion
* style
* take guidance scale as input
* up
* allow passing PIL to export_video
* accept fps as arg
* add pipeline and vae in init
* remove hack
* use AutoencoderKLTemporalDecoder
* don't scale image latents
* add unet tests
* clean up unet
* clean TransformerSpatioTemporalModel
* add slow svd test
* clean up
* make temb optional in Decoder mid block
* fix norm eps in TransformerSpatioTemporalModel
* clean up temp decoder
* clean up
* clean up
* use c_noise values for timesteps
* use math for log
* update
* fix copies
* doc
* upcast vae
* update forward pass for gradient checkpointing
* make added_time_ids is tensor
* up
* fix upcasting
* remove post quant conv
* add _resize_with_antialiasing
* fix _compute_padding
* cleanup model
* more cleanup
* more cleanup
* more cleanup
* remove freeu
* remove attn slice
* small clean
* up
* up
* remove extra step kwargs
* remove eta
* remove dropout
* remove callback
* remove merge factor args
* clean
* clean up
* move to dedicated folder
* remove attention_head_dim
* docstr and small fix
* update unet doc strings
* rename decoding_t
* correct linting
* store c_skip and c_out
* cleanup
* clean TemporalResnetBlock
* more cleanup
* clean up vae
* clean up
* begin doc
* more cleanup
* up
* up
* doc
* Improve
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* better naming
* Apply suggestions from code review
* Default chunk size to None
* add example
* Better
* Apply suggestions from code review
* update doc
* Update src/diffusers/pipelines/stable_diffusion_video/pipeline_stable_diffusion_video.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* style
* Get torch compile working
* up
* rename
* fix doc
* add chunking
* torch compile
* torch compile
* add modelling outputs
* torch compile
* Improve chunking
* Apply suggestions from code review
* Update docs/source/en/using-diffusers/svd.md
* Close diff tag
* remove slicing
* resnet docstr
* add docstr in resnet
* rename
* Apply suggestions from code review
* update tests
* Fix output type latents
* fix more
* fix more
* Update docs/source/en/using-diffusers/svd.md
* fix more
* add pipeline tests
* remove unused arg
* clean up
* make sure get_scaling receives tensors
* fix euler scheduler
* fix get_scalings
* simply euler for now
* remove old test file
* use randn_tensor to create noise
* fix device for rand tensor
* increase expected_max_difference
* fix test_inference_batch_single_identical
* actually fix test_inference_batch_single_identical
* disable test_save_load_float16
* skip test_float16_inference
* skip test_inference_batch_single_identical
* fix test_xformers_attention_forwardGenerator_pass
* Apply suggestions from code review
* update StableVideoDiffusionPipelineSlowTests
* update image
* add diffusers example
* fix more
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
2023-11-29 19:13:36 +01:00
Patrick von Platen
b978334d71
[@cene555][Kandinsky 3.0] Add Kandinsky 3.0 ( #5913 )
...
* finalize
* finalize
* finalize
* add slow test
* add slow test
* add slow test
* Fix more
* add slow test
* fix more
* fix more
* fix more
* fix more
* fix more
* fix more
* fix more
* fix more
* fix more
* Better
* Fix more
* Fix more
* add slow test
* Add auto pipelines
* add slow test
* Add all
* add slow test
* add slow test
* add slow test
* add slow test
* add slow test
* Apply suggestions from code review
* add slow test
* add slow test
2023-11-24 17:46:00 +01:00
Kashif Rasul
6b04d61cf6
[Styling] stylify using ruff ( #5841 )
...
* ruff format
* not need to use doc-builder's black styling as the doc is styled in ruff
* make fix-copies
* comment
* use run_ruff
2023-11-20 11:48:34 +01:00
Lucain
c896b841e4
Set usedforsecurity=False in hashlib methods (FIPS compliance) ( #5790 )
...
* Set usedforsecurity=False in hashlib methods (FIPS compliance)
* update version dependency
* bump hfh version
* bump hfh version
2023-11-17 14:56:58 +01:00
Will Berman
2fd46405cd
consistency decoder ( #5694 )
...
* consistency decoder
* rename
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update src/diffusers/pipelines/consistency_models/pipeline_consistency_models.py
* uP
* Apply suggestions from code review
* uP
* uP
* uP
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-11-09 12:21:41 +01:00
Sayak Paul
d61889fc17
[Feat] PixArt-Alpha ( #5642 )
...
* init pixart alpha pipeline
* fix: import
* script
* script
* script
* add: vae to the pipeline
* add: vae_scale_factor
* add: checkpoint_path
* clean conversion script a bit.
* size embeddings.
* fix: size embedding
* update scrip
* support for interpolation of position embedding.
* support for conditioning.
* ..
* ..
* ..
* final layer
* final layer
* align if encode_prompt
* support for caption embedding
* refactor
* refactor
* refactor
* start cross attention
* start cross attention
* cross_attention_dim
* cross
* cross
* support for resolution and aspect_ratio
* support for caption projection
* refactor patch embeddings
* batch_size
* up
* commit
* commit
* commit.
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze
* squeeze.
* squeeze.
* fix final block./
* fix final block./
* fix final block./
* clean
* fix: interpolation scale.
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging'
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* make --checkpoint_path non-required.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* remove num_tokens
* timesteps -> timestep
* timesteps -> timestep
* timesteps -> timestep
* timesteps -> timestep
* timesteps -> timestep
* timesteps -> timestep
* debug
* debug
* update conversion script.
* update conversion script.
* update conversion script.
* debug
* debug
* debug
* clean
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* debug
* deug
* debug
* debug
* debug
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* clean
* fix
* fix
* boom
* boom
* some changes
* boom
* save
* up
* remove i
* fix more tests
* DPMSolverMultistepScheduler
* fix
* offloading
* fix conversion script
* fix conversion script
* remove print
* remove support for negative prompt embeds.
* typo.
* remove extra kwargs
* bring conversion script to where it was
* fix
* trying mu luck
* trying my luck again
* again
* again
* again
* clean up
* up
* up
* update example
* support for 512
* remove spacing
* finalize docs.
* test debug
* fix: assertion values.
* debug
* debug
* debug
* fix: repeat
* remove prints.
* Apply suggestions from code review
* Apply suggestions from code review
* Correct more
* Apply suggestions from code review
* Change all
* Clean more
* fix more
* Fix more
* Fix more
* Correct more
* address patrick's comments.
* remove unneeded args
* clean up pipeline.
* sty;e
* make the use of additional conditions better conditioned.
* None better
* dtype
* height and width validation
* add a note about size brackets.
* fix
* spit out slow test outputs.
* fix?
* fix optional test
* fix more
* remove unneeded comment
* debug
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-11-06 08:40:04 +01:00
dg845
cd1b8d7ca8
[WIP] Refactor UniDiffuser Pipeline and Tests ( #4948 )
...
* Add VAE slicing and tiling methods.
* Switch to using VaeImageProcessing for preprocessing and postprocessing of images.
* Rename the VaeImageProcessor to vae_image_processor to avoid a name clash with the CLIPImageProcessor (image_processor).
* Remove the postprocess() function because we're using a VaeImageProcessor instead.
* Remove UniDiffuserPipeline.decode_image_latents because we're using VaeImageProcessor instead.
* Refactor generating text from text latents into a decode_text_latents method.
* Add enable_full_determinism() to UniDiffuser tests.
* make style
* Add PipelineLatentTesterMixin to UniDiffuserPipelineFastTests.
* Remove enable_model_cpu_offload since it is now part of DiffusionPipeline.
* Rename the VaeImageProcessor instance to self.image_processor for consistency with other pipelines and rename the CLIPImageProcessor instance to clip_image_processor to avoid a name clash.
* Update UniDiffuser conversion script.
* Make safe_serialization configurable in UniDiffuser conversion script.
* Rename image_processor to clip_image_processor in UniDiffuser tests.
* Add PipelineKarrasSchedulerTesterMixin to UniDiffuserPipelineFastTests.
* Add initial test for compiling the UniDiffuser model (not tested yet).
* Update encode_prompt and _encode_prompt to match that of StableDiffusionPipeline.
* Turn off standard classifier-free guidance for now.
* make style
* make fix-copies
* apply suggestions from review
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-10-02 18:24:55 +02:00
Ayush Mangal
157c9011d8
Add BLIP Diffusion ( #4388 )
...
* Add BLIP Diffusion skeleton
* Add other model components
* Add BLIP2, need to change it for now
* Fix pipeline imports
* Load pretrained ViT
* Make qformer fwd pass same
* Replicate fwd passes
* Fix device bug
* Add accelerate functions
* Remove extra functions from Blip2
* Minor bug
* Integrate initial review changes
* Refactoring
* Refactoring
* Refactor
* Add controlnet
* Refactor
* Update conversion script
* Add image processor
* Shift postprocessing to ImageProcessor
* Refactor
* Fix device
* Add fast tests
* Update conversion script
* Fix checkpoint conversion script
* Integrate review changes
* Integrate reivew changes
* Remove unused functions from test
* Reuse HF image processor in Cond image
* Create new BlipImageProcessor based on transfomers
* Fix image preprocessor
* Minor
* Minor
* Add canny preprocessing
* Fix controlnet preprocessing
* Fix blip diffusion test
* Add controlnet test
* Add initial doc strings
* Integrate review changes
* Refactor
* Update examples
* Remove DDIM comments
* Add copied from for prepare_latents
* Add type anotations
* Add docstrings
* Do black formatting
* Add batch support
* Make tests pass
* Make controlnet tests pass
* Black formatting
* Fix progress bar
* Fix some licensing comments
* Fix imports
* Refactor controlnet
* Make tests faster
* Edit examples
* Black formatting/Ruff
* Add doc
* Minor
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Move controlnet pipeline
* Make tests faster
* Fix imports
* Fix formatting
* Fix make errors
* Fix make errors
* Minor
* Add suggested doc changes
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Edit docs
* Fix 16 bit loading
* Update examples
* Edit toctree
* Update docs/source/en/api/pipelines/blip_diffusion.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Minor
* Add tips
* Edit examples
* Update model paths
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-21 17:05:35 +01:00
김태민
5b78141fd3
[FIX BUG] add config_files parser #5114 ( #5115 )
...
* add config_files parser #5114
* add config_files parser_fix #5114
2023-09-20 16:17:47 +02:00
dg845
4c8a05f115
Fix Consistency Models UNet2DMidBlock2D Attention GroupNorm Bug ( #4863 )
...
* Add attn_groups argument to UNet2DMidBlock2D to control theinternal Attention block's GroupNorm.
* Add docstring for attn_norm_num_groups in UNet2DModel.
* Since the test UNet config uses resnet_time_scale_shift == 'scale_shift', also set attn_norm_num_groups to 32.
* Add test for attn_norm_num_groups to UNet2DModelTests.
* Fix expected slices for slow tests.
* Also fix tolerances for slow tests.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-15 11:27:51 +01:00
Vladimir Mandic
ef29b24fda
allow loading of sd models from safetensors without online lookups using local config files ( #5019 )
...
finish config_files implementation
2023-09-14 12:30:15 +02:00
Kashif Rasul
16a056a7b5
Wuerstchen fixes ( #4942 )
...
* fix arguments and make example code work
* change arguments in combined test
* Add default timesteps
* style
* fixed test
* fix broken test
* formatting
* fix docstrings
* fix num_images_per_prompt
* fix doc styles
* please dont change this
* fix tests
* rename to DEFAULT_STAGE_C_TIMESTEPS
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
2023-09-11 15:47:53 +02:00
Kashif Rasul
541bb6ee63
Würstchen model ( #3849 )
...
* initial
* initial
* added initial convert script for paella vqmodel
* initial wuerstchen pipeline
* add LayerNorm2d
* added modules
* fix typo
* use model_v2
* embed clip caption amd negative_caption
* fixed name of var
* initial modules in one place
* WuerstchenPriorPipeline
* inital shape
* initial denoising prior loop
* fix output
* add WuerstchenPriorPipeline to __init__.py
* use the noise ratio in the Prior
* try to save pipeline
* save_pretrained working
* Few additions
* add _execution_device
* shape is int
* fix batch size
* fix shape of ratio
* fix shape of ratio
* fix output dataclass
* tests folder
* fix formatting
* fix float16 + started with generator
* Update pipeline_wuerstchen.py
* removed vqgan code
* add WuerstchenGeneratorPipeline
* fix WuerstchenGeneratorPipeline
* fix docstrings
* fix imports
* convert generator pipeline
* fix convert
* Work on Generator Pipeline. WIP
* Pipeline works with our diffuzz code
* apply scale factor
* removed vqgan.py
* use cosine schedule
* redo the denoising loop
* Update src/diffusers/models/resnet.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* use torch.lerp
* use warp-diffusion org
* clip_sample=False,
* some refactoring
* use model_v3_stage_c
* c_cond size
* use clip-bigG
* allow stage b clip to be None
* add dummy
* würstchen scheduler
* minor changes
* set clip=None in the pipeline
* fix attention mask
* add attention_masks to text_encoder
* make fix-copies
* add back clip
* add text_encoder
* gen_text_encoder and tokenizer
* fix import
* updated pipeline test
* undo changes to pipeline test
* nip
* fix typo
* fix output name
* set guidance_scale=0 and remove diffuze
* fix doc strings
* make style
* nip
* removed unused
* initial docs
* rename
* toc
* cleanup
* remvoe test script
* fix-copies
* fix multi images
* remove dup
* remove unused modules
* undo changes for debugging
* no new line
* remove dup conversion script
* fix doc string
* cleanup
* pass default args
* dup permute
* fix some tests
* fix prepare_latents
* move Prior class to modules
* offload only the text encoder and vqgan
* fix resolution calculation for prior
* nip
* removed testing script
* fix shape
* fix argument to set_timesteps
* do not change .gitignore
* fix resolution calculations + readme
* resolution calculation fix + readme
* small fixes
* Add combined pipeline
* rename generator -> decoder
* Update .gitignore
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* removed efficient_net
* create combined WuerstchenPipeline
* make arguments consistent with VQ model
* fix var names
* no need to return text_encoder_hidden_states
* add latent_dim_scale to config
* split model into its own file
* add WuerschenPipeline to docs
* remove unused latent_size
* register latent_dim_scale
* update script
* update docstring
* use Attention preprocessor
* concat with normed input
* fix-copies
* add docs
* fix test
* fix style
* add to cpu_offloaded_model
* updated type
* remove 1-line func
* updated type
* initial decoder test
* formatting
* formatting
* fix autodoc link
* num_inference_steps is int
* remove comments
* fix example in docs
* Update src/diffusers/pipelines/wuerstchen/diffnext.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* rename layernorm to WuerstchenLayerNorm
* rename DiffNext to WuerstchenDiffNeXt
* added comment about MixingResidualBlock
* move paella vq-vae to pipelines' folder
* initial decoder test
* increased test_float16_inference expected diff
* self_attn is always true
* more passing decoder tests
* batch image_embeds
* fix failing tests
* set the correct dtype
* relax inference test
* update prior
* added combined pipeline test
* faster test
* faster test
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix issues from review
* update wuerstchen.md + change generator name
* resolve issues
* fix copied from usage and add back batch_size
* fix API
* fix arguments
* fix combined test
* Added timesteps argument + fixes
* Update tests/pipelines/test_pipelines_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update tests/pipelines/wuerstchen/test_wuerstchen_prior.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
* up
* Fix more
* failing tests
* up
* up
* correct naming
* correct docs
* correct docs
* fix test params
* correct docs
* fix classifier free guidance
* fix classifier free guidance
* fix more
* fix all
* make tests faster
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Dominic Rampas <61938694+dome272@users.noreply.github.com >
2023-09-06 16:15:51 +02:00
Nguyễn Công Tú Anh
38466c369f
Add GLIGEN Text Image implementation ( #4777 )
...
* Add GLIGEN Text Image implementation
* add style transfer from image
* fix check_repository_consistency
* add convert script GLIGEN model to Diffusers
* rename attention type
* fix style code
* remove PositionNetTextImage
* Revert "fix check_repository_consistency"
This reverts commit 15f098c96e
2023-09-01 15:48:01 +05:30
Alexsey Shestacov
3eeaf4e041
Fix convert_original_stable_diffusion_to_diffusers script ( #4817 )
...
Fix stable diffusion conversion script
2023-08-29 09:14:45 +02:00
Sanchit Gandhi
b1290d3fb8
Convert MusicLDM ( #4579 )
...
* from audioldm
* fix vae
* move to new pipeline
* copied from audioldm
* remove redundant control flow
* iterate
* fix docstring
* finish pipeline
* tests: from audioldm2
* iterate
* finish fast tests
* finish slow integration tests
* add docs
* remove dtype test
* update toctree
* "copied from" in conversion (where possible)
* Update docs/source/en/api/pipelines/musicldm.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix docstring
* make nightly
* style
* fix dtype test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-25 13:31:00 +01:00
realliujiaxu
ecded50ad5
add convert diffuser pipeline of XL to original stable diffusion ( #4596 )
...
convert diffuser pipeline of XL to original stable diffusion
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-22 19:11:06 +05:30
Sanchit Gandhi
7a24977ce3
Add AudioLDM 2 ( #4549 )
...
* from audioldm
* unet down + mid
* vae, clap, flan-t5
* start sequence audio mae
* iterate on audioldm encoder
* finish encoder
* finish weight conversion
* text pre-processing
* gpt2 pre-processing
* fix projection model
* working
* unet equivalence
* finish in base
* add unet cond
* finish unet
* finish custom unet
* start clean-up
* revert base unet changes
* refactor pre-processing
* tests: from audioldm
* fix some tests
* more fixes
* iterate on tests
* make fix copies
* harden fast tests
* slow integration tests
* finish tests
* update checkpoint
* update copyright
* docs
* remove outdated method
* add docstring
* make style
* remove decode latents
* enable cpu offload
* (text_encoder_1, tokenizer_1) -> (text_encoder, tokenizer)
* more clean up
* more refactor
* build pr docs
* Update docs/source/en/api/pipelines/audioldm2.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* small clean
* tidy conversion
* update for large checkpoint
* generate -> generate_language_model
* full clap model
* shrink clap-audio in tests
* fix large integration test
* fix fast tests
* use generation config
* make style
* update docs
* finish docs
* finish doc
* update tests
* fix last test
* syntax
* finalise tests
* refactor projection model in prep for TTS
* fix fast tests
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-21 12:34:21 +01:00
AisingioroHao
1b739e7344
Fixed invalid pipeline_class_name parameter. ( #4590 )
...
* Fixed invalid pipeline_class_name parameter.
* Fix the format
2023-08-14 17:21:17 +05:30
Abhipsha Das
c8d86e9f0a
Remove code snippets containing is_safetensors_available() ( #4521 )
...
* [WIP] Remove code snippets containing `is_safetensors_available()`
* Modifying `import_utils.py`
* update pipeline tests for safetensor default
* fix test related to cached requests
* address import nits
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-11 11:05:22 +05:30
dotieuthien
b28cd3fba0
Convert Stable Diffusion ControlNet to TensorRT ( #4465 )
...
* convert tensorrt controlnet
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix number controlnet condition
* Add convert SD XL to onnx
* Add convert SD XL to tensorrt
* Add convert SD XL to tensorrt
* Add examples in comments
* Add examples in comments
* Add test onnx controlnet
* Add tensorrt test
* Remove copied
* Move file test to examples/community
* Remove script
* Remove script
* Remove text
---------
Co-authored-by: dotieuthien <thien.do@mservice.com.vn >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-11 08:12:26 +05:30
VV-A-VV
3fd45eb10f
fix some typo error ( #4546 )
...
* fix some typo error
* Undo changes to capitalization
2023-08-10 06:49:25 +05:30
YiYi Xu
aef11cbf66
add pipeline_class_name argument to Stable Diffusion conversion script ( #4461 )
...
* add pipeline class
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* style
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-07 06:44:31 -10:00
Sayak Paul
18fc40c169
[Feat] add tiny Autoencoder for (almost) instant decoding ( #4384 )
...
* add: model implementation of tiny autoencoder.
* add: inits.
* push the latest devs.
* add: conversion script and finish.
* add: scaling factor args.
* debugging
* fix denormalization.
* fix: positional argument.
* handle use_torch_2_0_or_xformers.
* handle post_quant_conv
* handle dtype
* fix: sdxl image processor for tiny ae.
* fix: sdxl image processor for tiny ae.
* unify upcasting logic.
* copied from madness.
* remove trailing whitespace.
* set is_tiny_vae = False
* address PR comments.
* change to AutoencoderTiny
* make act_fn an str throughout
* fix: apply_forward_hook decorator call
* get rid of the special is_tiny_vae flag.
* directly scale the output.
* fix dummies?
* fix: act_fn.
* get rid of the Clamp() layer.
* bring back copied from.
* movement of the blocks to appropriate modules.
* add: docstrings to AutoencoderTiny
* add: documentation.
* changes to the conversion script.
* add doc entry.
* settle tests.
* style
* add one slow test.
* fix
* fix 2
* fix 2
* fix: 4
* fix: 5
* finish integration tests
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* style
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-02 23:58:05 +05:30
