M. Tolga Cangöz
ef4c2fa4f1
Update alt_diffusion.mdx ( #2865 )
...
Fix typos
2023-03-28 17:17:53 +01:00
M. Tolga Cangöz
3980858ad4
Update overview.mdx ( #2864 )
...
Fix typos
2023-03-28 17:17:33 +01:00
M. Tolga Cangöz
37c82480bb
Update evaluation.mdx ( #2862 )
...
Fix typos
2023-03-28 17:15:37 +01:00
Sayak Paul
13845462db
[Tests] Adds a test to check if image_embeds None case is handled properly in StableUnCLIPImg2ImgPipeline ( #2861 )
...
* improve stable unclip doc.
* add: test to check if image_emebds None case is handled.
* apply formatting/
2023-03-28 17:14:08 +01:00
Nipun Jindal
53377ef83c
[2761]: Add documentation for extra_in_channels UNet1DModel ( #2817 )
...
Co-authored-by: njindal <njindal@adobe.com >
2023-03-28 16:56:45 +01:00
dg845
4d0f412d0d
[WIP] Check UNet shapes in StableDiffusionInpaintPipeline __init__ ( #2853 )
...
Add warning in __init__ if user loads a checkpoint with pipeline.unet.config.in_channels other than 9.
2023-03-28 16:53:52 +01:00
Felix Blanke
25d927aa51
Add last_epoch argument to optimization.get_scheduler ( #2850 )
...
Add last_epoch arg to optimization.get_scheduler.
Allows the specification of the index of the last epoch when
resuming training.
2023-03-28 16:46:41 +01:00
dg845
663c654577
[WIP][Docs] Use DiffusionPipeline Instead of Child Classes when Loading Pipeline ( #2809 )
...
* Change the docs to use the parent DiffusionPipeline class when loading a checkpoint using from_pretrained() instead of a child class (e.g. StableDiffusionPipeline) where possible.
* Run make style to fix style issues.
* Change more docs to use DiffusionPipeline rather than a subclass.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-28 16:44:34 +01:00
John HU
920a15cf70
Fix link to LoRA training guide in DreamBooth training guide ( #2836 )
...
Fix link to LoRA training guide
2023-03-28 16:35:41 +01:00
cmdr2
7d756813d4
Update the legacy inpainting SD pipeline, to allow calling it with only prompt_embeds (instead of always requiring a prompt) ( #2842 )
...
Fix error 'required positional argument: prompt' when Legacy Inpaint is called only with prompt_embeds
2023-03-28 16:30:49 +01:00
Li-Huai (Allan) Lin
159a0bff34
Remove duplicate sentence in docstrings ( #2834 )
...
* Remove duplicate sentence
* format
2023-03-28 16:27:51 +01:00
Sandeep
b76d9fde8d
Remove suggestion to use cuDNN benchmark in docs ( #2793 )
...
* Remove suggestion to use cuDNN benchmark in docs
* removing the wrong line
2023-03-28 16:01:30 +01:00
Aki Sakurai
0f14335af3
StableDiffusionLongPromptWeightingPipeline: Do not hardcode pad token ( #2832 )
2023-03-28 16:00:56 +01:00
junhsss
8bdf423645
fix KarrasVePipeline bug ( #2828 )
2023-03-28 15:58:19 +01:00
Stax124
585f621af2
[Stable Diffusion] Allow users to disable Safety checker if loading model from checkpoint ( #2768 )
...
* Allow user to disable SafetyChecker and enable dtypes if loading models from .ckpt or .safetensors
* Fix Import sorting (Ruff error)
* Get rid of the dtype convert method as it was implemented all along
* Fix the docstring
* Fix ruff formatting
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-28 15:06:48 +01:00
Kashif Rasul
c0afca2d12
updated onnx pndm test ( #2811 )
2023-03-28 13:43:24 +01:00
Patrick von Platen
42d950174f
[Init] Make sure shape mismatches are caught early ( #2847 )
...
Improve init
2023-03-28 09:08:28 +01:00
Pedro Cuenca
81125d8499
Make dynamo wrapped modules work with save_pretrained ( #2726 )
...
* Workaround for saving dynamo-wrapped models.
* Accept suggestion from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Apply workaround when overriding pipeline components.
* Ensure the correct config.json is saved to disk.
Instead of the dynamo class.
* Save correct module (not compiled one)
* Add test
* style
* fix docstrings
* Go back to using string comparisons.
PyTorch CPU does not have _dynamo.
* Simple test for save_pretrained of compiled models.
* Helper function to test whether module is compiled.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-28 09:03:21 +02:00
YiYi Xu
d4f846fa74
[WIP]Flax training script for controlnet ( #2818 )
...
* add train_controlnet_flax
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-27 19:13:35 -10:00
Sayak Paul
58fc824488
add: better warning messages when handling multiple conditionings. ( #2804 )
...
* add: better warning messages when handling multiple conditioning.
* fix: handling of controlnet_conditioning_scale
2023-03-28 08:19:39 +05:30
Sayak Paul
fab4f3d6e4
improve stable unclip doc. ( #2823 )
2023-03-28 08:18:29 +05:30
Pedro Cuenca
b10f527577
Helper function to disable custom attention processors ( #2791 )
...
* Helper function to disable custom attention processors.
* Restore code deleted by mistake.
* Format
* Fix modeling_text_unet copy.
2023-03-27 20:31:19 +02:00
Eugene Lyapustin
7bc2fff1a5
Fix StableUnCLIPImg2ImgPipeline handling of explicitly passed image embeddings ( #2845 )
2023-03-27 19:03:59 +01:00
Patrick von Platen
4c26cb9cc8
[Tests] Fix slow tests ( #2846 )
2023-03-27 18:45:49 +01:00
Pedro Cuenca
1d7b4b60b7
Ruff: apply same rules as in transformers ( #2827 )
...
* Apply same ruff settings as in transformers
See https://github.com/huggingface/transformers/blob/main/pyproject.toml
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com >
* Apply new style rules
* Style
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com >
* style
* remove list, ruff wouldn't auto fix.
---------
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com >
2023-03-27 16:18:57 +02:00
Sayak Paul
abb22b4eeb
Update examples README.md to include the latest examples ( #2839 )
2023-03-27 19:34:58 +05:30
Bahjat Kawar
9fb0217548
StableDiffusionModelEditingPipeline documentation ( #2810 )
...
* comment update
* comment update
2023-03-24 22:41:31 +05:30
Sayak Paul
5883d8d4d1
[Docs] update docs (Stable unCLIP) to reflect the updated ckpts. ( #2815 )
...
* update docs to reflect the updated ckpts.
* update: point about prompt.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* emove image resizing.
* Apply suggestions from code review
* Apply suggestions from code review
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-24 17:24:19 +01:00
Patrick von Platen
dbcb15c25f
[Stable UnCLIP] Finish Stable UnCLIP ( #2814 )
...
* up
* fix more 7
* up
* finish
2023-03-24 17:04:41 +01:00
PeixuanZuo
c4892f1855
Update onnxruntime package candidates ( #2666 )
...
* update import onnxruntime package, enable onnxruntime-rocm and onnxruntime-training
* add ort_nightly_gpu
2023-03-24 12:23:05 +01:00
Kashif Rasul
f6feb69991
Relax DiT test ( #2808 )
...
* Relax DiT test
* relax 2 more tests
* fix style
* skip test on mac due to older protobuf
2023-03-24 11:28:55 +01:00
Bahjat Kawar
37a44bb283
Add ModelEditing pipeline ( #2721 )
...
* TIME first commit
* styling.
* styling 2.
* fixes; tests
* apply styling and doc fix.
* remove sups.
* fixes
* remove temp file
* move augmentations to const
* added doc entry
* code quality
* customize augmentations
* quality
* quality
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-03-24 13:01:39 +05:30
Haofan Wang
4a98d6e097
Update train_text_to_image_lora.py ( #2795 )
2023-03-24 11:45:35 +05:30
Sanchit Gandhi
b94880e536
Add AudioLDM ( #2232 )
...
* Add AudioLDM
* up
* add vocoder
* start unet
* unconditional unet
* clap, vocoder and vae
* clean-up: conversion scripts
* fix: conversion script token_type_ids
* clean-up: pipeline docstring
* tests: from SD
* clean-up: cpu offload vocoder instead of safety checker
* feat: adapt tests to audioldm
* feat: add docs
* clean-up: amend pipeline docstrings
* clean-up: make style
* clean-up: make fix-copies
* fix: add doc path to toctree
* clean-up: args for conversion script
* clean-up: paths to checkpoints
* fix: use conditional unet
* clean-up: make style
* fix: type hints for UNet
* clean-up: docstring for UNet
* clean-up: make style
* clean-up: remove duplicate in docstring
* clean-up: make style
* clean-up: make fix-copies
* clean-up: move imports to start in code snippet
* fix: pass cross_attention_dim as a list/tuple to unet
* clean-up: make fix-copies
* fix: update checkpoint path
* fix: unet cross_attention_dim in tests
* film embeddings -> class embeddings
* Apply suggestions from code review
Co-authored-by: Will Berman <wlbberman@gmail.com >
* fix: unet film embed to use existing args
* fix: unet tests to use existing args
* fix: make style
* fix: transformers import and version in init
* clean-up: make style
* Revert "clean-up: make style"
This reverts commit 5d6d1f8b32patrick.v.platen@gmail.com >
Co-authored-by: William Berman <WLBberman@gmail.com >
2023-03-23 19:00:21 +01:00
Steven Liu
1870fb05a9
[docs] Add Colab notebooks and Spaces ( #2713 )
...
* add colab notebook and spaces
* fix image link
2023-03-23 09:48:58 -07:00
YiYi Xu
df91c44712
Flax controlnet ( #2727 )
...
* add contronet flax
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-03-23 05:46:23 -10:00
Pedro Cuenca
aa0531fa8d
Skip mps in text-to-video tests ( #2792 )
...
* Skip mps in text-to-video tests.
* style
* Skip UNet3D mps tests.
2023-03-23 14:39:03 +01:00
Haofan Wang
dc5b4e2342
Update train_text_to_image_lora.py ( #2767 )
...
* Update train_text_to_image_lora.py
* Update train_text_to_image_lora.py
* Update train_text_to_image_lora.py
* Update train_text_to_image_lora.py
* format
2023-03-23 14:28:47 +01:00
Sayak Paul
0d7aac3e8d
[Docs] small fixes to the text to video doc. ( #2787 )
...
* small fixes to the text to video doc.
* add: Spaces link.
* add: warning on research-only model.
2023-03-23 18:57:02 +05:30
Nipun Jindal
055c90f589
[2737]: Add DPMSolverMultistepScheduler to CLIP guided community pipeline ( #2779 )
...
[2737]: Add DPMSolverMultistepScheduler to CLIP guided community pipelines
Co-authored-by: njindal <njindal@adobe.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-23 14:20:24 +01:00
Kashif Rasul
2ef9bdd76f
Music Spectrogram diffusion pipeline ( #1044 )
...
* initial TokenEncoder and ContinuousEncoder
* initial modules
* added ContinuousContextTransformer
* fix copy paste error
* use numpy for get_sequence_length
* initial terminal relative positional encodings
* fix weights keys
* fix assert
* cross attend style: concat encodings
* make style
* concat once
* fix formatting
* Initial SpectrogramPipeline
* fix input_tokens
* make style
* added mel output
* ignore weights for config
* move mel to numpy
* import pipeline
* fix class names and import
* moved models to models folder
* import ContinuousContextTransformer and SpectrogramDiffusionPipeline
* initial spec diffusion converstion script
* renamed config to t5config
* added weight loading
* use arguments instead of t5config
* broadcast noise time to batch dim
* fix call
* added scale_to_features
* fix weights
* transpose laynorm weight
* scale is a vector
* scale the query outputs
* added comment
* undo scaling
* undo depth_scaling
* inital get_extended_attention_mask
* attention_mask is none in self-attention
* cleanup
* manually invert attention
* nn.linear need bias=False
* added T5LayerFFCond
* remove to fix conflict
* make style and dummy
* remove unsed variables
* remove predict_epsilon
* Move accelerate to a soft-dependency (#1134 )
* finish
* finish
* Update src/diffusers/modeling_utils.py
* Update src/diffusers/pipeline_utils.py
Co-authored-by: Anton Lozhkov <anton@huggingface.co >
* more fixes
* fix
Co-authored-by: Anton Lozhkov <anton@huggingface.co >
* fix order
* added initial midi to note token data pipeline
* added int to int tokenizer
* remove duplicate
* added logic for segments
* add melgan to pipeline
* move autoregressive gen into pipeline
* added note_representation_processor_chain
* fix dtypes
* remove immutabledict req
* initial doc
* use np.where
* require note_seq
* fix typo
* update dependency
* added note-seq to test
* added is_note_seq_available
* fix import
* added toc
* added example usage
* undo for now
* moved docs
* fix merge
* fix imports
* predict first segment
* avoid un-needed copy to and from cpu
* make style
* Copyright
* fix style
* add test and fix inference steps
* remove bogus files
* reorder models
* up
* remove transformers dependency
* make work with diffusers cross attention
* clean more
* remove @
* improve further
* up
* uP
* Apply suggestions from code review
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
* loop over all tokens
* make style
* Added a section on the model
* fix formatting
* grammer
* formatting
* make fix-copies
* Update src/diffusers/pipelines/__init__.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/spectrogram_diffusion/pipeline_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* added callback ad optional ionnx
* do not squeeze batch dim
* clean up more
* upload
* convert jax to nnumpy
* make style
* fix warning
* make fix-copies
* fix warning
* add initial fast tests
* add initial pipeline_params
* eval mode due to dropout
* skip batch tests as pipeline runs on a single file
* make style
* fix relative path
* fix doc tests
* Update src/diffusers/models/t5_film_transformer.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/models/t5_film_transformer.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update docs/source/en/api/pipelines/spectrogram_diffusion.mdx
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* add MidiProcessor
* format
* fix org
* Apply suggestions from code review
* Update tests/pipelines/spectrogram_diffusion/test_spectrogram_diffusion.py
* make style
* pin protobuf to <4
* fix formatting
* white space
* tensorboard needs protobuf
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Anton Lozhkov <anton@huggingface.co >
2023-03-23 14:06:17 +01:00
Naoki Ainoya
14e3a28c12
Rename 'CLIPFeatureExtractor' class to 'CLIPImageProcessor' ( #2732 )
...
The 'CLIPFeatureExtractor' class name has been renamed to 'CLIPImageProcessor' in order to comply with future deprecation. This commit includes the necessary changes to the affected files.
2023-03-23 13:49:22 +01:00
Mishig
8e35ef0142
[doc wip] literalinclude ( #2718 )
2023-03-23 13:42:54 +01:00
Patrick von Platen
a8315ce1a9
[UNet3DModel] Fix with attn processor ( #2790 )
...
* [UNet3DModel] Fix attn processor
* make style
2023-03-23 09:56:02 +01:00
Sayak Paul
0d633a42f4
deduplicate training section in the docs. ( #2788 )
2023-03-23 11:21:53 +05:30
Sayak Paul
9dc84448ac
[Examples] InstructPix2Pix instruct training script ( #2478 )
...
* add: initial implementation of the pix2pix instruct training script.
* shorten cli arg.
* fix: main process check.
* fix: dataset column names.
* simplify tokenization.
* proper placement of null conditions.
* apply styling.
* remove debugging message for conditioning do.
* complete license.
* add: requirements.tzt
* wandb column name order.
* fix: augmentation.
* change: dataset_id.
* fix: convert_to_np() call.
* fix: reshaping.
* fix: final ema copy.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* address PR comments.
* add: readme details.
* config fix.
* downgrade version.
* reduce image width in the readme.
* note on hyperparameters during generation.
* add: output images.
* update readme.
* minor edits to readme.
* debugging statement.
* explicitly placement of the pipeline.
* bump minimum diffusers version.
* fix: device attribute error.
* weight dtype.
* debugging.
* add dtype inform.
* add seoarate te and vae.
* add: explicit casting/
* remove casting.
* up.
* up 2.
* up 3.
* autocast.
* disable mixed-precision in the final inference.
* debugging information.
* autocasting.
* add: instructpix2pix training section to the docs.
* Empty-Commit
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-23 10:15:01 +05:30
Sayak Paul
c681ad1af2
add: section on multiple controlnets. ( #2762 )
...
* add: section on multiple controlnets.
Co-authored-by: William Berman <WLBberman@gmail.com >
* fix: docs.
* fix: docs.
---------
Co-authored-by: William Berman <WLBberman@gmail.com >
2023-03-23 09:55:25 +05:30
Haofan Wang
e0d8c9ef83
Support for Offset Noise in examples ( #2753 )
...
* add noise offset
* make style
2023-03-23 09:36:17 +05:30
Pedro Cuenca
92e1164e2e
mps: remove warmup passes (#2771 )
...
* Remove warmup passes in mps tests.
* Update mps docs: no warmup pass in PyTorch 2
* Update imports.
2023-03-22 19:29:27 +01:00
Patrick von Platen
ca1a22296d
[MS Text To Video] Add first text to video ( #2738 )
...
* [MS Text To Video} Add first text to video
* upload
* make first model example
* match unet3d params
* make sure weights are correcctly converted
* improve
* forward pass works, but diff result
* make forward work
* fix more
* finish
* refactor video output class.
* feat: add support for a video export utility.
* fix: opencv availability check.
* run make fix-copies.
* add: docs for the model components.
* add: standalone pipeline doc.
* edit docstring of the pipeline.
* add: right path to TransformerTempModel
* add: first set of tests.
* complete fast tests for text to video.
* fix bug
* up
* three fast tests failing.
* add: note on slow tests
* make work with all schedulers
* apply styling.
* add slow tests
* change file name
* update
* more correction
* more fixes
* finish
* up
* Apply suggestions from code review
* up
* finish
* make copies
* fix pipeline tests
* fix more tests
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* apply suggestions
* up
* revert
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2023-03-22 18:39:33 +01:00
