Sayak Paul
88735249da
[Docs] fix: minor formatting in the Würstchen docs ( #4965 )
...
fix: minor formatting in the docs
2023-09-11 09:12:53 +02:00
Will Berman
4191ddee11
Revert revert and install accelerate main ( #4963 )
...
* Revert "Temp Revert "[Core] better support offloading when side loading is enabled… (#4927 )"
This reverts commit 2ab170499e
2023-09-11 08:49:46 +02:00
Will Berman
2ab170499e
Temp Revert "[Core] better support offloading when side loading is enabled… ( #4927 )
...
Revert "[Core] better support offloading when side loading is enabled. (#4855 )"
This reverts commit e4b8e7928b
2023-09-08 19:54:59 -07:00
Sayak Paul
914c513ee0
[Docs] add t2i adapter entry to overview of training scripts. ( #4946 )
...
add t2i adapter entry to overview of training scripts.
2023-09-09 06:52:11 +05:30
Will Berman
d73e6ad050
guard save model hooks to only execute on main process ( #4929 )
2023-09-08 10:30:06 -07:00
Sayak Paul
d0cf681a1f
[Tests] add: tests for t2i adapter training. ( #4947 )
...
add: tests for t2i adapter training.
2023-09-08 19:45:39 +05:30
Suraj Patil
dfec61f4b3
[examples] T2IAdapter training script ( #4934 )
...
* add t2i_example script
* remove in channels logic
* remove comments
* remove use_euler arg
* add requirements
* only use canny example
* use datasets
* comments
* make log_validation consistent with other scripts
* add readme
* fix title in readme
* update check_min_version
* change a few minor things.
* add doc entry
* add: test for t2i adapter training
* remove use_auth_token
* fix: logged info.
* remove tests for now.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-08 10:03:02 +05:30
Suraj Patil
0ec7a02b6a
[StableDiffusionXLAdapterPipeline] allow negative micro conds ( #4941 )
...
* allow negative micro conds in t2i pipeline
* Empty-Commit
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-09-08 07:59:42 +05:30
Suraj Patil
626284f8d1
[StableDiffusionXLAdapterPipeline] add adapter_conditioning_factor ( #4937 )
...
add adapter_conditioning_factor
2023-09-07 19:05:28 +02:00
Sayak Paul
9800cc5ece
[InstructPix2Pix] Fix pipeline implementation and add docs ( #4844 )
...
* initial evident fixes.
* instructpix2pix fixes.
* add: entry to doc.
* address PR feedback.
* make fix-copies
2023-09-07 15:34:19 +05:30
Kashif Rasul
541bb6ee63
Würstchen model ( #3849 )
...
* initial
* initial
* added initial convert script for paella vqmodel
* initial wuerstchen pipeline
* add LayerNorm2d
* added modules
* fix typo
* use model_v2
* embed clip caption amd negative_caption
* fixed name of var
* initial modules in one place
* WuerstchenPriorPipeline
* inital shape
* initial denoising prior loop
* fix output
* add WuerstchenPriorPipeline to __init__.py
* use the noise ratio in the Prior
* try to save pipeline
* save_pretrained working
* Few additions
* add _execution_device
* shape is int
* fix batch size
* fix shape of ratio
* fix shape of ratio
* fix output dataclass
* tests folder
* fix formatting
* fix float16 + started with generator
* Update pipeline_wuerstchen.py
* removed vqgan code
* add WuerstchenGeneratorPipeline
* fix WuerstchenGeneratorPipeline
* fix docstrings
* fix imports
* convert generator pipeline
* fix convert
* Work on Generator Pipeline. WIP
* Pipeline works with our diffuzz code
* apply scale factor
* removed vqgan.py
* use cosine schedule
* redo the denoising loop
* Update src/diffusers/models/resnet.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* use torch.lerp
* use warp-diffusion org
* clip_sample=False,
* some refactoring
* use model_v3_stage_c
* c_cond size
* use clip-bigG
* allow stage b clip to be None
* add dummy
* würstchen scheduler
* minor changes
* set clip=None in the pipeline
* fix attention mask
* add attention_masks to text_encoder
* make fix-copies
* add back clip
* add text_encoder
* gen_text_encoder and tokenizer
* fix import
* updated pipeline test
* undo changes to pipeline test
* nip
* fix typo
* fix output name
* set guidance_scale=0 and remove diffuze
* fix doc strings
* make style
* nip
* removed unused
* initial docs
* rename
* toc
* cleanup
* remvoe test script
* fix-copies
* fix multi images
* remove dup
* remove unused modules
* undo changes for debugging
* no new line
* remove dup conversion script
* fix doc string
* cleanup
* pass default args
* dup permute
* fix some tests
* fix prepare_latents
* move Prior class to modules
* offload only the text encoder and vqgan
* fix resolution calculation for prior
* nip
* removed testing script
* fix shape
* fix argument to set_timesteps
* do not change .gitignore
* fix resolution calculations + readme
* resolution calculation fix + readme
* small fixes
* Add combined pipeline
* rename generator -> decoder
* Update .gitignore
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* removed efficient_net
* create combined WuerstchenPipeline
* make arguments consistent with VQ model
* fix var names
* no need to return text_encoder_hidden_states
* add latent_dim_scale to config
* split model into its own file
* add WuerschenPipeline to docs
* remove unused latent_size
* register latent_dim_scale
* update script
* update docstring
* use Attention preprocessor
* concat with normed input
* fix-copies
* add docs
* fix test
* fix style
* add to cpu_offloaded_model
* updated type
* remove 1-line func
* updated type
* initial decoder test
* formatting
* formatting
* fix autodoc link
* num_inference_steps is int
* remove comments
* fix example in docs
* Update src/diffusers/pipelines/wuerstchen/diffnext.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* rename layernorm to WuerstchenLayerNorm
* rename DiffNext to WuerstchenDiffNeXt
* added comment about MixingResidualBlock
* move paella vq-vae to pipelines' folder
* initial decoder test
* increased test_float16_inference expected diff
* self_attn is always true
* more passing decoder tests
* batch image_embeds
* fix failing tests
* set the correct dtype
* relax inference test
* update prior
* added combined pipeline test
* faster test
* faster test
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix issues from review
* update wuerstchen.md + change generator name
* resolve issues
* fix copied from usage and add back batch_size
* fix API
* fix arguments
* fix combined test
* Added timesteps argument + fixes
* Update tests/pipelines/test_pipelines_common.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update tests/pipelines/wuerstchen/test_wuerstchen_prior.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
* up
* Fix more
* failing tests
* up
* up
* correct naming
* correct docs
* correct docs
* fix test params
* correct docs
* fix classifier free guidance
* fix classifier free guidance
* fix more
* fix all
* make tests faster
---------
Co-authored-by: Dominic Rampas <d6582533@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Dominic Rampas <61938694+dome272@users.noreply.github.com >
2023-09-06 16:15:51 +02:00
dg845
b76274cb53
[docs] Fix typo in Inpainting force unmasked area unchanged example ( #4910 )
...
Fix typo by replacing init_image_arr and repainted_image_arr with init_image and repainted_image, respectively.
2023-09-06 10:49:01 +02:00
Patrick von Platen
dc3e0ca59b
[Textual inversion] Relax loading textual inversion ( #4903 )
...
* [Textual inversion] Relax loading textual inversion
* up
2023-09-06 10:39:44 +02:00
Sayak Paul
6c314ad0ce
[Docs] add doc entry to explain lora fusion and use of different scales. ( #4893 )
...
* add doc entry to explain lora fusion and use of different scales.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-09-06 07:38:13 +05:30
Steven Liu
946bb53c56
[docs] Add stronger warning for SDXL height/width ( #4867 )
...
* add size warning
* feedback
2023-09-05 10:50:42 -07:00
YiYi Xu
ea311e6989
remove latent input for kandinsky prior_emb2emb pipeline ( #4887 )
...
* remove latent input
* fix test
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-04 22:19:49 -10:00
YiYi Xu
4c5718a09c
fix a bug in StableDiffusionUpscalePipeline.run_safety_checker ( #4886 )
...
fix
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-04 22:18:59 -10:00
Patrick von Platen
2340ed629e
[Test] Reduce CPU memory ( #4897 )
...
* [Test] Reduce CPU memory
* [Test] Reduce CPU memory
2023-09-05 13:18:35 +05:30
Bagheera
cfdfcf2018
Add --vae_precision option to the SDXL pix2pix script so that we have… ( #4881 )
...
* Add --vae_precision option to the SDXL pix2pix script so that we have the option of avoiding float32 overhead
* style
---------
Co-authored-by: bghira <bghira@users.github.com >
2023-09-05 09:04:06 +02:00
Sayak Paul
e4b8e7928b
[Core] better support offloading when side loading is enabled. ( #4855 )
...
* better support offloading when side loading is enabled.
* load_textual_inversion
* better messaging for textual inversion.
* fixes
* address PR feedback.
* sdxl support.
* improve messaging
* recursive removal when cpu sequential offloading is enabled.
* add: lora tests
* recruse.
* add: offload tests for textual inversion.
2023-09-05 06:55:13 +05:30
dg845
55e17907f9
Add dropout parameter to UNet2DModel/UNet2DConditionModel ( #4882 )
...
* Add dropout param to get_down_block/get_up_block and UNet2DModel/UNet2DConditionModel.
* Add dropout param to Versatile Diffusion modeling, which has a copy of UNet2DConditionModel and its own get_down_block/get_up_block functions.
2023-09-05 00:02:21 +02:00
Sayak Paul
c81a88b239
[Core] LoRA improvements pt. 3 ( #4842 )
...
* throw warning when more than one lora is attempted to be fused.
* introduce support of lora scale during fusion.
* change test name
* changes
* change to _lora_scale
* lora_scale to call whenever applicable.
* debugging
* lora_scale additional.
* cross_attention_kwargs
* lora_scale -> scale.
* lora_scale fix
* lora_scale in patched projection.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* styling.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* remove unneeded prints.
* remove unneeded prints.
* assign cross_attention_kwargs.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* clean up.
* refactor scale retrieval logic a bit.
* fix nonetypw
* fix: tests
* add more tests
* more fixes.
* figure out a way to pass lora_scale.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* unify the retrieval logic of lora_scale.
* move adjust_lora_scale_text_encoder to lora.py.
* introduce dynamic adjustment lora scale support to sd
* fix up copies
* Empty-Commit
* add: test to check fusion equivalence on different scales.
* handle lora fusion warning.
* make lora smaller
* make lora smaller
* make lora smaller
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-04 23:52:31 +02:00
YiYi Xu
2c1677eefe
allow passing components to connected pipelines when use the combined pipeline ( #4883 )
...
* fix
* add test
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-04 06:21:36 -10:00
dg845
c73e609aae
Fix get_dummy_inputs for Stable Diffusion Inpaint Tests ( #4845 )
...
* Change StableDiffusionInpaintPipelineFastTests.get_dummy_inputs to produce a random image and a white mask_image.
* Add dummy expected slices for the test_stable_diffusion_inpaint tests.
* Remove print statement
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-04 12:04:59 +02:00
Erwann Millon
2fa4b3ffb0
check for unet_lora_layers in sdxl pipeline's save_lora_weights method ( #4821 )
...
run make fix-copies and make style
2023-09-04 09:59:59 +02:00
Isamu Isozaki
3201903d94
Retrieval Augmented Diffusion Models ( #3297 )
...
* Resetting rdm pr
* Fixed styles
* Fixed style
* Moved to rdm folder+fixed slight errors
* Removed config diff
* Started adding tests
* Adding retrieved images
* Fixed faiss import
* Fixed import errors
* Fixing tests
* Added require_faiss
* Updated dependency table
* Attempt solving consistency test
* Fixed truncation and vocab size issue
* Passed common tests
* Finished up cpu testing on pipeline
* Passed all tests locally
* Removed some slow tests
* Removed diffs from test_pipeline_common
* Remove logs
* Removed diffs from test_pipelines_common
* Fixed style
* Fully fixed styles on diffs
* Fixed name
* Proper rename
* Fixed dummies
* Fixed issue with dummyonnx
* Fixed black style
* Fixed dummies
* Changed ordering
* Fixed logging
* Fixing
* Fixing
* quality
* Debugging regex
* Fix dummies with guess
* Fixed typo
* Attempt fix dummies
* black
* ruff
* fixed ordering
* Logging
* Attempt fix
* Attempt fix dummy
* Attempt fixing styles
* Fixed faiss dependency
* Removed unnecessary deprecations
* Finished up main changes
* Added doc
* Passed tests
* Fixed tests
* Remove invisible watermark
* Fixed ruff errors
* Added prompt embed to tests
* Added tests and made retriever an optional component
* Fixed styles
* Made faiss a dependency of pipeline
* Logging
* Fixed dummies
* Make pipeline test work
* Fixed style
* Moved to research projects
* Remove diff
* Fixed style error
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-04 09:42:04 +02:00
Patrick von Platen
705c592ea9
[Tests] Add combined pipeline tests ( #4869 )
...
* [Tests] Add combined pipeline tests
* Update tests/pipelines/kandinsky_v22/test_kandinsky.py
2023-09-02 21:36:20 +02:00
Harutatsu Akiyama
c52acaaf17
[ControlNet SDXL Inpainting] Support inpainting of ControlNet SDXL ( #4694 )
...
* [ControlNet SDXL Inpainting] Support inpainting of ControlNet SDXL
Co-authored-by: Jiabin Bai 1355864570@qq.com
---------
Co-authored-by: Harutatsu Akiyama <kf.zy.qin@gmail.com >
2023-09-02 08:04:22 -10:00
Steven Liu
2c45a53aef
[docs] Shap-E guide ( #4700 )
...
* first draft
* fixes
* more fixes
* fix toctree
2023-09-01 19:52:41 -07:00
Steven Liu
22ea35cf23
[docs] DiffEdit guide ( #4722 )
...
* first draft
* minor edits
2023-09-01 14:18:41 -07:00
YiYi Xu
5c404f20f4
[WIP] masked_latent_inputs for inpainting pipeline ( #4819 )
...
* add
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-01 06:55:31 -10:00
YiYi Xu
d8b6f5d09e
support AutoPipeline.from_pipe between a pipeline and its ControlNet pipeline counterpart ( #4861 )
...
add
2023-09-01 06:53:03 -10:00
YiYi Xu
30a5acc39f
fix a bug in sdxl-controlnet-img2img when using MultiControlNetModel ( #4862 )
...
fix
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-09-01 06:51:59 -10:00
Seongsu Park
0c775544dd
[Docs] Korean translation update ( #4684 )
...
* Docs kr update 3
controlnet, reproducibility 업로드
generator 그대로 사용
seamless multi-GPU 그대로 사용
create_dataset 번역 1차
stable_diffusion_jax
new translation
Add coreml, tome
kr docs minor fix
translate training/instructpix2pix
fix training/instructpix2pix.mdx
using-diffusers/weighting_prompts 번역 1차
add SDXL docs
Translate using-diffuers/loading_overview.md
translate using-diffusers/textual_inversion_inference.md
Conditional image generation (#37 )
* stable_diffusion_jax
* index_update
* index_update
* condition_image_generation
---------
Co-authored-by: Seongsu Park <tjdtnsu@gmail.com >
jihwan/stable_diffusion.mdx
custom_diffusion 작업 완료
quicktour 작업 완료
distributed inference & control brightness (#40 )
* distributed_inference.mdx
* control_brightness
---------
Co-authored-by: idra79haza <idra79haza@github.com >
Co-authored-by: Seongsu Park <tjdtnsu@gmail.com >
using_safetensors (#41 )
* distributed_inference.mdx
* control_brightness
* using_safetensors.mdx
---------
Co-authored-by: idra79haza <idra79haza@github.com >
Co-authored-by: Seongsu Park <tjdtnsu@gmail.com >
delete safetensor short
* Repace mdx to md
* toctree update
* Add controlling_generation
* toctree fix
* colab link, minor fix
* docs name typo fix
* frontmatter fix
* translation fix
2023-09-01 09:23:45 -07:00
Pedro Cuenca
60d259add1
Fix link from API to using-diffusers ( #4856 )
...
* Fix link from API to using-diffusers
* Fix link
2023-09-01 15:05:01 +02:00
Dhruv Nair
189e9f01b3
Test Cleanup Precision issues ( #4812 )
...
* proposal for flaky tests
* more precision fixes
* move more tests to use cosine distance
* more test fixes
* clean up
* use default attn
* clean up
* update expected value
* make style
* make style
* Apply suggestions from code review
* Update src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
* make style
* fix failing tests
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-09-01 17:58:37 +05:30
Nguyễn Công Tú Anh
38466c369f
Add GLIGEN Text Image implementation ( #4777 )
...
* Add GLIGEN Text Image implementation
* add style transfer from image
* fix check_repository_consistency
* add convert script GLIGEN model to Diffusers
* rename attention type
* fix style code
* remove PositionNetTextImage
* Revert "fix check_repository_consistency"
This reverts commit 15f098c96e
2023-09-01 15:48:01 +05:30
dg845
5f740d0f55
[docs] Add inpainting example for forcing the unmasked area to remain unchanged to the docs ( #4536 )
...
* Initial code to add force_unmasked_unchanged argument to StableDiffusionInpaintPipeline.__call__.
* Try to improve StableDiffusionInpaintPipelineFastTests.get_dummy_inputs.
* Use original mask to preserve unmasked pixels in pixel space rather than latent space.
* make style
* start working on note in docs to force unmasked area to be unchanged
* Add example of forcing the unmasked area to remain unchanged.
* Revert "make style"
This reverts commit fa7759293a092bd0e9e9ff41cf43c5989979752awlbberman@gmail.com >
2023-08-31 21:29:16 -07:00
YiYi Xu
75f81c25d1
fix sdxl-inpaint fast test ( #4859 )
...
fix inpaint test
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-08-31 15:42:58 -10:00
Patrick von Platen
bbf733ab70
[SDXL Inpaint] Correct strength default ( #4858 )
2023-08-31 20:34:33 +02:00
Steven Liu
aedd78767c
[docs] ControlNet guide ( #4640 )
...
* first draft
* finish first draft
* feedback and remove sections from API pages
* clean docstrings
* add full code example
2023-08-31 10:02:02 -04:00
Patrick von Platen
7caa3682e4
Remove warn with deprecate ( #4850 )
...
* Remove warn with deprecate
* Fix typo with 1.0,0
2023-08-31 15:08:41 +02:00
Ella Charlaix
0edb4cac78
Fix image processor inputs width ( #4853 )
...
fix width for np array inputs
2023-08-31 14:50:55 +02:00
Yukun Huang
85b3f08c26
Fix potential type mismatch errors in SDXL pipelines ( #4796 )
...
* Fix potential type conversion errors in SDXL pipelines
* make sure vae stays in fp16
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-31 09:22:18 +02:00
Sayak Paul
19f3161d94
[Docs] improve the LoRA doc. ( #4838 )
...
* improve the LoRA doc.
* include fuse_lora and unfuse_lora
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-31 00:13:15 +05:30
Steven Liu
a1fdfca36f
[docs] SDXL ( #4428 )
...
* first draft
* reorg toctree
* note about minsdxl
* feedback
* fix
* micro-conditionings
* add tip
* fix section levels
* d'oh fix pipeline names
* feedback
* remove old section
2023-08-30 11:34:55 -04:00
Patrick von Platen
d1e20be664
make style
2023-08-30 14:13:14 +02:00
Anatoly Belikov
af3854d6ad
sketch inpaint from a1111 for non-inpaint models ( #4824 )
...
* Create masked_stable_diffusion_img2img.py
* add MaskedIm2ImPipeline to readme
* Update README.md
2023-08-30 09:51:28 +02:00
Patrick von Platen
9f1936d2fc
Fix Unfuse Lora ( #4833 )
...
* Fix Unfuse Lora
* add tests
* Fix more
* Fix more
* Fix all
* make style
* make style
2023-08-30 09:32:25 +05:30
Eugene Antropov
fbca2e0a7a
Add loading ckpt from file for SDXL controlNet ( #4683 )
...
* Add load ckpt from file for ControlNet SDXL
* Reformat code
* Resort imports
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-30 09:00:53 +05:30
