Yinzhen Wang
d185b5ed5f
change validation scheduler for train_dreambooth.py when training IF ( #4333 )
...
* dreambooth training
* train_dreambooth validation scheduler
* set a particular scheduler via a string
* modify readme after setting a particular scheduler via a string
* modify readme after setting a particular scheduler
* use importlib to set a particular scheduler
* import with correct sort
2023-08-23 22:18:17 +02:00
Sayak Paul
4909b1e3ac
[Examples] fix checkpointing and casting bugs in train_text_to_image_lora_sdxl.py ( #4632 )
...
* fix: casting issues.
* fix checkpointing.
* tests
* fix: bugs
2023-08-23 10:58:54 +05:30
Patrick von Platen
4f6399bedd
rename test file to run, so that examples tests do not fail ( #4715 )
...
* rename test file to run, so that examples tests do not fail
* [Tests] Rename community tests
2023-08-22 19:54:46 +02:00
zideliu
f22aad6e3a
Add reference_attn & reference_adain support for sdxl ( #4502 )
...
* ADD SDXL reference & reference adain
* Update README.md
* Update README.md
* format stable_diffusion_xl_reference.py
* format file
* Format file
* format file
* fix format
* fix format with ruff
* fix format
* Update examples/community/README.md
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* Update examples/community/README.md
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* Update README.md
* Update README.md & fix typo
* Update README.md
* fix format
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-22 20:22:01 +05:30
Andrew Zhu
d7c4ae619d
Add SDXL long weighted prompt pipeline (replace pr:4629) ( #4661 )
...
* Add SDXL long weighted prompt pipeline
* Add SDXL long weighted prompt pipeline usage sample in the readme document
* Add SDXL long weighted prompt pipeline usage sample in the readme document, add result image
2023-08-18 11:30:10 +05:30
Sayak Paul
d0c30cfd37
make post-release ( #4650 )
2023-08-17 14:16:25 +05:30
Patrick von Platen
029fb41695
[Safetensors] Make safetensors the default way of saving weights ( #4235 )
...
* make safetensors default
* set default save method as safetensors
* update tests
* update to support saving safetensors
* update test to account for safetensors default
* update example tests to use safetensors
* update example to support safetensors
* update unet tests for safetensors
* fix failing loader tests
* fix qc issues
* fix pipeline tests
* fix example test
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-17 10:54:28 +05:30
Steven Liu
4ff7264d9b
[docs] PushToHubMixin ( #4622 )
...
* push to hub docs
* fix typo
* feedback
* make style
2023-08-16 13:20:59 -06:00
Suraj Patil
7b93c2a882
[research_projects] SDXL controlnet script ( #4633 )
...
add controlent script,
2023-08-16 18:27:08 +05:30
Sayak Paul
5175d3d7a5
add: train to text image with sdxl script. ( #4505 )
...
* add: train to text image with sdxl script.
Co-authored-by: CaptnSeraph <s3raph1m@gmail.com >
* fix: partial func.
* fix: default value of output_dir.
* make style
* set num inference steps to 25.
* remove mentions of LoRA.
* up min version
* add: ema cli arg
* run device placement while running step.
* precompute vae encodings too.
* fix
* debug
* should work now.
* debug
* debug
* goes alright?
* style
* debugging
* debugging
* debugging
* debugging
* fix
* reinit scheduler if prediction_type was passed.
* akways cast vae in float32
* better handling of snr.
Co-authored-by: bghira <bghira@users.github.com >
* the vae should be also passed
* add: docs.
* add: sdlx t2i tests
* save the pipeline
* autocast.
* fix: save_model_card
* fix: save_model_card.
---------
Co-authored-by: CaptnSeraph <s3raph1m@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: bghira <bghira@users.github.com >
2023-08-16 09:02:49 +05:30
Wang Qiang
078df46bc9
An invalid clerical error in sdxl finetune ( #4608 )
2023-08-15 10:41:51 +05:30
Sayak Paul
d93ca26893
[Examples] Update InstructPix2Pix README_sdxl.md to fix mentions ( #4574 )
...
* Update README_sdxl.md to fix mentions
* add --push_to_hub
* add --push_to_hub
* fix: mention
2023-08-14 17:48:13 +05:30
Sayak Paul
d67eba0f31
[Utility] adds an image grid utility ( #4576 )
...
* add: utility for image grid.
* add: return type.
* change necessary places.
* add to utility page.
2023-08-12 10:34:51 +05:30
Sayak Paul
d5983a6779
[Examples] fix: network_alpha -> network_alphas ( #4572 )
...
network_alpha
2023-08-11 14:18:49 +05:30
Abhipsha Das
c8d86e9f0a
Remove code snippets containing is_safetensors_available() ( #4521 )
...
* [WIP] Remove code snippets containing `is_safetensors_available()`
* Modifying `import_utils.py`
* update pipeline tests for safetensor default
* fix test related to cached requests
* address import nits
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-08-11 11:05:22 +05:30
dotieuthien
b28cd3fba0
Convert Stable Diffusion ControlNet to TensorRT ( #4465 )
...
* convert tensorrt controlnet
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix code quality
* Fix number controlnet condition
* Add convert SD XL to onnx
* Add convert SD XL to tensorrt
* Add convert SD XL to tensorrt
* Add examples in comments
* Add examples in comments
* Add test onnx controlnet
* Add tensorrt test
* Remove copied
* Move file test to examples/community
* Remove script
* Remove script
* Remove text
---------
Co-authored-by: dotieuthien <thien.do@mservice.com.vn >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-11 08:12:26 +05:30
Sayak Paul
e731ae0ec8
Update README_sdxl.md to include the free-tier Colab Notebook ( #4540 )
...
Update README_sdxl.md
2023-08-09 14:32:14 +05:30
Rastislav Švarba
6c5b5b260e
Fix push_to_hub in train_text_to_image_lora_sdxl.py example ( #4535 )
...
fix: push_to_hub in train text2image lora sdxl
2023-08-09 11:48:24 +05:30
Dhruv Nair
c91272d631
fix indexing issue in sd reference pipeline ( #4531 )
2023-08-08 15:14:19 +02:00
Patrick von Platen
4367b8a300
move pipeline only when running validation ( #4515 )
2023-08-07 17:20:18 +02:00
Patrick von Platen
e1b5b8ba13
Make sure fp16-fix is used as default ( #4510 )
...
* Make sue fp16-fix is used as default
* fix vae
* finish
* fix
2023-08-07 15:16:37 +02:00
Patrick von Platen
dff5ff35a9
[SDXL LoRA] fix batch size lora ( #4509 )
...
fix batch size lora
2023-08-07 13:27:13 +02:00
Vladislav Artemyev
2e69cf16fe
Log global_step instead of epoch to tensorboard ( #4493 )
...
Co-authored-by: mrlzla <noname@noname.com >
2023-08-07 07:49:39 +05:30
takuoko
9c29bc2df8
[Examples] Support train_text_to_image_lora_sdxl.py ( #4365 )
...
* add train_text_to_image_lora_sdxl.py
* add train_text_to_image_lora_sdxl.py
* add test and minor fix
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* fix unwrap_model rule
* add invisible-watermark in requirements
* del invisible-watermark
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update examples/text_to_image/train_text_to_image_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* del comment & update readme
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-06 13:47:20 +05:30
AisingioroHao
70d098540d
Add a data_dir parameter to the load_dataset method. ( #4482 )
...
Co-authored-by: AisingioroHao0 <1286098622@qq.com >
2023-08-06 08:45:48 +05:30
Patrick von Platen
ea1fcc28a4
[SDXL] Allow SDXL LoRA to be run with less than 16GB of VRAM ( #4470 )
...
* correct
* correct blocks
* finish
* finish
* finish
* Apply suggestions from code review
* fix
* up
* up
* up
* Update examples/dreambooth/README_sdxl.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Apply suggestions from code review
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-04 20:06:38 +02:00
Patrick von Platen
66de221409
Update README_sdxl.md ( #4472 )
2023-08-04 16:23:35 +02:00
asfiyab-nvidia
c14c141b86
TensorRT Inpaint pipeline: minor fixes ( #4457 )
...
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com >
2023-08-04 12:28:28 +02:00
manosplitsis
79ef9e528c
Fixed multi-token textual inversion training ( #4452 )
...
* added placeholder token concatenation during training
* Update examples/textual_inversion/textual_inversion.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-04 12:21:31 +02:00
JinK
e391b789ac
Support different strength for Stable Diffusion TensorRT Inpainting pipeline ( #4216 )
...
* Support different strength
* run make style
2023-08-03 21:32:44 +02:00
Levi McCallum
4188f3063a
Add rank argument to train_dreambooth_lora_sdxl.py ( #4343 )
...
* Add rank argument to train_dreambooth_lora_sdxl.py
* Update train_dreambooth_lora_sdxl.py
2023-08-03 23:27:30 +05:30
Xin Kong
615c04db15
[Pipelines] Add community pipeline for Zero123 ( #4295 )
...
* add zero123 pipeline to community
* add community doc
* reformat
* update zero123 pipeline, including cc_projection within diffusers; add convert ckpt scripts; support diffusers weights
2023-08-02 19:36:49 +02:00
Will Berman
160474ac61
train dreambooth fix pre encode class prompt ( #4395 )
2023-08-01 12:00:05 -07:00
Harutatsu Akiyama
aae27262f4
[SDXL-IP2P] Add gif for demonstrating training processes ( #4342 )
...
* [SDXL-IP2P] Add gif for demonstrating training processes
* [SDXL-IP2P] Add gif for demonstrating training processes
* [SDXL-IP2P] Change gif to URLs
* [SDXL-IP2P] Add URLs in case gif now show
---------
Co-authored-by: Harutatsu Akiyama <kf.zy.qin@gmail.com >
2023-07-30 10:07:10 +05:30
Sayak Paul
4a4cdd6b07
[Feat] Support SDXL Kohya-style LoRA ( #4287 )
...
* sdxl lora changes.
* better name replacement.
* better replacement.
* debugging
* debugging
* debugging
* debugging
* debugging
* remove print.
* print state dict keys.
* print
* distingisuih better
* debuggable.
* fxi: tyests
* fix: arg from training script.
* access from class.
* run style
* debug
* save intermediate
* some simplifications for SDXL LoRA
* styling
* unet config is not needed in diffusers format.
* fix: dynamic SGM block mapping for SDXL kohya loras (#4322 )
* Use lora compatible layers for linear proj_in/proj_out (#4323 )
* improve condition for using the sgm_diffusers mapping
* informative comment.
* load compatible keys and embedding layer maaping.
* Get SDXL 1.0 example lora to load
* simplify
* specif ranks and hidden sizes.
* better handling of k rank and hidden
* debug
* debug
* debug
* debug
* debug
* fix: alpha keys
* add check for handling LoRAAttnAddedKVProcessor
* sanity comment
* modifications for text encoder SDXL
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* denugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* up
* up
* up
* up
* up
* up
* unneeded comments.
* unneeded comments.
* kwargs for the other attention processors.
* kwargs for the other attention processors.
* debugging
* debugging
* debugging
* debugging
* improve
* debugging
* debugging
* more print
* Fix alphas
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* clean up
* clean up.
* debugging
* fix: text
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Batuhan Taskaya <batuhan@python.org >
2023-07-28 19:49:49 +02:00
Patrick von Platen
b7b6d6138d
[SDXL] Make watermarker optional under certain circumstances to improve usability of SDXL 1.0 ( #4346 )
...
* improve sdxl
* more fixes
* improve sdxl
* improve sdxl
* improve sdxl
* finish
2023-07-28 19:29:22 +02:00
Tanupriya Singh
c7250f2b8a
correct doc string for default value of guidance_scale ( #4339 )
2023-07-28 13:54:28 +02:00
Sayak Paul
54fab2cd5f
Update README_sdxl.md to correct the header ( #4330 )
...
Update README_sdxl.md
2023-07-28 09:22:14 +05:30
Sayak Paul
961173064d
Honor the SDXL 1.0 licensing from the training scripts. ( #4319 )
...
* honor the original license.
* train_instruct_pix2pix_xl -> train_instruct_pix2pix_sdxl
2023-07-28 01:28:36 +05:30
Xinyang Li
01b6ec21fa
fix validation option for dreambooth training example ( #4317 )
2023-07-27 09:58:52 -07:00
Patrick von Platen
20e92586c1
0.20.0dev0 ( #4299 )
...
* 0.20.0dev0
* make style
2023-07-26 23:06:18 +02:00
Patrick von Platen
6a6dfe1cbd
Rename ( #4294 )
...
* up
* Apply suggestions from code review
* Apply suggestions from code review
* up
2023-07-26 20:41:21 +02:00
Sayak Paul
161449d51a
[SDXL DreamBooth LoRA] multiple fixes ( #4262 )
...
* add automatic licensing.
* debugging
* debugging
* more debugging
* more debugging.
* run make fix-copies.
* change to default tracker.
2023-07-25 21:10:01 +02:00
Harutatsu Akiyama
428dbfecd9
[SDXL and IP2P]: instruction pix2pix XL training and pipeline ( #4079 )
...
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* [Community] Implementation of the IADB community pipeline (#3996 )
* community pipeline: implementation of iadb
* iadb.py: reformat using black
* iadb.py: linting update
* add kandinsky to readme table (#4081 )
Co-authored-by: yiyixuxu <yixu310@gmail,com>
* [From Single File] Force accelerate to be installed (#4078 )
force accelerate to be installed
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Support instruction pix2pix sdxl
* Clean up IP2P SDXL code
* Clean up IP2P SDXL code
* [IP2P and SDXL] clean up code
* [IP2P and SDXL] clean up code
* [IP2P and SDXL] clean up code
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews, add docs, tests
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Add README_SDXL
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Address code reviews
* [IP2P SDXL] Fix the copy problems
* [IP2P SDXL] Add license
* [IP2P SDXL] Add license
* [IP2P SDXL] Add license
* [IP2P SDXL] Address code reivew for selecting VAE andd others
* [IP2P SDXL] Update README_sdxl
* [IP2P SDXL] Update __init__
* [IP2P SDXL] Update dummy_torch_and_transformers_and_invisible_watermark_objects
* address patrick's comments and some additions to readmes.
---------
Co-authored-by: Harutatsu Akiyama <kf.zy.qin@gmail.com >
Co-authored-by: Thomas Chambon <36728882+tchambon@users.noreply.github.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-07-25 18:19:35 +05:30
Ragnar Rova
4e2a021829
Model path for sdxl wrong in dreambooth README ( #4261 )
2023-07-25 18:06:50 +05:30
Sayak Paul
5ef6b8fa53
Update README_sdxl.md to change the note on default hyperparameters ( #4258 )
2023-07-25 16:57:48 +05:30
Will Berman
3dd339379d
do not pass list to accelerator.init_trackers ( #4248 )
2023-07-24 21:10:37 -07:00
nupurkmr9
5652c43f83
Resolve bf16 error as mentioned in this [issue]( https://github.com/huggingface/diffusers/issues/4139#issuecomment-1639977304 ) ( #4214 )
...
* resolve bf16 error
* resolve bf16 error
* resolve bf16 error
* resolve bf16 error
* resolve bf16 error
* resolve bf16 error
* resolve bf16 error
2023-07-25 05:41:19 +05:30
Sayak Paul
365e8461ac
[SDXL DreamBooth LoRA] add support for text encoder fine-tuning ( #4097 )
...
* Allow low precision sd xl
* finish
* finish
* feat: initial draft for supporting text encoder lora finetuning for SDXL DreamBooth
* fix: variable assignments.
* add: autocast block.
* add debugging
* vae dtype hell
* fix: vae dtype hell.
* fix: vae dtype hell 3.
* clean up
* lora text encoder loader.
* fix: unwrapping models.
* add: tests.
* docs.
* handle unexpected keys.
* fix vae dtype in the final inference.
* fix scope problem.
* fix: save_model_card args.
* initialize: prefix to None.
* fix: dtype issues.
* apply gixes.
* debgging.
* debugging
* debugging
* debugging
* debugging
* debugging
* add: fast tests.
* pre-tokenize.
* address: will's comments.
* fix: loader and tests.
* fix: dataloader.
* simplify dataloader.
* length.
* simplification.
* make style && make quality
* simplify state_dict munging
* fix: tests.
* fix: state_dict packing.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-07-25 05:35:48 +05:30
Sayak Paul
fed12376c5
[ControlNet SDXL training] fixes in the training script ( #4223 )
...
* fix : #4206
* add: sdxl controlnet training smoketest.
* remove unnecessary token inits.
* add: licensing to model card.
* include SDXL licensing in the model card and make public visibility default
* debugging
* debugging
* disable local file download.
* fix: training test.
* fix: ckpt prefix.
2023-07-25 05:31:48 +05:30
