AI/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-01-29 07:22:12 +03:00
Code Activity
2,908 Commits 742 Branches 94 Tags
9858053bfeb61ba8e7bbecc9d30defab825cbcaf
Commit Graph

184 Commits

Author SHA1 Message Date
Patrick von Platen
29f15673ed Release: v0.21.0 2023-09-13 15:58:24 +02:00
Will Berman
d73e6ad050 guard save model hooks to only execute on main process (#4929) 2023-09-08 10:30:06 -07:00
Will Berman
7e5587a5ac instance_prompt->class_prompt (#4784) 2023-08-25 20:06:55 +02:00
Sayak Paul
4447547eda [Examples] fix sdxl dreambooth lora checkpointing. (#4749) 
* fix sdxl dreambooth lora checkpointing.

* style
 2023-08-25 09:50:02 +05:30
Yinzhen Wang
d185b5ed5f change validation scheduler for train_dreambooth.py when training IF (#4333) 
* dreambooth training

* train_dreambooth validation scheduler

* set a particular scheduler via a string

* modify readme after setting a particular scheduler via a string

* modify readme after setting a particular scheduler

* use importlib to set a particular scheduler

* import with correct sort
 2023-08-23 22:18:17 +02:00
Sayak Paul
d0c30cfd37 make post-release (#4650) 2023-08-17 14:16:25 +05:30
Patrick von Platen
029fb41695 [Safetensors] Make safetensors the default way of saving weights (#4235) 
* make safetensors default

* set default save method as safetensors

* update tests

* update to support saving safetensors

* update test to account for safetensors default

* update example tests to use safetensors

* update example to support safetensors

* update unet tests for safetensors

* fix failing loader tests

* fix qc issues

* fix pipeline tests

* fix example test

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
 2023-08-17 10:54:28 +05:30
Sayak Paul
e731ae0ec8 Update README_sdxl.md to include the free-tier Colab Notebook (#4540) 
Update README_sdxl.md
 2023-08-09 14:32:14 +05:30
Patrick von Platen
4367b8a300 move pipeline only when running validation (#4515) 2023-08-07 17:20:18 +02:00
Patrick von Platen
e1b5b8ba13 Make sure fp16-fix is used as default (#4510) 
* Make sue fp16-fix is used as default

* fix vae

* finish

* fix
 2023-08-07 15:16:37 +02:00
Patrick von Platen
dff5ff35a9 [SDXL LoRA] fix batch size lora (#4509) 
fix batch size lora
 2023-08-07 13:27:13 +02:00
Vladislav Artemyev
2e69cf16fe Log global_step instead of epoch to tensorboard (#4493) 
Co-authored-by: mrlzla <noname@noname.com>
 2023-08-07 07:49:39 +05:30
Patrick von Platen
ea1fcc28a4 [SDXL] Allow SDXL LoRA to be run with less than 16GB of VRAM (#4470) 
* correct

* correct blocks

* finish

* finish

* finish

* Apply suggestions from code review

* fix

* up

* up

* up

* Update examples/dreambooth/README_sdxl.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Apply suggestions from code review

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2023-08-04 20:06:38 +02:00
Patrick von Platen
66de221409 Update README_sdxl.md (#4472) 2023-08-04 16:23:35 +02:00
Levi McCallum
4188f3063a Add rank argument to train_dreambooth_lora_sdxl.py (#4343) 
* Add rank argument to train_dreambooth_lora_sdxl.py

* Update train_dreambooth_lora_sdxl.py
 2023-08-03 23:27:30 +05:30
Will Berman
160474ac61 train dreambooth fix pre encode class prompt (#4395) 2023-08-01 12:00:05 -07:00
Sayak Paul
4a4cdd6b07 [Feat] Support SDXL Kohya-style LoRA (#4287) 
* sdxl lora changes.

* better name replacement.

* better replacement.

* debugging

* debugging

* debugging

* debugging

* debugging

* remove print.

* print state dict keys.

* print

* distingisuih better

* debuggable.

* fxi: tyests

* fix: arg from training script.

* access from class.

* run style

* debug

* save intermediate

* some simplifications for SDXL LoRA

* styling

* unet config is not needed in diffusers format.

* fix: dynamic SGM block mapping for SDXL kohya loras (#4322)

* Use lora compatible layers for linear proj_in/proj_out (#4323)

* improve condition for using the sgm_diffusers mapping

* informative comment.

* load compatible keys and embedding layer maaping.

* Get SDXL 1.0 example lora to load

* simplify

* specif ranks and hidden sizes.

* better handling of k rank and hidden

* debug

* debug

* debug

* debug

* debug

* fix: alpha keys

* add check for handling LoRAAttnAddedKVProcessor

* sanity comment

* modifications for text encoder SDXL

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* denugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* up

* up

* up

* up

* up

* up

* unneeded comments.

* unneeded comments.

* kwargs for the other attention processors.

* kwargs for the other attention processors.

* debugging

* debugging

* debugging

* debugging

* improve

* debugging

* debugging

* more print

* Fix alphas

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* clean up

* clean up.

* debugging

* fix: text

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Batuhan Taskaya <batuhan@python.org>
 2023-07-28 19:49:49 +02:00
Patrick von Platen
b7b6d6138d [SDXL] Make watermarker optional under certain circumstances to improve usability of SDXL 1.0 (#4346) 
* improve sdxl

* more fixes

* improve sdxl

* improve sdxl

* improve sdxl

* finish
 2023-07-28 19:29:22 +02:00
Sayak Paul
961173064d Honor the SDXL 1.0 licensing from the training scripts. (#4319) 
* honor the original license.

* train_instruct_pix2pix_xl -> train_instruct_pix2pix_sdxl
 2023-07-28 01:28:36 +05:30
Xinyang Li
01b6ec21fa fix validation option for dreambooth training example (#4317) 2023-07-27 09:58:52 -07:00
Patrick von Platen
20e92586c1 0.20.0dev0 (#4299) 
* 0.20.0dev0

* make style
 2023-07-26 23:06:18 +02:00
Patrick von Platen
6a6dfe1cbd Rename (#4294) 
* up

* Apply suggestions from code review

* Apply suggestions from code review

* up
 2023-07-26 20:41:21 +02:00
Sayak Paul
161449d51a [SDXL DreamBooth LoRA] multiple fixes (#4262) 
* add automatic licensing.

* debugging

* debugging

* more debugging

* more debugging.

* run make fix-copies.

* change to default tracker.
 2023-07-25 21:10:01 +02:00
Ragnar Rova
4e2a021829 Model path for sdxl wrong in dreambooth README (#4261) 2023-07-25 18:06:50 +05:30
Sayak Paul
5ef6b8fa53 Update README_sdxl.md to change the note on default hyperparameters (#4258) 2023-07-25 16:57:48 +05:30
Will Berman
3dd339379d do not pass list to accelerator.init_trackers (#4248) 2023-07-24 21:10:37 -07:00
Sayak Paul
365e8461ac [SDXL DreamBooth LoRA] add support for text encoder fine-tuning (#4097) 
* Allow low precision sd xl

* finish

* finish

* feat: initial draft for supporting text encoder lora finetuning for SDXL DreamBooth

* fix: variable assignments.

* add: autocast block.

* add debugging

* vae dtype hell

* fix: vae dtype hell.

* fix: vae dtype hell 3.

* clean up

* lora text encoder loader.

* fix: unwrapping models.

* add: tests.

* docs.

* handle unexpected keys.

* fix vae dtype in the final inference.

* fix scope problem.

* fix: save_model_card args.

* initialize: prefix to None.

* fix: dtype issues.

* apply gixes.

* debgging.

* debugging

* debugging

* debugging

* debugging

* debugging

* add: fast tests.

* pre-tokenize.

* address: will's comments.

* fix: loader and tests.

* fix: dataloader.

* simplify dataloader.

* length.

* simplification.

* make style && make quality

* simplify state_dict munging

* fix: tests.

* fix: state_dict packing.

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-07-25 05:35:48 +05:30
Apoorva Kulkarni
2e53936c97 docs: Typo in dreambooth example README.md (#4203) 
fix: Typo in dreambooth example README.md
 2023-07-21 15:16:38 -07:00
takuoko
6427aa995e [Enhance] Add rank in dreambooth (#4112) 
add rank in dreambooth
 2023-07-18 11:30:06 +05:30
Patrick von Platen
71c918b848 [Invisible watermark] Correct version (#4087) 2023-07-14 09:30:43 +05:30
Gabriel Birnbaum
f3802eb805 fix requirement in SDXL (#4082) 2023-07-14 02:58:20 +02:00
Ruoxi
ece55227ff Multiply lr scheduler steps by num_processes. (#3983) 
* Multiply lr scheduler steps by `num_processes`.

* Stop multiplying steps by gradient accumulation.
 2023-07-13 17:50:25 +05:30
Patrick von Platen
b9feed8795 move to 0.19.0dev (#4048) 2023-07-11 22:49:12 +02:00
Sayak Paul
3d74dc2abd [Examples] Add a training script for SDXL DreamBooth LoRA (#4016) 
* add dreambooth lora script for SDXL incorporating latest changes.

* remove use_auth_token=True.

* add: documentation

* remove unneeded cli.

* increase the number of training steps in the readme.

* add LoraLoaderMixin to the subclassing mix.

* add sdxl lora dreambooth test.

* add: inference code sample.

* add: refiner output.

* add LoraLoaderMixin to the mix of classes of StableDiffusionXLImg2ImgPipeline.

* change default resolution of DreamBoothDataset.

* better sdxl report path.

* Apply suggestions from code review

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

---------

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
 2023-07-11 07:38:41 +05:30
Patrick von Platen
4a3e574807 make style 2023-07-09 16:02:59 +00:00
Will Berman
c2a28c346c Refactor LoRA (#3778) 
* refactor to support patching LoRA into T5

instantiate the lora linear layer on the same device as the regular linear layer

get lora rank from state dict

tests

fmt

can create lora layer in float32 even when rest of model is float16

fix loading model hook

remove load_lora_weights_ and T5 dispatching

remove Unet#attn_processors_state_dict

docstrings

* text encoder monkeypatch class method

* fix test

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-07-09 18:02:46 +02:00
Batuhan Taskaya
04ddad484e Add 'rank' parameter to Dreambooth LoRA training script (#3945) 2023-07-07 17:26:10 +05:30
Will Berman
d49e2dd54c manual check for checkpoints_total_limit instead of using accelerate (#3681) 
* manual check for checkpoints_total_limit instead of using accelerate

* remove controlnet_conditioning_embedding_out_channels
 2023-06-15 15:38:54 -07:00
Patrick von Platen
908e5e9cc6 Fix some bad comment in training scripts (#3798) 
* relax tolerance slightly

* correct incorrect naming
 2023-06-15 15:07:51 +02:00
Patrick von Platen
c42f6ee43e Post 0.17.0 release (#3721) 
* Post release

* Post release
 2023-06-08 18:08:49 +02:00
Zachary Mueller
79fa94ea8b Apply deprecations from Accelerate (#3714) 
Apply deprecations
 2023-06-08 16:44:22 +02:00
Sayak Paul
8669e8313d [LoRA] feat: add lora attention processor for pt 2.0. (#3594) 
* feat: add lora attention processor for pt 2.0.

* explicit context manager for SDPA.

* switch to flash attention

* make shapes compatible to work optimally with SDPA.

* fix: circular import problem.

* explicitly specify the flash attention kernel in sdpa

* fall back to efficient attention context manager.

* remove explicit dispatch.

* fix: removed processor.

* fix: remove optional from type annotation.

* feat: make changes regarding LoRAAttnProcessor2_0.

* remove confusing warning.

* formatting.

* relax tolerance for PT 2.0

* fix: loading message.

* remove unnecessary logging.

* add: entry to the docs.

* add: network_alpha argument.

* relax tolerance.
 2023-06-06 14:56:05 +05:30
Patrick von Platen
262d539a8a Correct multi gpu dreambooth (#3673) 
Correct multi gpu
 2023-06-05 11:03:11 +01:00
Will Berman
0fc2fb71c1 dreambooth upscaling fix added latents (#3659) 2023-06-05 10:32:16 +01:00
Will Berman
5911a3aa47 dreambooth if docs - stage II, more info (#3628) 
* dreambooth if docs - stage II, more info

* Update docs/source/en/training/dreambooth.mdx

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update docs/source/en/training/dreambooth.mdx

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update docs/source/en/training/dreambooth.mdx

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* download instructions for downsized images

* update source README to match docs

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2023-06-02 10:37:13 -07:00
Takuma Mori
8e552bb4fe Support Kohya-ss style LoRA file format (in a limited capacity) (#3437) 
* add _convert_kohya_lora_to_diffusers

* make style

* add scaffold

* match result: unet attention only

* fix monkey-patch for text_encoder

* with CLIPAttention

While the terrible images are no longer produced,
the results do not match those from the hook ver.
This may be due to not setting the network_alpha value.

* add to support network_alpha

* generate diff image

* fix monkey-patch for text_encoder

* add test_text_encoder_lora_monkey_patch()

* verify that it's okay to release the attn_procs

* fix closure version

* add comment

* Revert "fix monkey-patch for text_encoder"

This reverts commit bb9c61e6fa.

* Fix to reuse utility functions

* make LoRAAttnProcessor targets to self_attn

* fix LoRAAttnProcessor target

* make style

* fix split key

* Update src/diffusers/loaders.py

* remove TEXT_ENCODER_TARGET_MODULES loop

* add print memory usage

* remove test_kohya_loras_scaffold.py

* add: doc on LoRA civitai

* remove print statement and refactor in the doc.

* fix state_dict test for kohya-ss style lora

* Apply suggestions from code review

Co-authored-by: Takuma Mori <takuma104@gmail.com>

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
 2023-06-02 17:40:24 +05:30
Will Berman
4f14b36329 Full Dreambooth IF stage II upscaling (#3561) 
* update dreambooth lora to work with IF stage II

* Update dreambooth script for IF stage II upscaler
 2023-05-31 09:39:31 -07:00
Will Berman
f751b8844e update dreambooth lora to work with IF stage II (#3560) 2023-05-31 09:39:03 -07:00
Leon Lin
1d1f648c6b fix dreambooth attention mask (#3541) 2023-05-26 10:58:50 -07:00
Sayak Paul
8e69708b0d [Examples/DreamBooth] refactor save_model_card utility in dreambooth examples (#3543) 
refactor save_model_card utility in dreambooth examples.
 2023-05-24 16:16:28 +05:30