* Additions:
- support for different lr for text encoder
- support for Prodigy optimizer
- support for min snr gamma
- support for custom captions and dataset loading from the hub
* adjusted --caption_column behaviour (to -not- use the second column of the dataset by default if --caption_column is not provided)
* fixed --output_dir / --model_dir_name confusion
* added --repeats, --adam_weight_decay_text_encoder
+ some fixes
* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* - import compute_snr from diffusers/training_utils.py
- cluster adamw together
- when using 'prodigy', if --train_text_encoder == True and --text_encoder_lr != --learning rate, changes the lr of the text encoders optimization params to be --learning_rate (otherwise errors)
* shape fixes when custom captions are used
* formatting and a little cleanup
* code styling
* --repeats default value fixed, changed to 1
* bug fix - removed redundant lines of embedding concatenation when using prior_preservation (that duplicated class_prompt embeddings)
* changed dataset loading logic according to the following usecases (to avoid unnecessary dependency on datasets)-
1. user provides --dataset_name
2. user provides local dir --instance_data_dir that contains a metadata .jsonl file
3. user provides local dir --instance_data_dir that contains only images
in cases [1,2] we import datasets and use load_dataset method, in case [3] we process the data same as in the original script setting
* styling fix
* arg name fix
* adjusted the --repeats logic
* -removed redundant arg and 'if' when loading local folder with prompts
-updated readme template
-some default val fixes
-custom caption tests
* image path fix for readme
* code style
* bug fix
* --caption_column arg
* readme fix
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Linoy Tsaban <linoy@huggingface.co>
* Update textual_inversion.py
fixed safe_path bug in textual inversion training
* Update test_examples.py
update test_textual_inversion for updating saved file's name
* Update textual_inversion.py
fixed some formatting issues
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* make safetensors default
* set default save method as safetensors
* update tests
* update to support saving safetensors
* update test to account for safetensors default
* update example tests to use safetensors
* update example to support safetensors
* update unet tests for safetensors
* fix failing loader tests
* fix qc issues
* fix pipeline tests
* fix example test
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
* add: train to text image with sdxl script.
Co-authored-by: CaptnSeraph <s3raph1m@gmail.com>
* fix: partial func.
* fix: default value of output_dir.
* make style
* set num inference steps to 25.
* remove mentions of LoRA.
* up min version
* add: ema cli arg
* run device placement while running step.
* precompute vae encodings too.
* fix
* debug
* should work now.
* debug
* debug
* goes alright?
* style
* debugging
* debugging
* debugging
* debugging
* fix
* reinit scheduler if prediction_type was passed.
* akways cast vae in float32
* better handling of snr.
Co-authored-by: bghira <bghira@users.github.com>
* the vae should be also passed
* add: docs.
* add: sdlx t2i tests
* save the pipeline
* autocast.
* fix: save_model_card
* fix: save_model_card.
---------
Co-authored-by: CaptnSeraph <s3raph1m@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: bghira <bghira@users.github.com>
* add train_text_to_image_lora_sdxl.py
* add train_text_to_image_lora_sdxl.py
* add test and minor fix
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* fix unwrap_model rule
* add invisible-watermark in requirements
* del invisible-watermark
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update examples/text_to_image/train_text_to_image_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* del comment & update readme
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* fix: #4206
* add: sdxl controlnet training smoketest.
* remove unnecessary token inits.
* add: licensing to model card.
* include SDXL licensing in the model card and make public visibility default
* debugging
* debugging
* disable local file download.
* fix: training test.
* fix: ckpt prefix.
* update IF stage I pipelines
add fixed variance schedulers and lora loading
* added kv lora attn processor
* allow loading into alternative lora attn processor
* make vae optional
* throw away predicted variance
* allow loading into added kv lora layer
* allow load T5
* allow pre compute text embeddings
* set new variance type in schedulers
* fix copies
* refactor all prompt embedding code
class prompts are now included in pre-encoding code
max tokenizer length is now configurable
embedding attention mask is now configurable
* fix for when variance type is not defined on scheduler
* do not pre compute validation prompt if not present
* add example test for if lora dreambooth
* add check for train text encoder and pre compute text embeddings
* 👽 qol improvements for LoRA.
* better function name?
* fix: LoRA weight loading with the new format.
* address Patrick's comments.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* change wording around encouraging the use of load_lora_weights().
* fix: function name.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* add: LoRA text encoder support for DreamBooth example.
* fix initialization.
* fix: modification call.
* add: entry in the readme.
* use dog dataset from hub.
* fix: params to clip.
* add entry to the LoRA doc.
* add: tests for lora.
* remove unnecessary list comprehension./
* Update README.md
Additionally add FLAX so the model card can be slimmer and point to this page
* Find and replace all
* v-1-5 -> v1-5
* revert test changes
* Update README.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update docs/source/quicktour.mdx
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update README.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update docs/source/quicktour.mdx
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update README.md
Co-authored-by: Suraj Patil <surajp815@gmail.com>
* Revert certain references to v1-5
* Docs changes
* Apply suggestions from code review
Co-authored-by: apolinario <joaopaulo.passos+multimodal@gmail.com>
Co-authored-by: anton-l <anton@huggingface.co>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Suraj Patil <surajp815@gmail.com>
