* Additions:
- support for different lr for text encoder
- support for Prodigy optimizer
- support for min snr gamma
- support for custom captions and dataset loading from the hub
* adjusted --caption_column behaviour (to -not- use the second column of the dataset by default if --caption_column is not provided)
* fixed --output_dir / --model_dir_name confusion
* added --repeats, --adam_weight_decay_text_encoder
+ some fixes
* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* - import compute_snr from diffusers/training_utils.py
- cluster adamw together
- when using 'prodigy', if --train_text_encoder == True and --text_encoder_lr != --learning rate, changes the lr of the text encoders optimization params to be --learning_rate (otherwise errors)
* shape fixes when custom captions are used
* formatting and a little cleanup
* code styling
* --repeats default value fixed, changed to 1
* bug fix - removed redundant lines of embedding concatenation when using prior_preservation (that duplicated class_prompt embeddings)
* changed dataset loading logic according to the following usecases (to avoid unnecessary dependency on datasets)-
1. user provides --dataset_name
2. user provides local dir --instance_data_dir that contains a metadata .jsonl file
3. user provides local dir --instance_data_dir that contains only images
in cases [1,2] we import datasets and use load_dataset method, in case [3] we process the data same as in the original script setting
* styling fix
* arg name fix
* adjusted the --repeats logic
* -removed redundant arg and 'if' when loading local folder with prompts
-updated readme template
-some default val fixes
-custom caption tests
* image path fix for readme
* code style
* bug fix
* --caption_column arg
* readme fix
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Linoy Tsaban <linoy@huggingface.co>
* fix an issue that ipex occupy too much memory, it will not impact performance
* make style
---------
Co-authored-by: root <jun.chen@intel.com>
Co-authored-by: Meng Guoqing <guoqing.meng@intel.com>
* Fix the pipeline name in the examples for LMD+ pipeline
* Add LMD+ colab link
* Apply code formatting
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* fix error reported 'find_unused_parameters' running in mutiple GPUs or NPUs
* fix code check of importing module by its alphabetic order
---------
Co-authored-by: jiaqiw <wangjiaqi50@huawei.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
* Add a new community pipeline
examples/community/latent_consistency_img2img.py
which can be called like this
import torch
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained(
"SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_txt2img", custom_revision="main")
# To save GPU memory, torch.float16 can be used, but it may compromise image quality.
pipe.to(torch_device="cuda", torch_dtype=torch.float32)
img2img=LatentConsistencyModelPipeline_img2img(
vae=pipe.vae,
text_encoder=pipe.text_encoder,
tokenizer=pipe.tokenizer,
unet=pipe.unet,
#scheduler=pipe.scheduler,
scheduler=None,
safety_checker=None,
feature_extractor=pipe.feature_extractor,
requires_safety_checker=False,
)
img = Image.open("thisismyimage.png")
result = img2img(prompt,img,strength,num_inference_steps=4)
* Apply suggestions from code review
Fix name formatting for scheduler
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* update readme (and run formatter on latent_consistency_img2img.py)
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
