* add: LoRA text encoder support for DreamBooth example.
* fix initialization.
* fix: modification call.
* add: entry in the readme.
* use dog dataset from hub.
* fix: params to clip.
* add entry to the LoRA doc.
* add: tests for lora.
* remove unnecessary list comprehension./
* Update Pix2PixZero Auto-correlation Loss
* Add fast inversion tests
* Clarify purpose and mark as deprecated
Fix inversion prompt broadcasting
* Register modules set to `None` in config for `test_save_load_optional_components`
* Update new tests to coordinate with #2953
* Added distillation for quantization example on textual inversion.
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
* refined readme and code style.
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
* Update text2images.py
* refined code of model load and added compatibility check.
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
* fixed code style.
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
* fix C403 [*] Unnecessary `list` comprehension (rewrite as a `set` comprehension)
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
---------
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
controlnet training center crop input images to multiple of 8
The pipeline code resizes inputs to multiples of 8.
Not doing this resizing in the training script is causing
the encoded image to have different height/width dimensions
than the encoded conditioning image (which uses a separate
encoder that's part of the controlnet model).
We resize and center crop the inputs to make sure they're the
same size (as well as all other images in the batch). We also
check that the initial resolution is a multiple of 8.
* Modified altdiffusion pipline to support altdiffusion-m18
* Modified altdiffusion pipline to support altdiffusion-m18
* Modified altdiffusion pipline to support altdiffusion-m18
* Modified altdiffusion pipline to support altdiffusion-m18
* Modified altdiffusion pipline to support altdiffusion-m18
* Modified altdiffusion pipline to support altdiffusion-m18
* Modified altdiffusion pipline to support altdiffusion-m18
---------
Co-authored-by: root <fulong_ye@163.com>
* Add SD/txt2img Community Pipeline to diffusers along with TensorRT utils
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
* update installation command
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
* update tensorrt installation
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
* changes
1. Update setting of cache directory
2. Address comments: merge utils and pipeline code.
3. Address comments: Add section in README
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
* apply make style
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
---------
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* add mixin class for pipeline from original sd ckpt
* Improve
* make style
* merge main into
* Improve more
* fix more
* up
* Apply suggestions from code review
* finish docs
* rename
* make style
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* [ckpt loader] Allow loading the Inpaint and Img2Img pipelines, while loading a ckpt model
* Address review comment from PR
* PyLint formatting
* Some more pylint fixes, unrelated to our change
* Another pylint fix
* Styling fix
Adding act fn config to the unet timestep class embedding and conv
activation.
The custom activation defaults to silu which is the default
activation function for both the conv act and the timestep class
embeddings so default behavior is not changed.
The only unet which use the custom activation is the stable diffusion
latent upscaler https://huggingface.co/stabilityai/sd-x2-latent-upscaler/blob/main/unet/config.json
(I ran a script against the hub to confirm).
The latent upscaler does not use the conv activation nor the timestep
class embeddings so we don't change its behavior.
add custom timesteps test
add custom timesteps descending order check
docs
timesteps -> custom_timesteps
can only pass one of num_inference_steps and timesteps
* add guess mode (WIP)
* fix uncond/cond order
* support guidance_scale=1.0 and batch != 1
* remove magic coeff
* add docstring
* add intergration test
* add document to controlnet.mdx
* made the comments a bit more explanatory
* fix table
* WIP controlnet training
- bugfix --streaming
- bugfix running report_to!='wandb'
- adds memory profile before validation
* Adds final logging statement.
* Sets train epochs to 11.
Looking at a longer ~16ep run, we see only good validation images
after ~11ep:
https://wandb.ai/andsteing/controlnet_fill50k/runs/3j2hx6n8
* Removes --logging_dir (it's not used).
* Adds --profile flags.
* Updates --output_dir=runs/fill-circle-{timestamp}.
* Compute mean of `train_metrics`.
Previously `train_metrics[-1]` was logged, resulting in very bumpy train
metrics.
* Improves logging a bit.
- adds l2_grads gradient norm logging
- adds steps_per_sec
- sets walltime as x coordinate of train/step
- logs controlnet_params config
* Adds --ccache (doesn't really help though).
* minor fix in controlnet flax example (#2986)
* fix the error when push_to_hub but not log validation
* contronet_from_pt & controlnet_revision
* add intermediate checkpointing to the guide
* Bugfix --profile_steps
* Sets `RACKER_PROJECT_NAME='controlnet_fill50k'`.
* Logs fractional epoch.
* Adds relative `walltime` metric.
* Adds `StepTraceAnnotation` and uses `global_step` insetad of `step`.
* Applied `black`.
* Streamlines commands in README a bit.
* Removes `--ccache`.
This makes only a very small difference (~1 min) with this model size, so removing
the option introduced in cdb3cc.
* Re-ran `black`.
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Converts spaces to tab.
* Removes repeated args.
* Skips first step (compilation) in profiling
* Updates README with profiling instructions.
* Unifies tabs/spaces in README.
* Re-ran style & quality.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* fix: norm group test for UNet3D.
* chore: speed up the panorama tests (fast).
* set default value of _test_inference_batch_single_identical.
* fix: batch_sizes default value.
* fix progress bar issue in pipeline_text_to_video_zero.py. Copy scheduler after first backward
* fix tensor loading in test_text_to_video_zero.py
* make style && make quality
* add support for prompt embeds to SD ONNX pipeline
* fix up the pipeline copies
* add prompt embeds param to other ONNX pipelines
* fix up prompt embeds param for SD upscaling ONNX pipeline
* add missing type annotations to ONNX pipes
* inital commit for lora test cases
* help a bit with lora for 3d
* fixed lora tests
* replaced redundant code
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
