Patrick von Platen
091a058236
make style
2023-04-11 15:51:21 +00:00
J N Hearns
881a6b58c3
Fix imports for composable_stable_diffusion pipeline ( #3002 )
...
* Update composable_stable_diffusion.py
Fix imports
* Formatting
* Formatting
* Formatting
2023-04-11 16:50:25 +01:00
Patrick von Platen
8b451eb63b
Fix config prints and save, load of pipelines ( #2849 )
...
* [Config] Fix config prints and save, load
* Only use potential nn.Modules for dtype and device
* Correct vae image processor
* make sure in_channels is not accessed directly
* make sure in channels is only accessed via config
* Make sure schedulers only access config attributes
* Make sure to access config in SAG
* Fix vae processor and make style
* add tests
* uP
* make style
* Fix more naming issues
* Final fix with vae config
* change more
2023-04-11 13:35:42 +02:00
Rogério Júnior
67c3518f68
Small typo correction in comments ( #3012 )
2023-04-10 13:48:35 -07:00
luanjintai
85f1c19282
find another one accelerate parameter error
2023-04-10 12:23:17 -07:00
luanjintai
b5d0a9131d
fix wrong parameter name for accelerate
2023-04-10 12:23:17 -07:00
YiYi Xu
dcfa6e1d20
add Min-SNR loss to Controlnet flax train script ( #3016 )
...
* add wandb team and min-snr loss
* make style
* apply feedbacks
2023-04-10 07:56:54 +05:30
Patrick von Platen
1c96f82ed9
Update one_step_unet.py
...
Fix dummy community pipeline
2023-04-09 19:22:18 +01:00
YiYi Xu
2de36fae7b
minor fix in controlnet flax example ( #2986 )
...
* fix the error when push_to_hub but not log validation
* contronet_from_pt & controlnet_revision
* add intermediate checkpointing to the guide
2023-04-06 10:27:41 -10:00
Sayak Paul
24947317a6
[Examples] Add support for Min-SNR weighting strategy for better convergence ( #2899 )
...
* improve stable unclip doc.
* feat: support for applying min-snr weighting for faster convergence.
* add: support for validation logging with wandb
* make not a required arg.
* fix: arg name.
* fix: cli args.
* fix: tracker config.
* fix: loss calculation.
* fix: validation logging.
* fix: unwrap call.
* fix: validation logging.
* fix: internval.
* fix: checkpointing push to hub.
* fix: c8a2856c6dsurajp815@gmail.com >
---------
Co-authored-by: Suraj Patil <surajp815@gmail.com >
2023-04-06 19:08:40 +05:30
Kadir Nar
37b359b2bd
The variable name has been updated. ( #2970 )
2023-04-06 10:55:43 +01:00
YiYi Xu
ee20d1f8b9
update flax controlnet training script ( #2951 )
...
* load_from_disk + checkpointing_steps
* apply feedback
2023-04-04 15:49:44 -10:00
YiYi Xu
0c63c3839a
allow use custom local dataset for controlnet training scripts ( #2928 )
...
use custom local datset
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-04-04 10:37:47 -07:00
Lucain
a87e88b783
Use upload_folder in training scripts ( #2934 )
...
use upload folder in training scripts
Co-authored-by: testbot <lucainp@hf.co >
2023-04-04 16:19:12 +01:00
Patrick von Platen
a0263b2e5b
make style
2023-04-04 15:18:39 +02:00
Ernie Chu
62c01d267a
Ensure validation image RGB not RGBA ( #2945 )
...
* ensure validation image RGB not RGBA
* ensure validation image RGB not RGBA
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-04-04 14:17:59 +01:00
mengfei25
7b6caca9eb
Modify example with intel optimization ( #2896 )
...
* modify intel opts inference script
* modify readme
* modify doc
* fix some issues
* reformat
* reformat script
* format issue
* format issue
2023-03-31 13:07:20 +01:00
Michael Gartsbein
1d033a95f6
img2img.multiple.controlnets.pipeline ( #2833 )
...
* img2img.multiple.controlnets.pipeline
* remove comments
---------
Co-authored-by: mishka <gartsocial@gmail.com >
2023-03-30 18:00:12 +01:00
YiYi Xu
b3d5cc4a36
add flax requirement ( #2894 )
...
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-03-30 17:10:26 +01:00
Yaman Ahlawat
3be489182e
feat: allow offset_noise in dreambooth training example ( #2826 )
2023-03-29 16:01:02 +05:30
Sayak Paul
d82b032319
[Examples] Add streaming support to the ControlNet training example in JAX ( #2859 )
...
* improve stable unclip doc.
* feat: add streaming support to controlnet flax training script.
* fix: CLI arg.
* fix: torch dataloader shuffle setting.
* fix: dataset length.
* fix: wandb config.
* fix: steps_per_epoch in the training loop.
* add: entry about streaming in the readme
* get column names from iterable dataset + fix final logging
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2023-03-29 06:42:08 +05:30
Li-Huai (Allan) Lin
159a0bff34
Remove duplicate sentence in docstrings ( #2834 )
...
* Remove duplicate sentence
* format
2023-03-28 16:27:51 +01:00
Aki Sakurai
0f14335af3
StableDiffusionLongPromptWeightingPipeline: Do not hardcode pad token ( #2832 )
2023-03-28 16:00:56 +01:00
YiYi Xu
d4f846fa74
[WIP]Flax training script for controlnet ( #2818 )
...
* add train_controlnet_flax
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-27 19:13:35 -10:00
Pedro Cuenca
1d7b4b60b7
Ruff: apply same rules as in transformers ( #2827 )
...
* Apply same ruff settings as in transformers
See https://github.com/huggingface/transformers/blob/main/pyproject.toml
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com >
* Apply new style rules
* Style
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com >
* style
* remove list, ruff wouldn't auto fix.
---------
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com >
2023-03-27 16:18:57 +02:00
Sayak Paul
abb22b4eeb
Update examples README.md to include the latest examples ( #2839 )
2023-03-27 19:34:58 +05:30
Haofan Wang
4a98d6e097
Update train_text_to_image_lora.py ( #2795 )
2023-03-24 11:45:35 +05:30
Haofan Wang
dc5b4e2342
Update train_text_to_image_lora.py ( #2767 )
...
* Update train_text_to_image_lora.py
* Update train_text_to_image_lora.py
* Update train_text_to_image_lora.py
* Update train_text_to_image_lora.py
* format
2023-03-23 14:28:47 +01:00
Nipun Jindal
055c90f589
[2737]: Add DPMSolverMultistepScheduler to CLIP guided community pipeline ( #2779 )
...
[2737]: Add DPMSolverMultistepScheduler to CLIP guided community pipelines
Co-authored-by: njindal <njindal@adobe.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-23 14:20:24 +01:00
Naoki Ainoya
14e3a28c12
Rename 'CLIPFeatureExtractor' class to 'CLIPImageProcessor' ( #2732 )
...
The 'CLIPFeatureExtractor' class name has been renamed to 'CLIPImageProcessor' in order to comply with future deprecation. This commit includes the necessary changes to the affected files.
2023-03-23 13:49:22 +01:00
Mishig
8e35ef0142
[doc wip] literalinclude ( #2718 )
2023-03-23 13:42:54 +01:00
Sayak Paul
9dc84448ac
[Examples] InstructPix2Pix instruct training script ( #2478 )
...
* add: initial implementation of the pix2pix instruct training script.
* shorten cli arg.
* fix: main process check.
* fix: dataset column names.
* simplify tokenization.
* proper placement of null conditions.
* apply styling.
* remove debugging message for conditioning do.
* complete license.
* add: requirements.tzt
* wandb column name order.
* fix: augmentation.
* change: dataset_id.
* fix: convert_to_np() call.
* fix: reshaping.
* fix: final ema copy.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* address PR comments.
* add: readme details.
* config fix.
* downgrade version.
* reduce image width in the readme.
* note on hyperparameters during generation.
* add: output images.
* update readme.
* minor edits to readme.
* debugging statement.
* explicitly placement of the pipeline.
* bump minimum diffusers version.
* fix: device attribute error.
* weight dtype.
* debugging.
* add dtype inform.
* add seoarate te and vae.
* add: explicit casting/
* remove casting.
* up.
* up 2.
* up 3.
* autocast.
* disable mixed-precision in the final inference.
* debugging information.
* autocasting.
* add: instructpix2pix training section to the docs.
* Empty-Commit
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-23 10:15:01 +05:30
Haofan Wang
e0d8c9ef83
Support for Offset Noise in examples ( #2753 )
...
* add noise offset
* make style
2023-03-23 09:36:17 +05:30
Patrick von Platen
ca1a22296d
[MS Text To Video] Add first text to video ( #2738 )
...
* [MS Text To Video} Add first text to video
* upload
* make first model example
* match unet3d params
* make sure weights are correcctly converted
* improve
* forward pass works, but diff result
* make forward work
* fix more
* finish
* refactor video output class.
* feat: add support for a video export utility.
* fix: opencv availability check.
* run make fix-copies.
* add: docs for the model components.
* add: standalone pipeline doc.
* edit docstring of the pipeline.
* add: right path to TransformerTempModel
* add: first set of tests.
* complete fast tests for text to video.
* fix bug
* up
* three fast tests failing.
* add: note on slow tests
* make work with all schedulers
* apply styling.
* add slow tests
* change file name
* update
* more correction
* more fixes
* finish
* up
* Apply suggestions from code review
* up
* finish
* make copies
* fix pipeline tests
* fix more tests
* Apply suggestions from code review
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
* apply suggestions
* up
* revert
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Pedro Cuenca <pedro@huggingface.co >
2023-03-22 18:39:33 +01:00
Hyowon Ha
58bcf46a8f
Add guidance start/end parameters to StableDiffusionControlNetImg2ImgPipeline ( #2731 )
...
* Add guidance start/end parameters to community controlnet img2img pipeline
* Fix formats
2023-03-21 14:38:43 +01:00
Nipun Jindal
0042efd015
[1929]: Add CLIP guidance for Img2Img stable diffusion pipeline ( #2723 )
...
* [Img2Img]: Copyover img2img pipeline
* [Img2Img]: img2img pipeline
* [Img2Img]: img2img pipeline
* [Img2Img]: img2img pipeline
---------
Co-authored-by: njindal <njindal@adobe.com >
2023-03-21 13:53:00 +01:00
Andy
116f70cbf8
Enabling gradient checkpointing for VAE ( #2536 )
...
* updated black format
* update black format
* make style format
* updated line endings
* update code formatting
* Update examples/research_projects/onnxruntime/text_to_image/train_text_to_image.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/models/vae.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/models/vae.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* added vae gradient checkpointing test
* make style
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Will Berman <wlbberman@gmail.com >
2023-03-17 14:59:38 -07:00
Will Berman
a4b2c2f150
train_unconditional save restore unet parameters ( #2706 )
2023-03-16 16:15:56 -07:00
Patrick von Platen
e828232780
Rename attention ( #2691 )
...
* rename file
* rename attention
* fix more
* rename more
* up
* more deprecation imports
* fixes
2023-03-16 00:35:54 +01:00
Henrik Forstén
79eb3d07d0
Controlnet training ( #2545 )
...
* Controlnet training code initial commit
Works with circle dataset: https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md
* Script for adding a controlnet to existing model
* Fix control image transform
Control image should be in 0..1 range.
* Add license header and remove more unused configs
* controlnet training readme
* Allow nonlocal model in add_controlnet.py
* Formatting
* Remove unused code
* Code quality
* Initialize controlnet in training script
* Formatting
* Address review comments
* doc style
* explicit constructor args and submodule names
* hub dataset
NOTE - not tested
* empty prompts
* add conditioning image
* rename
* remove instance data dir
* image_transforms -> -1,1 . conditioning_image_transformers -> 0, 1
* nits
* remove local rank config
I think this isn't necessary in any of our training scripts
* validation images
* proportion_empty_prompts typo
* weight copying to controlnet bug
* call log validation fix
* fix
* gitignore wandb
* fix progress bar and resume from checkpoint iteration
* initial step fix
* log multiple images
* fix
* fixes
* tracker project name configurable
* misc
* add controlnet requirements.txt
* update docs
* image labels
* small fixes
* log validation using existing models for pipeline
* fix for deepspeed saving
* memory usage docs
* Update examples/controlnet/train_controlnet.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/train_controlnet.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/controlnet/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* remove extra is main process check
* link to dataset in intro paragraph
* remove unnecessary paragraph
* note on deepspeed
* Update examples/controlnet/README.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* assert -> value error
* weights and biases note
* move images out of git
* remove .gitignore
---------
Co-authored-by: William Berman <WLBberman@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-14 20:16:30 -07:00
aengusng8
268ebcb015
Add ddim noise comparative analysis pipeline ( #2665 )
...
* add DDIM Noise Comparative Analysis pipeline
* update README
* add comments
* run BLACK format
2023-03-14 18:09:55 +01:00
Will Berman
ebd44957fc
image generation main process checks ( #2631 )
2023-03-14 01:28:03 -07:00
Ruizhe Wang
ce08cb72fb
[Dreambooth] Editable number of class images ( #2251 )
...
* [Dreambooth] Editable number of class images
* 'class_num=None' bug fix
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-10 14:15:16 +01:00
Patrick von Platen
cbbad0af69
correct example
2023-03-08 20:14:19 +01:00
Haofan Wang
00132de359
Support LoRA for text encoder ( #2588 )
...
* add lora
* Update examples/research_projects/lora/README.md
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-03-08 20:14:01 +01:00
Michael Gartsbein
46bef6e31d
community stablediffusion controlnet img2img pipeline ( #2584 )
...
Co-authored-by: mishka <gartsocial@gmail.com >
2023-03-07 13:31:56 -08:00
zxypro
f0b661b8fb
[Docs]Fix invalid link to Pokemons dataset ( #2583 )
2023-03-07 14:26:09 +01:00
Isamu Isozaki
8552fd7efa
Added multitoken training for textual inversion. Issue 369 ( #661 )
...
* Added multitoken training for textual inversion
* Updated assertion
* Removed duplicate save code
* Fixed undefined bug
* Fixed save
* Added multitoken clip model +util helper
* Removed code splitting
* Removed class
* Fixed errors
* Fixed errors
* Added loading functionality
* Loading via dict instead
* Fixed bug of invalid index being loaded
* Fixed adding placeholder token only adding 1 token
* Fixed bug when initializing tokens
* Fixed bug when initializing tokens
* Removed flawed logic
* Fixed vector shuffle
* Fixed tokenizer's inconsistent __call__ method
* Fixed tokenizer's inconsistent __call__ method
* Handling list input
* Added exception for adding invalid tokens to token map
* Removed unnecessary files and started working on progressive tokens
* Set at minimum load one token
* Changed to global step
* Added method to load automatic1111 tokens
* Fixed bug in load
* Quality+style fixes
* Update quality/style fixes
* Cast embeddings to fp16 when loading
* Fixed quality
* Started moving things over
* Clearing diffs
* Clearing diffs
* Moved everything
* Requested changes
2023-03-07 12:09:36 +01:00
Pedro Cuenca
d3ce6f4b1e
Support revision in Flax text-to-image training ( #2567 )
...
Support revision in Flax text-to-image training.
2023-03-07 08:16:31 +01:00
Will Berman
ca7ca11bcd
community controlnet inpainting pipelines ( #2561 )
...
* community controlnet inpainting pipelines
* add community member attribution re: @pcuenca
2023-03-06 12:55:31 -08:00
