AI/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-01-29 07:22:12 +03:00
Code Activity
2,155 Commits 742 Branches 94 Tags
0c8fd458942fabbd409a62c2bcbaaf8410f3307e
Commit Graph

395 Commits

Author SHA1 Message Date
Will Berman
53c0c6d372 accelerate min version for ProjectConfiguration import (#3042) 2023-05-05 07:22:12 -07:00
Will Berman
2bc0a15818 config fixes (#3060) 2023-05-05 07:22:12 -07:00
Patrick von Platen
1336668f0e make style 2023-05-05 07:22:12 -07:00
J N Hearns
1ee04a5e60 Fix imports for composable_stable_diffusion pipeline (#3002) 
* Update composable_stable_diffusion.py

Fix imports

* Formatting

* Formatting

* Formatting
 2023-05-05 07:22:12 -07:00
Patrick von Platen
473d4d3961 Fix config prints and save, load of pipelines (#2849) 
* [Config] Fix config prints and save, load

* Only use potential nn.Modules for dtype and device

* Correct vae image processor

* make sure in_channels is not accessed directly

* make sure in channels is only accessed via config

* Make sure schedulers only access config attributes

* Make sure to access config in SAG

* Fix vae processor and make style

* add tests

* uP

* make style

* Fix more naming issues

* Final fix with vae config

* change more
 2023-05-05 07:22:12 -07:00
Rogério Júnior
2751633e44 Small typo correction in comments (#3012) 2023-05-05 07:22:12 -07:00
luanjintai
3de609dd66 find another one accelerate parameter error 2023-05-05 07:22:12 -07:00
luanjintai
03866af5f9 fix wrong parameter name for accelerate 2023-05-05 07:22:12 -07:00
YiYi Xu
7ee2817ae2 add Min-SNR loss to Controlnet flax train script (#3016) 
* add wandb team and min-snr loss

* make style

* apply feedbacks
 2023-05-05 07:22:12 -07:00
Patrick von Platen
5326243241 Update one_step_unet.py 
Fix dummy community pipeline
 2023-05-05 07:22:12 -07:00
YiYi Xu
2f6b212c78 minor fix in controlnet flax example (#2986) 
* fix the error when push_to_hub but not log validation

* contronet_from_pt & controlnet_revision

* add intermediate checkpointing to the guide
 2023-05-05 07:22:12 -07:00
Sayak Paul
a7da9c1a8f [Examples] Add support for Min-SNR weighting strategy for better convergence (#2899) 
* improve stable unclip doc.

* feat: support for applying min-snr weighting for faster convergence.

* add: support for validation logging with wandb

* make  not a required arg.

* fix: arg name.

* fix: cli args.

* fix: tracker config.

* fix: loss calculation.

* fix: validation logging.

* fix: unwrap call.

* fix: validation logging.

* fix: internval.

* fix: checkpointing push to hub.

* fix: c8a2856c6d\#commitcomment-106913193

* fix: norm group test for UNet3D.

* address PR comments.

* remove unneeded code.

* add: entry in the readme and docs.

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>

---------

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2023-05-05 07:22:12 -07:00
Kadir Nar
912b8391b0 The variable name has been updated. (#2970) 2023-05-05 07:22:12 -07:00
YiYi Xu
fe03d5bce4 update flax controlnet training script (#2951) 
* load_from_disk + checkpointing_steps

* apply feedback
 2023-05-05 07:22:11 -07:00
YiYi Xu
a6f2d7224e allow use custom local dataset for controlnet training scripts (#2928) 
use custom local datset

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-05-05 07:22:11 -07:00
Lucain
1d3758f0a4 Use upload_folder in training scripts (#2934) 
use upload folder in training scripts

Co-authored-by: testbot <lucainp@hf.co>
 2023-05-05 07:22:11 -07:00
Patrick von Platen
72278553a3 make style 2023-05-05 07:22:11 -07:00
Ernie Chu
aee06e7cbb Ensure validation image RGB not RGBA (#2945) 
* ensure validation image RGB not RGBA

* ensure validation image RGB not RGBA

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-05-05 07:22:11 -07:00
mengfei25
7b6caca9eb Modify example with intel optimization (#2896) 
* modify intel opts inference script

* modify readme

* modify doc

* fix some issues

* reformat

* reformat script

* format issue

* format issue
 2023-03-31 13:07:20 +01:00
Michael Gartsbein
1d033a95f6 img2img.multiple.controlnets.pipeline (#2833) 
* img2img.multiple.controlnets.pipeline

* remove comments

---------

Co-authored-by: mishka <gartsocial@gmail.com>
 2023-03-30 18:00:12 +01:00
YiYi Xu
b3d5cc4a36 add flax requirement (#2894) 
Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2023-03-30 17:10:26 +01:00
Yaman Ahlawat
3be489182e feat: allow offset_noise in dreambooth training example (#2826) 2023-03-29 16:01:02 +05:30
Sayak Paul
d82b032319 [Examples] Add streaming support to the ControlNet training example in JAX (#2859) 
* improve stable unclip doc.

* feat: add streaming support to controlnet flax training script.

* fix: CLI arg.

* fix: torch dataloader shuffle setting.

* fix: dataset length.

* fix: wandb config.

* fix: steps_per_epoch in the training loop.

* add: entry about streaming in the readme

* get column names from iterable dataset + fix final logging

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2023-03-29 06:42:08 +05:30
Li-Huai (Allan) Lin
159a0bff34 Remove duplicate sentence in docstrings (#2834) 
* Remove duplicate sentence

* format
 2023-03-28 16:27:51 +01:00
Aki Sakurai
0f14335af3 StableDiffusionLongPromptWeightingPipeline: Do not hardcode pad token (#2832) 2023-03-28 16:00:56 +01:00
YiYi Xu
d4f846fa74 [WIP]Flax training script for controlnet (#2818) 
* add train_controlnet_flax

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-27 19:13:35 -10:00
Pedro Cuenca
1d7b4b60b7 Ruff: apply same rules as in transformers (#2827) 
* Apply same ruff settings as in transformers

See https://github.com/huggingface/transformers/blob/main/pyproject.toml
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>

* Apply new style rules

* Style

Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>

* style

* remove list, ruff wouldn't auto fix.

---------

Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
 2023-03-27 16:18:57 +02:00
Sayak Paul
abb22b4eeb Update examples README.md to include the latest examples (#2839) 2023-03-27 19:34:58 +05:30
Haofan Wang
4a98d6e097 Update train_text_to_image_lora.py (#2795) 2023-03-24 11:45:35 +05:30
Haofan Wang
dc5b4e2342 Update train_text_to_image_lora.py (#2767) 
* Update train_text_to_image_lora.py

* Update train_text_to_image_lora.py

* Update train_text_to_image_lora.py

* Update train_text_to_image_lora.py

* format
 2023-03-23 14:28:47 +01:00
Nipun Jindal
055c90f589 [2737]: Add DPMSolverMultistepScheduler to CLIP guided community pipeline (#2779) 
[2737]: Add DPMSolverMultistepScheduler to CLIP guided community pipelines

Co-authored-by: njindal <njindal@adobe.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-23 14:20:24 +01:00
Naoki Ainoya
14e3a28c12 Rename 'CLIPFeatureExtractor' class to 'CLIPImageProcessor' (#2732) 
The 'CLIPFeatureExtractor' class name has been renamed to 'CLIPImageProcessor' in order to comply with future deprecation. This commit includes the necessary changes to the affected files.
 2023-03-23 13:49:22 +01:00
Mishig
8e35ef0142 [doc wip] literalinclude (#2718) 2023-03-23 13:42:54 +01:00
Sayak Paul
9dc84448ac [Examples] InstructPix2Pix instruct training script (#2478) 
* add: initial implementation of the pix2pix instruct training script.

* shorten cli arg.

* fix: main process check.

* fix: dataset column names.

* simplify tokenization.

* proper placement of null conditions.

* apply styling.

* remove debugging message for conditioning do.

* complete license.

* add: requirements.tzt

* wandb column name order.

* fix: augmentation.

* change: dataset_id.

* fix: convert_to_np() call.

* fix: reshaping.

* fix: final ema copy.

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* address PR comments.

* add: readme details.

* config fix.

* downgrade version.

* reduce image width in the readme.

* note on hyperparameters during generation.

* add: output images.

* update readme.

* minor edits to readme.

* debugging statement.

* explicitly placement of the pipeline.

* bump minimum diffusers version.

* fix: device attribute error.

* weight dtype.

* debugging.

* add dtype inform.

* add seoarate te and vae.

* add: explicit casting/

* remove casting.

* up.

* up 2.

* up 3.

* autocast.

* disable mixed-precision in the final inference.

* debugging information.

* autocasting.

* add: instructpix2pix training section to the docs.

* Empty-Commit

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-23 10:15:01 +05:30
Haofan Wang
e0d8c9ef83 Support for Offset Noise in examples (#2753) 
* add noise offset

* make style
 2023-03-23 09:36:17 +05:30
Patrick von Platen
ca1a22296d [MS Text To Video] Add first text to video (#2738) 
* [MS Text To Video} Add first text to video

* upload

* make first model example

* match unet3d params

* make sure weights are correcctly converted

* improve

* forward pass works, but diff result

* make forward work

* fix more

* finish

* refactor video output class.

* feat: add support for a video export utility.

* fix: opencv availability check.

* run make fix-copies.

* add: docs for the model components.

* add: standalone pipeline doc.

* edit docstring of the pipeline.

* add: right path to TransformerTempModel

* add: first set of tests.

* complete fast tests for text to video.

* fix bug

* up

* three fast tests failing.

* add: note on slow tests

* make work with all schedulers

* apply styling.

* add slow tests

* change file name

* update

* more correction

* more fixes

* finish

* up

* Apply suggestions from code review

* up

* finish

* make copies

* fix pipeline tests

* fix more tests

* Apply suggestions from code review

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

* apply suggestions

* up

* revert

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
 2023-03-22 18:39:33 +01:00
Hyowon Ha
58bcf46a8f Add guidance start/end parameters to StableDiffusionControlNetImg2ImgPipeline (#2731) 
* Add guidance start/end parameters to community controlnet img2img pipeline

* Fix formats
 2023-03-21 14:38:43 +01:00
Nipun Jindal
0042efd015 [1929]: Add CLIP guidance for Img2Img stable diffusion pipeline (#2723) 
* [Img2Img]: Copyover img2img pipeline

* [Img2Img]: img2img pipeline

* [Img2Img]: img2img pipeline

* [Img2Img]: img2img pipeline

---------

Co-authored-by: njindal <njindal@adobe.com>
 2023-03-21 13:53:00 +01:00
Andy
116f70cbf8 Enabling gradient checkpointing for VAE (#2536) 
* updated black format

* update black format

* make style format

* updated line endings

* update code formatting

* Update examples/research_projects/onnxruntime/text_to_image/train_text_to_image.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/models/vae.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/models/vae.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* added vae gradient checkpointing test

* make style

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Will Berman <wlbberman@gmail.com>
 2023-03-17 14:59:38 -07:00
Will Berman
a4b2c2f150 train_unconditional save restore unet parameters (#2706) 2023-03-16 16:15:56 -07:00
Patrick von Platen
e828232780 Rename attention (#2691) 
* rename file

* rename attention

* fix more

* rename more

* up

* more deprecation imports

* fixes
 2023-03-16 00:35:54 +01:00
Henrik Forstén
79eb3d07d0 Controlnet training (#2545) 
* Controlnet training code initial commit

Works with circle dataset: https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md

* Script for adding a controlnet to existing model

* Fix control image transform

Control image should be in 0..1 range.

* Add license header and remove more unused configs

* controlnet training readme

* Allow nonlocal model in add_controlnet.py

* Formatting

* Remove unused code

* Code quality

* Initialize controlnet in training script

* Formatting

* Address review comments

* doc style

* explicit constructor args and submodule names

* hub dataset

NOTE -  not tested

* empty prompts

* add conditioning image

* rename

* remove instance data dir

* image_transforms -> -1,1 . conditioning_image_transformers -> 0, 1

* nits

* remove local rank config

I think this isn't necessary in any of our training scripts

* validation images

* proportion_empty_prompts typo

* weight copying to controlnet bug

* call log validation fix

* fix

* gitignore wandb

* fix progress bar and resume from checkpoint iteration

* initial step fix

* log multiple images

* fix

* fixes

* tracker project name configurable

* misc

* add controlnet requirements.txt

* update docs

* image labels

* small fixes

* log validation using existing models for pipeline

* fix for deepspeed saving

* memory usage docs

* Update examples/controlnet/train_controlnet.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/train_controlnet.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* remove extra is main process check

* link to dataset in intro paragraph

* remove unnecessary paragraph

* note on deepspeed

* Update examples/controlnet/README.md

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* assert -> value error

* weights and biases note

* move images out of git

* remove .gitignore

---------

Co-authored-by: William Berman <WLBberman@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-14 20:16:30 -07:00
aengusng8
268ebcb015 Add ddim noise comparative analysis pipeline (#2665) 
* add DDIM Noise Comparative Analysis pipeline

* update README

* add comments

* run BLACK format
 2023-03-14 18:09:55 +01:00
Will Berman
ebd44957fc image generation main process checks (#2631) 2023-03-14 01:28:03 -07:00
Ruizhe Wang
ce08cb72fb [Dreambooth] Editable number of class images (#2251) 
* [Dreambooth] Editable number of class images

* 'class_num=None' bug fix

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-10 14:15:16 +01:00
Patrick von Platen
cbbad0af69 correct example 2023-03-08 20:14:19 +01:00
Haofan Wang
00132de359 Support LoRA for text encoder (#2588) 
* add lora

* Update examples/research_projects/lora/README.md

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-08 20:14:01 +01:00
Michael Gartsbein
46bef6e31d community stablediffusion controlnet img2img pipeline (#2584) 
Co-authored-by: mishka <gartsocial@gmail.com>
 2023-03-07 13:31:56 -08:00
zxypro
f0b661b8fb [Docs]Fix invalid link to Pokemons dataset (#2583) 2023-03-07 14:26:09 +01:00
Isamu Isozaki
8552fd7efa Added multitoken training for textual inversion. Issue 369 (#661) 
* Added multitoken training for textual inversion

* Updated assertion

* Removed duplicate save code

* Fixed undefined bug

* Fixed save

* Added multitoken clip model +util helper

* Removed code splitting

* Removed class

* Fixed errors

* Fixed errors

* Added loading functionality

* Loading via dict instead

* Fixed bug of invalid index being loaded

* Fixed adding placeholder token only adding 1 token

* Fixed bug when initializing tokens

* Fixed bug when initializing tokens

* Removed flawed logic

* Fixed vector shuffle

* Fixed tokenizer's inconsistent __call__ method

* Fixed tokenizer's inconsistent __call__ method

* Handling list input

* Added exception for adding invalid tokens to token map

* Removed unnecessary files and started working on progressive tokens

* Set at minimum load one token

* Changed to global step

* Added method to load automatic1111 tokens

* Fixed bug in load

* Quality+style fixes

* Update quality/style fixes

* Cast embeddings to fp16 when loading

* Fixed quality

* Started moving things over

* Clearing diffs

* Clearing diffs

* Moved everything

* Requested changes
 2023-03-07 12:09:36 +01:00