AI/diffusers
1
0
Fork 0
mirror of https://github.com/huggingface/diffusers.git synced 2026-01-27 17:22:53 +03:00
Code Activity
2,160 Commits 740 Branches 94 Tags
fa736e321d85a49cd761fccc6dd70a66b562aa1c
Commit Graph

398 Commits

Author SHA1 Message Date
Sayak Paul
5a7d35e29c Fix InstructPix2Pix training in multi-GPU mode (#2978) 
* fix: norm group test for UNet3D.

* fix: unet rejig.

* fix: unwrapping when running validation inputs.

* unwrapping the unet too.

* fix: device.

* better unwrapping.

* unwrapping before ema.

* unwrapping.
 2023-04-12 10:13:53 +01:00
Sayak Paul
e607a582cf [Examples] Fix type-casting issue in the ControlNet training script (#2994) 
* fix: norm group test for UNet3D.

* fix: type-casting issue in controlnet training.
 2023-04-12 06:35:06 +05:30
Chanchana Sornsoontorn
52c4d32d41 Fix typo and format BasicTransformerBlock attributes (#2953) 
* ⚙️chore(train_controlnet) fix typo in logger message

* ⚙️chore(models) refactor modules order; make them the same as calling order

When printing the BasicTransformerBlock to stdout, I think it's crucial that the attributes order are shown in proper order. And also previously the "3. Feed Forward" comment was not making sense. It should have been close to self.ff but it's instead next to self.norm3

* correct many tests

* remove bogus file

* make style

* correct more tests

* finish tests

* fix one more

* make style

* make unclip deterministic

* ⚙️chore(models/attention) reorganize comments in BasicTransformerBlock class

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-04-12 00:31:05 +02:00
Will Berman
67ec9cf513 accelerate min version for ProjectConfiguration import (#3042) 2023-04-11 10:12:28 -07:00
Will Berman
80bc0c0ced config fixes (#3060) 2023-04-11 17:54:50 +01:00
Patrick von Platen
091a058236 make style 2023-04-11 15:51:21 +00:00
J N Hearns
881a6b58c3 Fix imports for composable_stable_diffusion pipeline (#3002) 
* Update composable_stable_diffusion.py

Fix imports

* Formatting

* Formatting

* Formatting
 2023-04-11 16:50:25 +01:00
Patrick von Platen
8b451eb63b Fix config prints and save, load of pipelines (#2849) 
* [Config] Fix config prints and save, load

* Only use potential nn.Modules for dtype and device

* Correct vae image processor

* make sure in_channels is not accessed directly

* make sure in channels is only accessed via config

* Make sure schedulers only access config attributes

* Make sure to access config in SAG

* Fix vae processor and make style

* add tests

* uP

* make style

* Fix more naming issues

* Final fix with vae config

* change more
 2023-04-11 13:35:42 +02:00
Rogério Júnior
67c3518f68 Small typo correction in comments (#3012) 2023-04-10 13:48:35 -07:00
luanjintai
85f1c19282 find another one accelerate parameter error 2023-04-10 12:23:17 -07:00
luanjintai
b5d0a9131d fix wrong parameter name for accelerate 2023-04-10 12:23:17 -07:00
YiYi Xu
dcfa6e1d20 add Min-SNR loss to Controlnet flax train script (#3016) 
* add wandb team and min-snr loss

* make style

* apply feedbacks
 2023-04-10 07:56:54 +05:30
Patrick von Platen
1c96f82ed9 Update one_step_unet.py 
Fix dummy community pipeline
 2023-04-09 19:22:18 +01:00
YiYi Xu
2de36fae7b minor fix in controlnet flax example (#2986) 
* fix the error when push_to_hub but not log validation

* contronet_from_pt & controlnet_revision

* add intermediate checkpointing to the guide
 2023-04-06 10:27:41 -10:00
Sayak Paul
24947317a6 [Examples] Add support for Min-SNR weighting strategy for better convergence (#2899) 
* improve stable unclip doc.

* feat: support for applying min-snr weighting for faster convergence.

* add: support for validation logging with wandb

* make  not a required arg.

* fix: arg name.

* fix: cli args.

* fix: tracker config.

* fix: loss calculation.

* fix: validation logging.

* fix: unwrap call.

* fix: validation logging.

* fix: internval.

* fix: checkpointing push to hub.

* fix: c8a2856c6d\#commitcomment-106913193

* fix: norm group test for UNet3D.

* address PR comments.

* remove unneeded code.

* add: entry in the readme and docs.

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>

---------

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2023-04-06 19:08:40 +05:30
Kadir Nar
37b359b2bd The variable name has been updated. (#2970) 2023-04-06 10:55:43 +01:00
YiYi Xu
ee20d1f8b9 update flax controlnet training script (#2951) 
* load_from_disk + checkpointing_steps

* apply feedback
 2023-04-04 15:49:44 -10:00
YiYi Xu
0c63c3839a allow use custom local dataset for controlnet training scripts (#2928) 
use custom local datset

Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-04-04 10:37:47 -07:00
Lucain
a87e88b783 Use upload_folder in training scripts (#2934) 
use upload folder in training scripts

Co-authored-by: testbot <lucainp@hf.co>
 2023-04-04 16:19:12 +01:00
Patrick von Platen
a0263b2e5b make style 2023-04-04 15:18:39 +02:00
Ernie Chu
62c01d267a Ensure validation image RGB not RGBA (#2945) 
* ensure validation image RGB not RGBA

* ensure validation image RGB not RGBA

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-04-04 14:17:59 +01:00
mengfei25
7b6caca9eb Modify example with intel optimization (#2896) 
* modify intel opts inference script

* modify readme

* modify doc

* fix some issues

* reformat

* reformat script

* format issue

* format issue
 2023-03-31 13:07:20 +01:00
Michael Gartsbein
1d033a95f6 img2img.multiple.controlnets.pipeline (#2833) 
* img2img.multiple.controlnets.pipeline

* remove comments

---------

Co-authored-by: mishka <gartsocial@gmail.com>
 2023-03-30 18:00:12 +01:00
YiYi Xu
b3d5cc4a36 add flax requirement (#2894) 
Co-authored-by: yiyixuxu <yixu310@gmail,com>
 2023-03-30 17:10:26 +01:00
Yaman Ahlawat
3be489182e feat: allow offset_noise in dreambooth training example (#2826) 2023-03-29 16:01:02 +05:30
Sayak Paul
d82b032319 [Examples] Add streaming support to the ControlNet training example in JAX (#2859) 
* improve stable unclip doc.

* feat: add streaming support to controlnet flax training script.

* fix: CLI arg.

* fix: torch dataloader shuffle setting.

* fix: dataset length.

* fix: wandb config.

* fix: steps_per_epoch in the training loop.

* add: entry about streaming in the readme

* get column names from iterable dataset + fix final logging

---------

Co-authored-by: yiyixuxu <yixu310@gmail.com>
 2023-03-29 06:42:08 +05:30
Li-Huai (Allan) Lin
159a0bff34 Remove duplicate sentence in docstrings (#2834) 
* Remove duplicate sentence

* format
 2023-03-28 16:27:51 +01:00
Aki Sakurai
0f14335af3 StableDiffusionLongPromptWeightingPipeline: Do not hardcode pad token (#2832) 2023-03-28 16:00:56 +01:00
YiYi Xu
d4f846fa74 [WIP]Flax training script for controlnet (#2818) 
* add train_controlnet_flax

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-27 19:13:35 -10:00
Pedro Cuenca
1d7b4b60b7 Ruff: apply same rules as in transformers (#2827) 
* Apply same ruff settings as in transformers

See https://github.com/huggingface/transformers/blob/main/pyproject.toml
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>

* Apply new style rules

* Style

Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>

* style

* remove list, ruff wouldn't auto fix.

---------

Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
 2023-03-27 16:18:57 +02:00
Sayak Paul
abb22b4eeb Update examples README.md to include the latest examples (#2839) 2023-03-27 19:34:58 +05:30
Haofan Wang
4a98d6e097 Update train_text_to_image_lora.py (#2795) 2023-03-24 11:45:35 +05:30
Haofan Wang
dc5b4e2342 Update train_text_to_image_lora.py (#2767) 
* Update train_text_to_image_lora.py

* Update train_text_to_image_lora.py

* Update train_text_to_image_lora.py

* Update train_text_to_image_lora.py

* format
 2023-03-23 14:28:47 +01:00
Nipun Jindal
055c90f589 [2737]: Add DPMSolverMultistepScheduler to CLIP guided community pipeline (#2779) 
[2737]: Add DPMSolverMultistepScheduler to CLIP guided community pipelines

Co-authored-by: njindal <njindal@adobe.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-23 14:20:24 +01:00
Naoki Ainoya
14e3a28c12 Rename 'CLIPFeatureExtractor' class to 'CLIPImageProcessor' (#2732) 
The 'CLIPFeatureExtractor' class name has been renamed to 'CLIPImageProcessor' in order to comply with future deprecation. This commit includes the necessary changes to the affected files.
 2023-03-23 13:49:22 +01:00
Mishig
8e35ef0142 [doc wip] literalinclude (#2718) 2023-03-23 13:42:54 +01:00
Sayak Paul
9dc84448ac [Examples] InstructPix2Pix instruct training script (#2478) 
* add: initial implementation of the pix2pix instruct training script.

* shorten cli arg.

* fix: main process check.

* fix: dataset column names.

* simplify tokenization.

* proper placement of null conditions.

* apply styling.

* remove debugging message for conditioning do.

* complete license.

* add: requirements.tzt

* wandb column name order.

* fix: augmentation.

* change: dataset_id.

* fix: convert_to_np() call.

* fix: reshaping.

* fix: final ema copy.

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* address PR comments.

* add: readme details.

* config fix.

* downgrade version.

* reduce image width in the readme.

* note on hyperparameters during generation.

* add: output images.

* update readme.

* minor edits to readme.

* debugging statement.

* explicitly placement of the pipeline.

* bump minimum diffusers version.

* fix: device attribute error.

* weight dtype.

* debugging.

* add dtype inform.

* add seoarate te and vae.

* add: explicit casting/

* remove casting.

* up.

* up 2.

* up 3.

* autocast.

* disable mixed-precision in the final inference.

* debugging information.

* autocasting.

* add: instructpix2pix training section to the docs.

* Empty-Commit

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-23 10:15:01 +05:30
Haofan Wang
e0d8c9ef83 Support for Offset Noise in examples (#2753) 
* add noise offset

* make style
 2023-03-23 09:36:17 +05:30
Patrick von Platen
ca1a22296d [MS Text To Video] Add first text to video (#2738) 
* [MS Text To Video} Add first text to video

* upload

* make first model example

* match unet3d params

* make sure weights are correcctly converted

* improve

* forward pass works, but diff result

* make forward work

* fix more

* finish

* refactor video output class.

* feat: add support for a video export utility.

* fix: opencv availability check.

* run make fix-copies.

* add: docs for the model components.

* add: standalone pipeline doc.

* edit docstring of the pipeline.

* add: right path to TransformerTempModel

* add: first set of tests.

* complete fast tests for text to video.

* fix bug

* up

* three fast tests failing.

* add: note on slow tests

* make work with all schedulers

* apply styling.

* add slow tests

* change file name

* update

* more correction

* more fixes

* finish

* up

* Apply suggestions from code review

* up

* finish

* make copies

* fix pipeline tests

* fix more tests

* Apply suggestions from code review

Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

* apply suggestions

* up

* revert

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
 2023-03-22 18:39:33 +01:00
Hyowon Ha
58bcf46a8f Add guidance start/end parameters to StableDiffusionControlNetImg2ImgPipeline (#2731) 
* Add guidance start/end parameters to community controlnet img2img pipeline

* Fix formats
 2023-03-21 14:38:43 +01:00
Nipun Jindal
0042efd015 [1929]: Add CLIP guidance for Img2Img stable diffusion pipeline (#2723) 
* [Img2Img]: Copyover img2img pipeline

* [Img2Img]: img2img pipeline

* [Img2Img]: img2img pipeline

* [Img2Img]: img2img pipeline

---------

Co-authored-by: njindal <njindal@adobe.com>
 2023-03-21 13:53:00 +01:00
Andy
116f70cbf8 Enabling gradient checkpointing for VAE (#2536) 
* updated black format

* update black format

* make style format

* updated line endings

* update code formatting

* Update examples/research_projects/onnxruntime/text_to_image/train_text_to_image.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/models/vae.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/diffusers/models/vae.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* added vae gradient checkpointing test

* make style

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Will Berman <wlbberman@gmail.com>
 2023-03-17 14:59:38 -07:00
Will Berman
a4b2c2f150 train_unconditional save restore unet parameters (#2706) 2023-03-16 16:15:56 -07:00
Patrick von Platen
e828232780 Rename attention (#2691) 
* rename file

* rename attention

* fix more

* rename more

* up

* more deprecation imports

* fixes
 2023-03-16 00:35:54 +01:00
Henrik Forstén
79eb3d07d0 Controlnet training (#2545) 
* Controlnet training code initial commit

Works with circle dataset: https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md

* Script for adding a controlnet to existing model

* Fix control image transform

Control image should be in 0..1 range.

* Add license header and remove more unused configs

* controlnet training readme

* Allow nonlocal model in add_controlnet.py

* Formatting

* Remove unused code

* Code quality

* Initialize controlnet in training script

* Formatting

* Address review comments

* doc style

* explicit constructor args and submodule names

* hub dataset

NOTE -  not tested

* empty prompts

* add conditioning image

* rename

* remove instance data dir

* image_transforms -> -1,1 . conditioning_image_transformers -> 0, 1

* nits

* remove local rank config

I think this isn't necessary in any of our training scripts

* validation images

* proportion_empty_prompts typo

* weight copying to controlnet bug

* call log validation fix

* fix

* gitignore wandb

* fix progress bar and resume from checkpoint iteration

* initial step fix

* log multiple images

* fix

* fixes

* tracker project name configurable

* misc

* add controlnet requirements.txt

* update docs

* image labels

* small fixes

* log validation using existing models for pipeline

* fix for deepspeed saving

* memory usage docs

* Update examples/controlnet/train_controlnet.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/train_controlnet.py

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/controlnet/README.md

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* remove extra is main process check

* link to dataset in intro paragraph

* remove unnecessary paragraph

* note on deepspeed

* Update examples/controlnet/README.md

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* assert -> value error

* weights and biases note

* move images out of git

* remove .gitignore

---------

Co-authored-by: William Berman <WLBberman@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-14 20:16:30 -07:00
aengusng8
268ebcb015 Add ddim noise comparative analysis pipeline (#2665) 
* add DDIM Noise Comparative Analysis pipeline

* update README

* add comments

* run BLACK format
 2023-03-14 18:09:55 +01:00
Will Berman
ebd44957fc image generation main process checks (#2631) 2023-03-14 01:28:03 -07:00
Ruizhe Wang
ce08cb72fb [Dreambooth] Editable number of class images (#2251) 
* [Dreambooth] Editable number of class images

* 'class_num=None' bug fix

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-10 14:15:16 +01:00
Patrick von Platen
cbbad0af69 correct example 2023-03-08 20:14:19 +01:00
Haofan Wang
00132de359 Support LoRA for text encoder (#2588) 
* add lora

* Update examples/research_projects/lora/README.md

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-03-08 20:14:01 +01:00