Chong Mou
12358b986f
add models for T2I-Adapter-XL ( #4696 )
...
* T2I-Adapter-XL
* update
* update
* add pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify pipeline
* modify modeling_text_unet
* fix styling.
* fix: copies.
* adapter settings
* new test case
* new test case
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* revert prints.
* new test case
* remove print
* org test case
* add test_pipeline
* styling.
* fix copies.
* modify test parameter
* style.
* add adapter-xl doc
* double quotes in docs
* Fix potential type mismatch
* style.
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
2023-08-29 10:34:07 +05:30
Shauray Singh
fd35689f25
[WIP] Add Fabric ( #4201 )
...
* empty PR
* init
* changes
* starting with the pipeline
* stable diff
* prev
* more things, getting started
* more functions
* makeing it more readable
* almost done testing
* var changes
* testing
* device
* device support
* maybe
* device malfunctions
* new new
* register
* testing
* exec does not work
* float
* change info
* change of architecture
* might work
* testing with colab
* more attn atuff
* stupid additions
* documenting and testing
* writing tests
* more docs
* tests and docs
* remove test
* empty PR
* init
* changes
* starting with the pipeline
* stable diff
* prev
* more things, getting started
* more functions
* makeing it more readable
* almost done testing
* var changes
* testing
* device
* device support
* maybe
* device malfunctions
* new new
* register
* testing
* exec does not work
* float
* change info
* change of architecture
* might work
* testing with colab
* more attn atuff
* stupid additions
* documenting and testing
* writing tests
* more docs
* tests and docs
* remove test
* change cross attention
* revert back
* tests
* reverting back to orig
* changes
* test passing
* pipeline changes
* before quality
* quality checks pass
* remove print statements
* doc fixes
* __init__ error something
* update docs, working on dim
* working on encoding
* doc fix
* more fixes
* no more dependent on 512*512
* update docs
* fixes
* test passing
* remove comment
* fixes and migration
* simpler tests
* doc changes
* green CI
* changes
* more docs
* changes
* new images
* to community examples
* selete
* more fixes
* changes
* fix
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-28 09:10:55 +02:00
Steven Liu
fed88195e3
[docs] Fix syntax for compel ( #4794 )
...
* fix syntax
* update image
2023-08-26 11:33:10 -07:00
Sayak Paul
3be0ff9056
[Core] Support negative conditions in SDXL ( #4774 )
...
* add: support negative conditions.
* fix: key
* add: tests
* address PR feedback.
* add documentation
* add img2img support.
* add inpainting support.
* ad controlnet support
* Apply suggestions from code review
* modify wording in the doc.
2023-08-26 09:13:44 +05:30
Mayank Khanduja
dc8da1d449
Fixed broken link of CLIP doc in evaluation doc ( #4760 )
2023-08-25 20:04:50 +02:00
Sanchit Gandhi
b1290d3fb8
Convert MusicLDM ( #4579 )
...
* from audioldm
* fix vae
* move to new pipeline
* copied from audioldm
* remove redundant control flow
* iterate
* fix docstring
* finish pipeline
* tests: from audioldm2
* iterate
* finish fast tests
* finish slow integration tests
* add docs
* remove dtype test
* update toctree
* "copied from" in conversion (where possible)
* Update docs/source/en/api/pipelines/musicldm.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* fix docstring
* make nightly
* style
* fix dtype test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-25 13:31:00 +01:00
Sanchit Gandhi
24c5e7708b
[AudioLDM2] Doc fixes ( #4739 )
...
* [AudioLDM2] Doc fixes
* update docstrings
* fix unet docstring
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-24 07:20:27 +05:30
realliujiaxu
124e76ddc6
[docs] add variant="fp16" flag ( #4678 )
2023-08-23 10:00:34 -07:00
Sanchit Gandhi
05b0ec63bc
[AudioLDM Docs] Fix docs for output ( #4737 )
2023-08-23 18:02:11 +02:00
Patrick von Platen
6abc66ef28
Fix all docs ( #4721 )
...
* [Docs] Fix all
* fix
2023-08-22 21:00:21 +02:00
Patrick von Platen
6e1af3a777
[Docs] Fix docs controlnet missing /Tip ( #4717 )
2023-08-22 18:40:26 +02:00
dg845
f75b8aa9dd
[docs] Add note in UniDiffusers Doc about PyTorch 1.X numerical stability issue ( #4703 )
...
* Add note regarding UniDiffuser pipeline numerical stability issues on PyTorch 1.X
* Use the doc-builder warning tag.
2023-08-22 07:12:06 +05:30
Sanchit Gandhi
7a24977ce3
Add AudioLDM 2 ( #4549 )
...
* from audioldm
* unet down + mid
* vae, clap, flan-t5
* start sequence audio mae
* iterate on audioldm encoder
* finish encoder
* finish weight conversion
* text pre-processing
* gpt2 pre-processing
* fix projection model
* working
* unet equivalence
* finish in base
* add unet cond
* finish unet
* finish custom unet
* start clean-up
* revert base unet changes
* refactor pre-processing
* tests: from audioldm
* fix some tests
* more fixes
* iterate on tests
* make fix copies
* harden fast tests
* slow integration tests
* finish tests
* update checkpoint
* update copyright
* docs
* remove outdated method
* add docstring
* make style
* remove decode latents
* enable cpu offload
* (text_encoder_1, tokenizer_1) -> (text_encoder, tokenizer)
* more clean up
* more refactor
* build pr docs
* Update docs/source/en/api/pipelines/audioldm2.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* small clean
* tidy conversion
* update for large checkpoint
* generate -> generate_language_model
* full clap model
* shrink clap-audio in tests
* fix large integration test
* fix fast tests
* use generation config
* make style
* update docs
* finish docs
* finish doc
* update tests
* fix last test
* syntax
* finalise tests
* refactor projection model in prep for TTS
* fix fast tests
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-08-21 12:34:21 +01:00
Sayak Paul
5333f4c0ec
make things clear in the controlnet sdxl doc. ( #4644 )
2023-08-17 09:04:28 +05:30
Steven Liu
bdc4c3265f
[docs] MultiControlNet ( #4635 )
...
multicontrolnet docs
2023-08-17 08:14:20 +05:30
Steven Liu
4ff7264d9b
[docs] PushToHubMixin ( #4622 )
...
* push to hub docs
* fix typo
* feedback
* make style
2023-08-16 13:20:59 -06:00
Sayak Paul
351aab60e9
Update text2image.md to fix the links ( #4626 )
2023-08-16 09:53:10 +05:30
nikhil-masterful
da5ab51d54
Add GLIGEN implementation ( #4441 )
...
* Add GLIGEN implementation
* GLIGEN: Fix code quality check failures
* GLIGEN: Fix Import block un-sorted or un-formatted failures
* GLIGEN: Fix check_repository_consistency failures
* GLIGEN: Add 'PositionNet' to versatile_diffusion/modeling_text_unet.py
* GLIGEN: check_repository_consistency: fix 'copy does not match' error
* GLIGEN: Fix review comments (1)
* GLIGEN: Fix E721 Do not compare types, use `isinstance()` failures
* GLIGEN : Ensure _encode_prompt() copy matches to StableDiffusionPipeline
* GLIGEN: Fix ruff E721 failure in unidiffuser/test_unidiffuser.py
* GLIGEN: doc_builder: restyle pipeline_stable_diffusion_gligen.py
* GIGLEN: reset files unrelated to gligen
* GLIGEN: Fix documentation comments (1)
* GLIGEN: Fix review comments (2)
* GLIGEN: Added FastTest
* GLIGEN: Fix review comments (3)
2023-08-16 09:34:17 +05:30
Sayak Paul
5175d3d7a5
add: train to text image with sdxl script. ( #4505 )
...
* add: train to text image with sdxl script.
Co-authored-by: CaptnSeraph <s3raph1m@gmail.com >
* fix: partial func.
* fix: default value of output_dir.
* make style
* set num inference steps to 25.
* remove mentions of LoRA.
* up min version
* add: ema cli arg
* run device placement while running step.
* precompute vae encodings too.
* fix
* debug
* should work now.
* debug
* debug
* goes alright?
* style
* debugging
* debugging
* debugging
* debugging
* fix
* reinit scheduler if prediction_type was passed.
* akways cast vae in float32
* better handling of snr.
Co-authored-by: bghira <bghira@users.github.com >
* the vae should be also passed
* add: docs.
* add: sdlx t2i tests
* save the pipeline
* autocast.
* fix: save_model_card
* fix: save_model_card.
---------
Co-authored-by: CaptnSeraph <s3raph1m@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: bghira <bghira@users.github.com >
2023-08-16 09:02:49 +05:30
Sayak Paul
a7508a76f0
add: pushtohubmixin to pipelines and schedulers docs overview. ( #4607 )
...
* add: pushtohubmixin to pipelines and schedulers docs overview.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-15 22:23:17 +05:30
Sayak Paul
aaef41b5fe
[Docs] fix links in the controlling generation doc. ( #4612 )
...
* fix links in the controlling generation doc.
* more fixes.
2023-08-15 20:27:13 +05:30
Sayak Paul
15782fd506
[Pipeline utils] feat: implement push_to_hub for standalone models, schedulers as well as pipelines ( #4128 )
...
* feat: implement push_to_hub for standalone models.
* address PR feedback.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* remove max_shard_size.
* add: support for scheduler push_to_hub
* enable push_to_hub support for flax schedulers.
* enable push_to_hub for pipelines.
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com >
* reflect pr feedback.
* address another round of deedback.
* better handling of kwargs.
* add: tests
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com >
* setting hub staging to False for now.
* incorporate staging test as a separate job.
Co-authored-by: ydshieh <2521628+ydshieh@users.noreply.github.com >
* fix: tokenizer loading.
* fix: json dumping.
* move is_staging_test to a better location.
* better treatment to tokens.
* define repo_id to better handle concurrency
* style
* explicitly set token
* Empty-Commit
* move SUER, TOKEN to test
* collate org_repo_id
* delete repo
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Lucain <lucainp@gmail.com >
Co-authored-by: ydshieh <2521628+ydshieh@users.noreply.github.com >
2023-08-15 07:39:22 +05:30
Claire Froelich
32963c24c5
Fix git-lfs command typo in docs ( #4586 )
...
fix typo in git-lfs command
added missing hyphen. "git lfs" is not a command
2023-08-14 17:21:45 +05:30
Sayak Paul
d67eba0f31
[Utility] adds an image grid utility ( #4576 )
...
* add: utility for image grid.
* add: return type.
* change necessary places.
* add to utility page.
2023-08-12 10:34:51 +05:30
Steven Liu
cd7071e750
[docs] Add safetensors flag ( #4245 )
...
* add safetensors flag
* apply review
2023-08-10 12:37:23 -07:00
Steven Liu
e31f38b5d6
[docs] Remove attention slicing ( #4518 )
...
* remove attention slicing
* apply feedback
2023-08-10 11:00:03 -07:00
Steven Liu
3bd5e073cb
[docs] Expand prompt weighting ( #4516 )
...
* add more weighting/blend/conjunction
* finish blend/conjunction
* add textual inversion example
* add dreambooth
2023-08-10 10:56:53 -07:00
YiYi Xu
3df52ba8dc
[Doc] update sdxl-controlnet repo name ( #4564 )
...
* rename
* style
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-08-10 22:02:32 +05:30
Sayak Paul
c697c5ab57
improve controlnet sdxl docs now that we have a good checkpoint. ( #4556 )
2023-08-10 08:21:36 +05:30
Patrick von Platen
5cbcbe3c63
Revert "introduce minimalistic reimplementation of SDXL on the SDXL doc" ( #4548 )
...
Revert "introduce minimalistic reimplementation of SDXL on the SDXL doc (#4532 )"
This reverts commit e7e3749498
2023-08-10 06:49:06 +05:30
Steven Liu
16ad13b61d
[docs] Clean scheduler api ( #4204 )
...
* clean scheduler mixin
* up to dpmsolvermultistep
* finish cleaning
* first draft
* fix overview table
* apply feedback
* update reference code
2023-08-09 09:00:35 -07:00
jere357
3c1b4933bd
Changed code that converts tensors to PIL images in the write_your_own_pipeline notebook ( #4489 )
...
changed code that converts tensors to PIL images
2023-08-09 15:00:51 +05:30
Simo Ryu
e7e3749498
introduce minimalistic reimplementation of SDXL on the SDXL doc ( #4532 )
...
minsdxl
2023-08-09 07:33:07 +05:30
Sayak Paul
b2456717e6
Update lora.md to clarify SDXL support ( #4503 )
...
* Update lora.md
* Update lora.md
2023-08-07 11:06:30 +05:30
takuoko
9c29bc2df8
[Examples] Support train_text_to_image_lora_sdxl.py ( #4365 )
...
* add train_text_to_image_lora_sdxl.py
* add train_text_to_image_lora_sdxl.py
* add test and minor fix
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* fix unwrap_model rule
* add invisible-watermark in requirements
* del invisible-watermark
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update examples/text_to_image/README_sdxl.md
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update examples/text_to_image/train_text_to_image_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* del comment & update readme
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-06 13:47:20 +05:30
Patrick von Platen
1a8843f93e
add sdxl to prompt weighting ( #4439 )
...
* add sdxl to prompt weighting
* Update docs/source/en/using-diffusers/weighted_prompts.md
* Update docs/source/en/using-diffusers/weighted_prompts.md
* add sdxl to prompt weighting
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Apply suggestions from code review
* Update docs/source/en/using-diffusers/weighted_prompts.md
* Apply suggestions from code review
* correct
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-03 21:41:48 +02:00
Steven Liu
5989a85edb
[docs] Distilled SD ( #4442 )
...
* first draft
* add blog link
2023-08-03 11:03:42 -07:00
Sayak Paul
18fc40c169
[Feat] add tiny Autoencoder for (almost) instant decoding ( #4384 )
...
* add: model implementation of tiny autoencoder.
* add: inits.
* push the latest devs.
* add: conversion script and finish.
* add: scaling factor args.
* debugging
* fix denormalization.
* fix: positional argument.
* handle use_torch_2_0_or_xformers.
* handle post_quant_conv
* handle dtype
* fix: sdxl image processor for tiny ae.
* fix: sdxl image processor for tiny ae.
* unify upcasting logic.
* copied from madness.
* remove trailing whitespace.
* set is_tiny_vae = False
* address PR comments.
* change to AutoencoderTiny
* make act_fn an str throughout
* fix: apply_forward_hook decorator call
* get rid of the special is_tiny_vae flag.
* directly scale the output.
* fix dummies?
* fix: act_fn.
* get rid of the Clamp() layer.
* bring back copied from.
* movement of the blocks to appropriate modules.
* add: docstrings to AutoencoderTiny
* add: documentation.
* changes to the conversion script.
* add doc entry.
* settle tests.
* style
* add one slow test.
* fix
* fix 2
* fix 2
* fix: 4
* fix: 5
* finish integration tests
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* style
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2023-08-02 23:58:05 +05:30
Steven Liu
ae82a3eb34
[docs] AutoPipeline tutorial ( #4273 )
...
* first draft
* tidy api
* apply feedback
* mdx to md
* apply feedback
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-02 10:32:02 -07:00
Sayak Paul
816ca0048f
[LoRA] Fix SDXL text encoder LoRAs ( #4371 )
...
* temporarily disable text encoder loras.
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debbuging.
* modify doc.
* rename tests.
* print slices.
* fix: assertions
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-08-02 17:00:56 +05:30
Ella Charlaix
579b4b2020
Update documentation ( #4422 )
...
* update documentation
* minor
2023-08-02 11:49:22 +02:00
estelleafl
05a1cb902c
[ldm3d] documentation fixing typos ( #4284 )
...
* fixed typo
* updated doc to be consistent in naming
* make style/quality
* preprocessing for 4 channels and not 6
* make style
* test for 4c
* make style/quality
* fixed test on cpu
* fixed doc typo
* changed default ckpt to 4c
* Update pipeline_stable_diffusion_ldm3d.py
---------
Co-authored-by: Aflalo <estellea@isl-iam1.rr.intel.com >
Co-authored-by: Aflalo <estellea@isl-gpu33.rr.intel.com >
Co-authored-by: Aflalo <estellea@isl-gpu38.rr.intel.com >
2023-08-01 09:03:29 -07:00
Patrick von Platen
c69526a3d5
[AutoPipeline] Correct naming ( #4420 )
2023-08-01 14:56:27 +02:00
Sayak Paul
ba43ce3476
minor doc fixes. ( #4380 )
2023-07-31 12:15:56 +05:30
Will Berman
2b1786735e
fix fp type in t2i adapter docs ( #4350 )
2023-07-28 13:01:52 -07:00
Sayak Paul
4a4cdd6b07
[Feat] Support SDXL Kohya-style LoRA ( #4287 )
...
* sdxl lora changes.
* better name replacement.
* better replacement.
* debugging
* debugging
* debugging
* debugging
* debugging
* remove print.
* print state dict keys.
* print
* distingisuih better
* debuggable.
* fxi: tyests
* fix: arg from training script.
* access from class.
* run style
* debug
* save intermediate
* some simplifications for SDXL LoRA
* styling
* unet config is not needed in diffusers format.
* fix: dynamic SGM block mapping for SDXL kohya loras (#4322 )
* Use lora compatible layers for linear proj_in/proj_out (#4323 )
* improve condition for using the sgm_diffusers mapping
* informative comment.
* load compatible keys and embedding layer maaping.
* Get SDXL 1.0 example lora to load
* simplify
* specif ranks and hidden sizes.
* better handling of k rank and hidden
* debug
* debug
* debug
* debug
* debug
* fix: alpha keys
* add check for handling LoRAAttnAddedKVProcessor
* sanity comment
* modifications for text encoder SDXL
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* denugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* up
* up
* up
* up
* up
* up
* unneeded comments.
* unneeded comments.
* kwargs for the other attention processors.
* kwargs for the other attention processors.
* debugging
* debugging
* debugging
* debugging
* improve
* debugging
* debugging
* more print
* Fix alphas
* debugging
* debugging
* debugging
* debugging
* debugging
* debugging
* clean up
* clean up.
* debugging
* fix: text
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Batuhan Taskaya <batuhan@python.org >
2023-07-28 19:49:49 +02:00
Patrick von Platen
b7b6d6138d
[SDXL] Make watermarker optional under certain circumstances to improve usability of SDXL 1.0 ( #4346 )
...
* improve sdxl
* more fixes
* improve sdxl
* improve sdxl
* improve sdxl
* finish
2023-07-28 19:29:22 +02:00
Ella Charlaix
92e5ddd295
Fix typo documentation ( #4320 )
...
fix typo documentation
2023-07-27 21:31:58 +05:30
YiYi Xu
80c10d8245
update Kandinsky doc ( #4301 )
...
* update doc
* fix an error in autopipe doc
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-07-27 13:10:41 +02:00
Patrick von Platen
20e92586c1
0.20.0dev0 ( #4299 )
...
* 0.20.0dev0
* make style
2023-07-26 23:06:18 +02:00
