TilmannR
4e77056885
Update README.md ( #6191 )
...
Typo: The script for LoRA training is `train_text_to_image_lora_prior.py` not `train_text_to_image_prior_lora.py`.
Alternatively you could rename the file and keep the README.md unchanged.
2023-12-18 19:08:29 +01:00
Dhruv Nair
a0c54828a1
Deprecate Pipelines ( #6169 )
...
* deprecate pipe
* make style
* update
* add deprecation message
* format
* remove tests for deprecated pipelines
* remove deprecation message
* make style
* fix copies
* clean up
* clean
* clean
* clean
* clean up
* clean up
* clean up toctree
* clean up
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-12-18 23:08:29 +05:30
Patrick von Platen
8d891e6e1b
[Torch Compile] Fix torch compile for svd vae ( #6217 )
2023-12-18 18:21:17 +01:00
Patrick von Platen
cce1fe2d41
[Text-to-Video] Clean up pipeline ( #6213 )
...
* make style
* make style
* make style
* make style
2023-12-18 18:21:09 +01:00
Abin Thomas
d816bcb5e8
Fix t2i. blog url ( #6205 )
2023-12-18 09:12:28 -08:00
d8ahazard
6976cab7ca
Fix possible re-conversion issues after extracting from safetensors ( #6097 )
...
* Fix possible re-conversion issues after extracting from diffusers
Properly rename specific vae keys.
* Whoops
2023-12-18 11:51:20 +01:00
Dhruv Nair
fcbed3fa79
Fix SDXL Inpainting from single file with Refiner Model ( #6147 )
...
* update
* update
* update
2023-12-18 11:45:37 +01:00
Sayak Paul
b98b314b7a
[Training] remove depcreated method from lora scripts. ( #6207 )
...
remove depcreated method from lora scripts.
2023-12-18 15:52:43 +05:30
Omar Sanseviero
74558ff65b
Nit fix to training params ( #6200 )
2023-12-18 11:06:16 +01:00
Yudong Jin
49644babd3
Fix the test script in examples/text_to_image/README.md ( #6209 )
...
* Update examples/text_to_image/README.md
* Update examples/text_to_image/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-18 15:36:00 +05:30
Sayak Paul
56b3b21693
[Refactor autoencoders] feat: introduce autoencoders module ( #6129 )
...
* feat: introduce autoencoders module
* more changes for styling and copy fixing
* path changes in the docs.
* fix: import structure in init.
* fix controlnetxs import
2023-12-18 12:42:15 +05:30
Sayak Paul
9cef07da5a
[Benchmarks] fix: lcm benchmarking reporting ( #6198 )
...
* fix: lcm benchmarking reporting
* fix generate_csv_dict call.
2023-12-17 15:32:11 +05:30
Sayak Paul
2d94c7838e
[Core] feat: enable fused attention projections for other SD and SDXL pipelines ( #6179 )
...
* feat: enable fused attention projections for other SD and SDXL pipelines
* add: test for SD fused projections.
2023-12-16 08:45:54 +05:30
Sayak Paul
a81334e3f0
[LoRA] add an error message when dealing with _best_guess_weight_name ofline ( #6184 )
...
* add an error message when dealing with _best_guess_weight_name ofline
* simplify condition
2023-12-16 08:36:08 +05:30
Dhruv Nair
d704a730cd
Compile test fix ( #6104 )
...
* update
* update
2023-12-15 18:34:46 +05:30
dg845
49db233b35
Clean Up Comments in LCM(-LoRA) Distillation Scripts. ( #6145 )
...
* Clean up comments in LCM(-LoRA) distillation scripts.
* Calculate predicted source noise noise_pred correctly for all prediction_types.
* make style
* apply suggestions from review
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-15 18:18:16 +05:30
Dhruv Nair
93ea26f272
Add PEFT to training deps ( #6148 )
...
add peft to training deps
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-15 08:39:59 +05:30
Dhruv Nair
f5dfe2a8b0
LoRA test fixes ( #6163 )
...
* update
* update
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-15 08:39:41 +05:30
Patrick von Platen
4836cfad98
[Sigmas] Keep sigmas on CPU ( #6173 )
...
* correct
* Apply suggestions from code review
* make style
2023-12-15 07:43:18 +05:30
Kuba
1ccbfbb663
[docs] Add missing \ in lora.md ( #6174 )
2023-12-14 16:55:43 -08:00
Linoy Tsaban
29dfe22a8e
[advanced dreambooth lora sdxl training script] load pipeline for inference only if validation prompt is used ( #6171 )
...
* load pipeline for inference only if validation prompt is used
* move things outside
* load pipeline for inference only if validation prompt is used
* fix readme when validation prompt is used
---------
Co-authored-by: linoytsaban <linoy@huggingface.co >
Co-authored-by: apolinário <joaopaulo.passos@gmail.com >
2023-12-14 11:45:33 -06:00
Aryan V S
56806cdbfd
Add missing subclass docs, Fix broken example in SD_safe ( #6116 )
...
* fix broken example in pipeline_stable_diffusion_safe
* fix typo in pipeline_stable_diffusion_pix2pix_zero
* add missing docs
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-14 09:20:30 -08:00
Steven Liu
8ccc76ab37
[docs] IP-Adapter API doc ( #6140 )
...
add ip-adapter
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-14 09:19:37 -08:00
Monohydroxides
c46711e895
[Community] Add SDE Drag pipeline ( #6105 )
...
* Add community pipeline: sde_drag.py
* Update README.md
* Update README.md
Update example code and visual example
* Update sde_drag.py
Update code example.
2023-12-14 20:47:20 +05:30
Sayak Paul
1d686bac81
[feat: Benchmarking Workflow] add stuff for a benchmarking workflow ( #5839 )
...
* add poc for benchmarking workflow.
* import
* fix argument
* fix: argument
* fix: path
* fix
* fix
* path
* output csv files.
* workflow cleanup
* append token
* add utility to push to hf dataset
* fix: kw arg
* better reporting
* fix: headers
* better formatting of the numbers.
* better type annotation
* fix: formatting
* moentarily disable check
* push results.
* remove disable check
* introduce base classes.
* img2img class
* add inpainting pipeline
* intoduce base benchmark class.
* add img2img and inpainting
* feat: utility to compare changes
* fix
* fix import
* add args
* basepath
* better exception handling
* better path handling
* fix
* fix
* remove
* ifx
* fix
* add: support for controlnet.
* image_url -> url
* move images to huggingface hub
* correct urls.
* root_ckpt
* flush before benchmarking
* don't install accelerate from source
* add runner
* simplify Diffusers Benchmarking step
* change runner
* fix: subprocess call.
* filter percentage values
* fix controlnet benchmark
* add t2i adapters.
* fix filter columns
* fix t2i adapter benchmark
* fix init.
* fix
* remove safetensors flag
* fix args print
* fix
* feat: run_command
* add adapter resolution mapping
* benchmark t2i adapter fix.
* fix adapter input
* fix
* convert to L.
* add flush() add appropriate places
* better filtering
* okay
* get env for torch
* convert to float
* fix
* filter out nans.
* better coment
* sdxl
* sdxl for other benchmarks.
* fix: condition
* fix: condition for inpainting
* fix: mapping for resolution
* fix
* include kandinsky and wuerstchen
* fix: Wuerstchen
* Empty-Commit
* [Community] AnimateDiff + Controlnet Pipeline (#5928 )
* begin work on animatediff + controlnet pipeline
* complete todos, uncomment multicontrolnet, input checks
Co-Authored-By: EdoardoBotta <botta.edoardo@gmail.com >
* update
Co-Authored-By: EdoardoBotta <botta.edoardo@gmail.com >
* add example
* update community README
* Update examples/community/README.md
---------
Co-authored-by: EdoardoBotta <botta.edoardo@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* EulerDiscreteScheduler add `rescale_betas_zero_snr` (#6024 )
* EulerDiscreteScheduler add `rescale_betas_zero_snr`
* Revert "[Community] AnimateDiff + Controlnet Pipeline (#5928 )"
This reverts commit 821726d7c0#6024 )"
This reverts commit 3dc2362b5apatrick.v.platen@gmail.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: Aryan V S <contact.aryanvs@gmail.com >
Co-authored-by: EdoardoBotta <botta.edoardo@gmail.com >
Co-authored-by: Beinsezii <39478211+Beinsezii@users.noreply.github.com >
2023-12-12 11:03:34 +05:30
M. Tolga Cangöz
0a401b95b7
[Docs] Fix typos ( #6122 )
...
Fix typos and trim trailing whitespaces
2023-12-11 10:55:28 -08:00
Edward Li
664e931bcb
Correct type annotation for VaeImageProcessor.numpy_to_pil ( #6111 )
...
From `(np.ndarray) -> PIL.Image.Image` to `(np.ndarray) -> List[PIL.Image.Image]`.
2023-12-11 15:22:04 +05:30
Aryan V S
88bdd97ccd
IP adapter support for most pipelines ( #5900 )
...
* support ip-adapter in src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_upscale.py
* support ip-adapter in src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_attend_and_excite.py
* support ip-adapter in src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_instruct_pix2pix.py
* update tests
* support ip-adapter in src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_panorama.py
* support ip-adapter in src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_sag.py
* support ip-adapter in src/diffusers/pipelines/stable_diffusion_safe/pipeline_stable_diffusion_safe.py
* support ip-adapter in src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_text2img.py
* support ip-adapter in src/diffusers/pipelines/latent_consistency_models/pipeline_latent_consistency_img2img.py
* support ip-adapter in src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_ldm3d.py
* revert changes to sd_attend_and_excite and sd_upscale
* make style
* fix broken tests
* update ip-adapter implementation to latest
* apply suggestions from review
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-10 21:19:14 +05:30
Charchit Sharma
08b453e382
IP-Adapter for StableDiffusionControlNetImg2ImgPipeline ( #5901 )
...
* adapter for StableDiffusionControlNetImg2ImgPipeline
* fix-copies
* fix-copies
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-09 11:02:55 +05:30
apolinário
2a111bc9fe
[Advanced Training Script] Fix pipe example ( #6106 )
2023-12-08 15:56:35 +01:00
apolinário
16e6997f0d
[Advanced Diffusion Script] Add Widget default text ( #6100 )
...
add widget
2023-12-08 12:45:27 +01:00
YiYi Xu
3b9b98656e
Fix a bug in add_noise function ( #6085 )
...
* fix
* copies
---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
2023-12-07 11:35:28 -10:00
Fabio Rigano
b65928b556
Add support for IPAdapterFull ( #5911 )
...
* Add support for IPAdapterFull
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-12-07 06:40:39 -10:00
Beinsezii
6bf1ca2c79
EulerDiscreteScheduler add rescale_betas_zero_snr ( #6024 )
...
* EulerDiscreteScheduler add `rescale_betas_zero_snr`
2023-12-06 21:51:04 -10:00
Aryan V S
978dec9014
[Community] AnimateDiff + Controlnet Pipeline ( #5928 )
...
* begin work on animatediff + controlnet pipeline
* complete todos, uncomment multicontrolnet, input checks
Co-Authored-By: EdoardoBotta <botta.edoardo@gmail.com >
* update
Co-Authored-By: EdoardoBotta <botta.edoardo@gmail.com >
* add example
* update community README
* Update examples/community/README.md
---------
Co-authored-by: EdoardoBotta <botta.edoardo@gmail.com >
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-12-06 21:01:41 -10:00
Dhruv Nair
79a7ab92d1
Fix clearing backend cache from device agnostic testing ( #6075 )
...
update
2023-12-07 11:18:31 +05:30
Younes Belkada
c2717317f0
[PEFT] Adapt example scripts to use PEFT ( #5388 )
...
* adapt example scripts to use PEFT
* Update examples/text_to_image/train_text_to_image_lora.py
* fix
* add for SDXL
* oops
* make sure to install peft
* fix
* fix
* fix dreambooth and lora
* more fixes
* add peft to requirements.txt
* fix
* final fix
* add peft version in requirements
* remove comment
* change variable names
* add few lines in readme
* add to reqs
* style
* fix issues
* fix lora dreambooth xl tests
* init_lora_weights to gaussian and add out proj where missing
* ammend requirements.
* ammend requirements.txt
* add correct peft versions
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-07 09:39:29 +05:30
Ian
bf7f9b49a2
Fix typing inconsistency in Euler discrete scheduler ( #6052 )
2023-12-06 23:45:16 +01:00
UmerHA
e192ae08d3
Add ControlNet-XS support ( #5827 )
...
* Check in 23-10-05
* check-in 23-10-06
* check-in 23-10-07 2pm
* check-in 23-10-08
* check-in 231009T1200
* check-in 230109
* checkin 231010
* init + forward run
* checkin
* checkin
* ControlNetXSModel is now saveable+loadable
* Forward works
* checkin
* Pipeline works with `no_control=True`
* checkin
* debug: save intermediate outputs of resnet
* checkin
* Understood time error + fixed connection error
* checkin
* checkin 231106T1600
* turned off detailled debug prints
* time debug logs
* small fix
* Separated control_scale for connections/time
* simplified debug logging
* Full denoising works with control scale = 0
* aligned logs
* Added control_attention_head_dim param
* Passing n_heads instead of dim_head into ctrl unet
* Fixed ctrl midblock bug
* Cleanup
* Fixed time dtype bug
* checkin
* 1. from_unet, 2. base passed, 3. all unet params
* checkin
* Finished docstrings
* cleanup
* make style
* checkin
* more tests pass
* Fixed tests
* removed debug logs
* make style + quality
* make fix-copies
* fixed documentation
* added cnxs to doc toc
* added control start/end param
* Update controlnetxs_sdxl.md
* tried to fix copies..
* Fixed norm_num_groups in from_unet
* added sdxl-depth test
* created SD2.1 controlnet-xs pipeline
* re-added debug logs
* Adjusting group norm ; readded logs
* Added debug log statements
* removed debug logs ; started tests for sd2.1
* updated sd21 tests
* fixed tests
* fixed tests
* slightly increased error tolerance for 1 test
* make style & quality
* Added docs for CNXS-SD
* make fix-copies
* Fixed sd compile test ; fixed gradient ckpointing
* vae downs = cnxs conditioning downs; removed guess
* make style & quality
* Fixed tests
* fixed test
* Incorporated review feedback
* simplified control model surgery
* fixed tests & make style / quality
* Updated docs; deleted pip & cursor files
* Rolled back minimal change to resnet
* Update resnet.py
* Update resnet.py
* Update src/diffusers/models/controlnetxs.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Update src/diffusers/models/controlnetxs.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* Incorporated review feedback
* Update docs/source/en/api/pipelines/controlnetxs_sdxl.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/api/pipelines/controlnetxs.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/api/pipelines/controlnetxs.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/api/pipelines/controlnetxs.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/controlnetxs.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/models/controlnetxs.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/pipelines/controlnet_xs/pipeline_controlnet_xs.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update docs/source/en/api/pipelines/controlnetxs.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Update src/diffusers/pipelines/controlnet_xs/pipeline_controlnet_xs_sd_xl.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
* Incorporated doc feedback
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2023-12-06 23:33:47 +01:00
Steven Liu
87a09d66f3
[docs] SDXL Turbo ( #6065 )
...
api docs
2023-12-06 14:33:14 -08:00
Lucain
75ada25048
Harmonize HF environment variables + deprecate use_auth_token ( #6066 )
...
* Harmonize HF environment variables + deprecate use_auth_token
* fix import
* fix
2023-12-06 22:22:31 +01:00
Patrick von Platen
2243a59483
[Euler Discrete] Fix sigma ( #6078 )
...
* [Euler Discrete] Fix sigma
* make style
2023-12-06 19:59:38 +01:00
apolinário
466d32c442
[Advanced Diffusion Training] Cache latents to avoid VAE passes for every training step ( #6076 )
...
* add cache latents
* style
2023-12-06 14:46:53 +01:00
Dhruv Nair
20ba1fdbbd
Disable Tests Fetcher ( #6060 )
...
update
2023-12-06 18:10:11 +05:30
Pedro Cuenca
ab6672fecd
Use CC12M for LCM WDS training example ( #5908 )
...
* Fix SD scripts - there are only 2 items per batch
* Adjustments to make the SDXL scripts work with other datasets
* Use public webdataset dataset for examples
* make style
* Minor tweaks to the readmes.
* Stress that the database is illustrative.
2023-12-06 10:35:36 +01:00
Dhruv Nair
f90a5139a2
fix
2023-12-06 06:03:58 +00:00
Sayak Paul
a2bc2e14b9
[feat] allow SDXL pipeline to run with fused QKV projections ( #6030 )
...
* debug
* from step
* print
* turn sigma a list
* make str
* init_noise_sigma
* comment
* remove prints
* feat: introduce fused projections
* change to a better name
* no grad
* device.
* device
* dtype
* okay
* print
* more print
* fix: unbind -> split
* fix: qkv >-> k
* enable disable
* apply attention processor within the method
* attn processors
* _enable_fused_qkv_projections
* remove print
* add fused projection to vae
* add todos.
* add: documentation and cleanups.
* add: test for qkv projection fusion.
* relax assertions.
* relax further
* fix: docs
* fix-copies
* correct error message.
* Empty-Commit
* better conditioning on disable_fused_qkv_projections
* check
* check processor
* bfloat16 computation.
* check latent dtype
* style
* remove copy temporarily
* cast latent to bfloat16
* fix: vae -> self.vae
* remove print.
* add _change_to_group_norm_32
* comment out stuff that didn't work
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* reflect patrick's suggestions.
* fix imports
* fix: disable call.
* fix more
* fix device and dtype
* fix conditions.
* fix more
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
2023-12-06 07:33:26 +05:30
Arsalan
f427345ab1
Device agnostic testing ( #5612 )
...
* utils and test modifications to enable device agnostic testing
* device for manual seed in unet1d
* fix generator condition in vae test
* consistency changes to testing
* make style
* add device agnostic testing changes to source and one model test
* make dtype check fns private, log cuda fp16 case
* remove dtype checks from import utils, move to testing_utils
* adding tests for most model classes and one pipeline
* fix vae import
2023-12-05 19:04:13 +05:30
apolinário
6e221334cd
[advanced_dreambooth_lora_sdxl_tranining_script] save embeddings locally fix ( #6058 )
...
* Update train_dreambooth_lora_sdxl_advanced.py
* remove global function args from dreamboothdataset class
* style
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2023-12-05 13:52:34 +01:00
Patrick von Platen
53bc30dd45
[From single file] remove depr warning ( #6043 )
2023-12-05 18:12:25 +05:30
