Aryan
ec8d483e72
deepspeed refactor
2024-09-16 13:54:34 +02:00
Aryan
b07ac7496a
add requirements.txt
2024-09-16 03:23:02 +02:00
Aryan
4679088bfe
deepspeed stuff
2024-09-16 03:05:29 +02:00
Aryan
a3ca2a2732
update lora tests
2024-09-16 02:57:10 +02:00
Aryan
e3a9fe7bde
Merge branch 'main' into cogvideox-lora-and-training
2024-09-15 22:25:25 +02:00
Aryan
e3c267750b
add test skeleton
2024-09-15 22:25:20 +02:00
Aryan
e83200ece5
update
2024-09-15 22:03:23 +02:00
Aryan
96b2f17b1a
fix
2024-09-15 20:27:44 +02:00
Linoy Tsaban
37e3603c4a
[Flux Dreambooth lora] add latent caching ( #9160 )
...
* add ostris trainer to README & add cache latents of vae
* add ostris trainer to README & add cache latents of vae
* style
* readme
* add test for latent caching
* add ostris noise scheduler
9ee1ef2a0a/toolkit/samplers/custom_flowmatch_sampler.py (L95)spsayakpaul@gmail.com >
* Update examples/dreambooth/README_flux.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* use clear_objs_and_retain_memory from utilities
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-09-15 15:30:31 +03:00
Aryan
0bd238a522
make style
2024-09-15 00:07:56 +02:00
Aryan
8aa62dd436
update
2024-09-15 00:07:29 +02:00
Aryan
f1740b0fb5
Merge branch 'main' into cogvideox-lora-and-training
2024-09-14 23:44:05 +02:00
Aryan
5e5ee430cb
update
2024-09-14 23:41:53 +02:00
Leo Jiang
e2ead7cdcc
Fix the issue on sd3 dreambooth w./w.t. lora training ( #9419 )
...
* Fix dtype error
* [bugfix] Fixed the issue on sd3 dreambooth training
* [bugfix] Fixed the issue on sd3 dreambooth training
---------
Co-authored-by: θη‘ <jiangshuo9@h-partners.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-09-14 16:29:38 +05:30
Aryan
e5a44fd55c
add README
2024-09-14 04:30:01 +02:00
Aryan
80c87718ad
apply suggestions from review; prodigy optimizer
...
YiYi Xu <yixu310@gmail.com >
2024-09-14 02:36:07 +02:00
Aryan
165d9b2dd9
Merge branch 'main' into cogvideox-lora-and-training
2024-09-12 13:04:54 +05:30
Aryan
7e9e25cf6c
Update examples/cogvideo/train_cogvideox_lora.py
...
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-12 13:04:31 +05:30
Juan Acevedo
45aa8bb187
Ptxla sd training ( #9381 )
...
* enable pxla training of stable diffusion 2.x models.
* run linter/style and run pipeline test for stable diffusion and fix issues.
* update xla libraries
* fix read me newline.
* move files to research folder.
* update per comments.
* rename readme.
---------
Co-authored-by: Juan Acevedo <jfacevedo@google.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-09-12 08:35:06 +05:30
Yu Zheng
c002731d93
[examples] add controlnet sd3 example ( #9249 )
...
* add controlnet sd3 example
* add controlnet sd3 example
* update controlnet sd3 example
* add controlnet sd3 example test
* fix quality and style
* update test
* update test
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-09-11 07:04:37 +05:30
Sayak Paul
adf1f911f0
[Tests] fix some fast gpu tests. ( #9379 )
...
fix some fast gpu tests.
2024-09-11 06:50:02 +05:30
Aryan
f35c36c16d
use correct lora scaling for final test pipeline
2024-09-09 23:45:47 +02:00
Aryan
b1ca3dbadf
fix lora alpha
2024-09-07 22:27:54 +02:00
Aryan
4159b3bc82
Update examples/cogvideo/train_cogvideox_lora.py
...
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-09-08 00:08:25 +05:30
Aryan
a063503044
apply suggestions from review
2024-09-07 14:46:29 +02:00
Aryan
038fec4ac9
update
2024-09-07 13:25:18 +02:00
Aryan
3afb2e2540
Merge branch 'main' into cogvideox-lora-and-training
2024-09-06 23:04:50 +02:00
Linoy Tsaban
55ac421f7b
improve README for flux dreambooth lora ( #9290 )
...
* improve readme
* improve readme
* improve readme
* improve readme
2024-09-05 17:53:23 +05:30
Aryan
4c56287535
update
2024-09-04 07:58:17 +02:00
Sayak Paul
8ba90aa706
chore: add a cleaning utility to be useful during training. ( #9240 )
2024-09-03 15:00:17 +05:30
Aryan
9a95d8de56
update
2024-09-03 05:57:48 +02:00
Aryan
63e80b77fc
Merge branch 'main' into cogvideox-lora-and-training
2024-09-03 05:22:22 +02:00
Aryan
74e6f90097
update
2024-09-01 02:35:19 +02:00
Aryan
588c6ee602
update
2024-08-31 15:41:26 +02:00
Aryan
24c362ca4f
update
2024-08-29 18:53:04 +02:00
Anand Kumar
40c13fe5b4
[train_custom_diffusion.py] Fix the LR schedulers when num_train_epochs is passed in a distributed training env ( #9308 )
...
* Update train_custom_diffusion.py to fix the LR schedulers for `num_train_epochs`
* Fix saving text embeddings during safe serialization
* Fixed formatting
2024-08-29 14:23:36 +05:30
Aryan
f12e669ed3
update
2024-08-28 15:51:57 +02:00
Aryan
dc08234a7c
cogvideox lora training draft
2024-08-28 15:35:30 +02:00
Linoy Tsaban
c977966502
[Dreambooth flux] bug fix for dreambooth script (align with dreambooth lora) ( #9257 )
...
* fix shape
* fix prompt encoding
* style
* fix device
* add comment
2024-08-26 17:29:58 +05:30
Leo Jiang
eda36c4c28
Fix dtype error for StableDiffusionXL ( #9217 )
...
Fix dtype error
Co-authored-by: θη‘ <jiangshuo9@h-partners.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-20 06:58:50 +05:30
Wenlong Wu
815d882217
Add loading text inversion ( #9130 )
2024-08-19 09:26:27 +05:30
townwish4git
d25eb5d385
fix(sd3): fix deletion of text_encoders etc ( #8951 )
...
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-08-18 15:37:40 -10:00
Tolga CangΓΆz
7ef8a46523
[Docs] Fix CPU offloading usage ( #9207 )
...
* chore: Fix cpu offloading usage
* Trim trailing white space
* docs: update Kolors model link in kolors.md
2024-08-18 13:12:12 -10:00
dependabot[bot]
0c1e63bd11
Bump jinja2 from 3.1.3 to 3.1.4 in /examples/research_projects/realfill ( #7873 )
...
Bumps [jinja2](https://github.com/pallets/jinja ) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases )
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst )
- [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4 )
---
updated-dependencies:
- dependency-name: jinja2
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-14 16:36:59 +05:30
dependabot[bot]
e7e45bd127
Bump torch from 2.0.1 to 2.2.0 in /examples/research_projects/realfill ( #8971 )
...
Bumps [torch](https://github.com/pytorch/pytorch ) from 2.0.1 to 2.2.0.
- [Release notes](https://github.com/pytorch/pytorch/releases )
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md )
- [Commits](https://github.com/pytorch/pytorch/compare/v2.0.1...v2.2.0 )
---
updated-dependencies:
- dependency-name: torch
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-14 16:36:46 +05:30
Γlvaro Somoza
82058a5413
post release 0.30.0 ( #9173 )
...
* post release
* fix quality
2024-08-14 12:55:55 +05:30
Linoy Tsaban
413ca29b71
[Flux Dreambooth LoRA] - te bug fixes & updates ( #9139 )
...
* add requirements + fix link to bghira's guide
* text ecnoder training fixes
* text encoder training fixes
* text encoder training fixes
* text encoder training fixes
* style
* add tests
* fix encode_prompt call
* style
* unpack_latents test
* fix lora saving
* remove default val for max_sequenece_length in encode_prompt
* remove default val for max_sequenece_length in encode_prompt
* style
* testing
* style
* testing
* testing
* style
* fix sizing issue
* style
* revert scaling
* style
* style
* scaling test
* style
* scaling test
* remove model pred operation left from pre-conditioning
* remove model pred operation left from pre-conditioning
* fix trainable params
* remove te2 from casting
* transformer to accelerator
* remove prints
* empty commit
2024-08-12 11:58:03 +05:30
Dibbla!
3ece143308
Errata - fix typo ( #9100 )
2024-08-12 07:30:19 +05:30
Daniel Socek
c1079f0887
Fix textual inversion SDXL and add support for 2nd text encoder ( #9010 )
...
* Fix textual inversion SDXL and add support for 2nd text encoder
Signed-off-by: Daniel Socek <daniel.socek@intel.com >
* Fix style/quality of text inv for sdxl
Signed-off-by: Daniel Socek <daniel.socek@intel.com >
---------
Signed-off-by: Daniel Socek <daniel.socek@intel.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-09 20:23:06 +05:30
Linoy Tsaban
65e30907b5
[Flux] Dreambooth LoRA training scripts ( #9086 )
...
* initial commit - dreambooth for flux
* update transformer to be FluxTransformer2DModel
* update training loop and validation inference
* fix sd3->flux docs
* add guidance handling, not sure if it makes sense(?)
* inital dreambooth lora commit
* fix text_ids in compute_text_embeddings
* fix imports of static methods
* fix pipeline loading in readme, remove auto1111 docs for now
* fix pipeline loading in readme, remove auto1111 docs for now, remove some irrelevant text_encoder_3 refs
* Update examples/dreambooth/train_dreambooth_flux.py
Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com >
* fix te2 loading and remove te2 refs from text encoder training
* fix tokenizer_2 initialization
* remove text_encoder training refs from lora script (for now)
* try with vae in bfloat16, fix model hook save
* fix tokenization
* fix static imports
* fix CLIP import
* remove text_encoder training refs (for now) from lora script
* fix minor bug in encode_prompt, add guidance def in lora script, ...
* fix unpack_latents args
* fix license in readme
* add "none" to weighting_scheme options for uniform sampling
* style
* adapt model saving - remove text encoder refs
* adapt model loading - remove text encoder refs
* initial commit for readme
* Update examples/dreambooth/train_dreambooth_lora_flux.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update examples/dreambooth/train_dreambooth_lora_flux.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* fix vae casting
* remove precondition_outputs
* readme
* readme
* style
* readme
* readme
* update weighting scheme default & docs
* style
* add text_encoder training to lora script, change vae_scale_factor value in both
* style
* text encoder training fixes
* style
* update readme
* minor fixes
* fix te params
* fix te params
---------
Co-authored-by: Bagheera <59658056+bghira@users.noreply.github.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-08-09 07:31:04 +05:30
