Mark Van Aken
be4afa0bb4
#7535 Update FloatTensor type hints to Tensor ( #7883 )
...
* find & replace all FloatTensors to Tensor
* apply formatting
* Update torch.FloatTensor to torch.Tensor in the remaining files
* formatting
* Fix the rest of the places where FloatTensor is used as well as in documentation
* formatting
* Update new file from FloatTensor to Tensor
2024-05-10 09:53:31 -10:00
Sayak Paul
04f4bd54ea
[Core] introduce videoprocessor. ( #7776 )
...
* introduce videoprocessor.
* fix quality
* address yiyi's feedback
* fix preprocess_video call.
* video_processor -> image_processor
* fix
* fix more.
* quality
* image_processor -> video_processor
* support List[List[PIL.Image.Image]]
* change to video_processor.
* documentation
* Apply suggestions from code review
* changes
* remove print.
* refactor video processor (part # 7776) (#7861 )
* update
* update remove deprecate
* Update src/diffusers/video_processor.py
* update
* Apply suggestions from code review
* deprecate list of 5d for video and list of 4d for image + apply other feedbacks
* up
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* add doc.
* tensor2vid -> postprocess_video.
* refactor preprocess with preprocess_video
* set default values.
* empty commit
* more refactoring of prepare_latents in animatediff vid2vid
* checking documentation
* remove documentation for now.
* fix animatediff sdxl
* fix test failure [part of video processor PR] (#7905 )
up
* remove preceed_with_frames.
* doc
* fix
* fix
* remove video input as a single-frame video.
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-05-10 21:02:36 +02:00
Sayak Paul
82be58c512
add missing image processors to the docs ( #7910 )
...
add missing processors.
2024-05-10 14:53:57 +02:00
Sayak Paul
6695635696
upgrade to python 3.10 in the Dockerfiles ( #7893 )
...
* upgrade to python 3.10
* fix
* try https://askubuntu.com/questions/1459694/can-not-find-python3-10-after-apt-get-installation
* fix
* up
* yes
* okay
* up
* up
* up
* up
* up
* check
* okay
* up
* i[
* fix
2024-05-10 14:29:08 +02:00
YiYi Xu
b934215d4c
[scheduler] support custom timesteps and sigmas ( #7817 )
...
* support custom sigmas and timesteps, dpm euler
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com >
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com >
2024-05-09 11:07:43 -10:00
YiYi Xu
5ed3abd371
fix _optional_components in StableCascadeCombinedPipeline ( #7894 )
...
* fix
* up
2024-05-09 06:32:55 -10:00
Dhruv Nair
1087a510b5
Set max parallel jobs on slow test runners ( #7878 )
...
* set max parallel
* update
* update
* update
2024-05-09 19:42:18 +05:30
Sayak Paul
305f2b4498
[Tests] fix things after #7013 ( #7899 )
...
* debugging
* save the resulting image
* check if order reversing works.
* checking values.
* up
* okay
* checking
* fix
* remove print
2024-05-09 16:05:35 +02:00
Dhruv Nair
cb0f3b49cb
[Refactor] Better align from_single_file logic with from_pretrained ( #7496 )
...
* refactor unet single file loading a bit.
* retrieve the unet from create_diffusers_unet_model_from_ldm
* update
* update
* updae
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* tests
* update
* update
* update
* Update docs/source/en/api/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update src/diffusers/loaders/single_file.py
Co-authored-by: YiYi Xu <yixu310@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* Update docs/source/en/api/loaders/single_file.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-05-09 19:00:19 +05:30
Tolga Cangöz
caf9e985df
Fix several imports ( #7712 )
...
Fix imports
2024-05-09 07:34:44 +02:00
Tolga Cangöz
c1c42698c9
Remove dead code and fix f-string issue ( #7720 )
...
* Remove dead code
* PylancereportGeneralTypeIssues: Strings nested within an f-string cannot use the same quote character as the f-string prior to Python 3.12.
* Remove dead code
2024-05-08 13:15:28 -10:00
Pierre Dulac
75aab34675
Allow users to save SDXL LoRA weights for only one text encoder ( #7607 )
...
SDXL LoRA weights for text encoders should be decoupled on save
The method checks if at least one of unet, text_encoder and
text_encoder_2 lora weights are passed, which was not reflected in the
implentation.
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-05-08 10:41:58 -10:00
YiYi Xu
35358a2dec
fix offload test ( #7868 )
...
fix
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-05-08 07:59:08 -10:00
Aryan
818f760732
[Pipeline] AnimateDiff SDXL ( #6721 )
...
* update conversion script to handle motion adapter sdxl checkpoint
* add animatediff xl
* handle addition_embed_type
* fix output
* update
* add imports
* make fix-copies
* add decode latents
* update docstrings
* add animatediff sdxl to docs
* remove unnecessary lines
* update example
* add test
* revert conv_in conv_out kernel param
* remove unused param addition_embed_type_num_heads
* latest IPAdapter impl
* make fix-copies
* fix return
* add IPAdapterTesterMixin to tests
* fix return
* revert based on suggestion
* add freeinit
* fix test_to_dtype test
* use StableDiffusionMixin instead of different helper methods
* fix progress bar iterations
* apply suggestions from review
* hardcode flip_sin_to_cos and freq_shift
* make fix-copies
* fix ip adapter implementation
* fix last failing test
* make style
* Update docs/source/en/api/pipelines/animatediff.md
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
* remove todo
* fix doc-builder errors
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-05-08 21:27:14 +05:30
Philip Pham
f29b93488d
Check shape and remove deprecated APIs in scheduling_ddpm_flax.py ( #7703 )
...
`model_output.shape` may only have rank 1.
There are warnings related to use of random keys.
```
tests/schedulers/test_scheduler_flax.py: 13 warnings
/Users/phillypham/diffusers/src/diffusers/schedulers/scheduling_ddpm_flax.py:268: FutureWarning: normal accepts a single key, but was given a key array of shape (1, 2) != (). Use jax.vmap for batching. In a future JAX version, this will be an error.
noise = jax.random.normal(split_key, shape=model_output.shape, dtype=self.dtype)
tests/schedulers/test_scheduler_flax.py::FlaxDDPMSchedulerTest::test_betas
/Users/phillypham/virtualenv/diffusers/lib/python3.9/site-packages/jax/_src/random.py:731: FutureWarning: uniform accepts a single key, but was given a key array of shape (1,) != (). Use jax.vmap for batching. In a future JAX version, this will be an error.
u = uniform(key, shape, dtype, lo, hi) # type: ignore[arg-type]
```
2024-05-08 13:57:19 +02:00
Tolga Cangöz
d50baf0c63
Fix image upcasting ( #7858 )
...
Fix image's upcasting before `vae.encode()` when using `fp16`
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-05-07 16:45:02 -10:00
Hyoungwon Cho
c2217142bd
Modification on the PAG community pipeline (re) ( #7876 )
...
* edited_pag_implementation
* update
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com >
2024-05-07 16:35:15 -10:00
Bagheera
8edaf3b79c
7879 - adjust documentation to use naruto dataset, since pokemon is now gated ( #7880 )
...
* 7879 - adjust documentation to use naruto dataset, since pokemon is now gated
* replace references to pokemon in docs
* more references to pokemon replaced
* Japanese translation update
---------
Co-authored-by: bghira <bghira@users.github.com >
2024-05-07 09:36:39 -07:00
Álvaro Somoza
23e091564f
Fix for "no lora weight found module" with some loras ( #7875 )
...
* return layer weight if not found
* better system and test
* key example and typo
2024-05-07 13:54:57 +02:00
Steven Liu
0d23645bd1
[docs] Distilled inference ( #7834 )
...
* combine
* edits
2024-05-06 15:07:25 -07:00
Guillaume LEGENDRE
7fa3e5b0f6
Ci - change cache folder ( #7867 )
2024-05-06 17:55:24 +05:30
Steven Liu
49b959b540
[docs] LCM ( #7829 )
...
* lcm
* lcm lora
* fix
* fix hfoption
* edits
2024-05-03 16:08:27 -07:00
HelloWorldBeginner
58237364b1
Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed. ( #7816 )
...
* Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed.
* fix check code quality
* Decouple the NPU flash attention and make it an independent module.
* add doc and unit tests for npu flash attention.
---------
Co-authored-by: mhh001 <mahonghao1@huawei.com >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-05-03 08:14:34 -10:00
Dhruv Nair
3e35628873
Remove installing python again in container ( #7852 )
...
update
2024-05-03 15:09:15 +05:30
Lucain
6a479588db
Respect resume_download deprecation ( #7843 )
...
* Deprecate resume_download
* align docstring with transformers
* style
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-05-03 08:42:57 +02:00
Aritra Roy Gosthipaty
fa489eaed6
[Tests] reduce the model size in the blipdiffusion fast test ( #7849 )
...
reducing model size
2024-05-03 07:46:48 +05:30
Dhruv Nair
0d7c479023
Update deps for pipe test fetcher ( #7838 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-05-02 20:36:47 +05:30
Guillaume LEGENDRE
ce97d7e19b
Change GPU Runners ( #7840 )
...
* Move to new GPU Runners for slow tests
* Move to new GPU Runners for nightly tests
2024-05-02 18:48:46 +05:30
Guillaume LEGENDRE
44ba90caff
move to new runners ( #7839 )
2024-05-02 14:53:38 +02:00
Dhruv Nair
3c85a57297
Update CI cache ( #7832 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-05-02 14:03:35 +05:30
Dhruv Nair
03ca11318e
Update download diff format tests ( #7831 )
...
update
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-05-02 13:15:38 +05:30
Dhruv Nair
3ffa7b46e5
Fix hanging pipeline fetching ( #7837 )
...
update
2024-05-02 13:08:57 +05:30
yunseong Cho
c1b2a89e34
Fix key error for dictionary with randomized order in convert_ldm_unet_checkpoint ( #7680 )
...
fix key error for different order
Co-authored-by: yunseong <yunseong.cho@superlabs.us >
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com >
2024-05-02 10:29:55 +05:30
Aritra Roy Gosthipaty
435d37ce5a
[Tests] reduce the model size in the audioldm fast test ( #7833 )
...
chore: initial size reduction of models
2024-05-02 06:03:52 +05:30
YiYi Xu
5915c2985d
[ip-adapter] fix ip-adapter for StableDiffusionInstructPix2PixPipeline ( #7820 )
...
update prepare_ip_adapter_ for pix2pix
2024-05-01 06:27:43 -10:00
YiYi Xu
21a7ff12a7
update the logic of is_sequential_cpu_offload ( #7788 )
...
* up
* add comment to the tests + fix dit
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-05-01 06:25:57 -10:00
Sayak Paul
8909ab4b19
[Tests] fix: device map tests for models ( #7825 )
...
* fix: device module tests
* remove patch file
* Empty-Commit
2024-05-01 18:45:47 +05:30
Dhruv Nair
c1edb03c37
Fix for pipeline slow test fetcher ( #7824 )
...
* update
* update
2024-05-01 17:36:54 +05:30
Steven Liu
0d08370263
[docs] Community pipelines ( #7819 )
...
* community pipelines
* feedback
* consolidate
2024-04-30 14:10:14 -07:00
Tolga Cangöz
b8ccb46259
Fix CPU offload in docstring ( #7827 )
...
Fix cpu offload
2024-04-30 10:53:27 -07:00
Dhruv Nair
725ead2f5e
SSH Runner Workflow Update ( #7822 )
...
* add debug workflow
* update
2024-04-30 20:14:18 +05:30
Linoy Tsaban
26a7851e1e
Add B-Lora training option to the advanced dreambooth lora script ( #7741 )
...
* add blora
* add blora
* add blora
* add blora
* little changes
* little changes
* remove redundancies
* fixes
* add B LoRA to readme
* style
* inference
* defaults + path to loras+ generation
* minor changes
* style
* minor changes
* minor changes
* blora arg
* added --lora_unet_blocks
* style
* Update examples/advanced_diffusion_training/README.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
* add commit hash to B-LoRA repo cloneing
* change inference, remove cloning
* change inference, remove cloning
add section about configureable unet blocks
* change inference, remove cloning
add section about configureable unet blocks
* Apply suggestions from code review
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-04-30 09:46:30 +05:30
Sayak Paul
3fd31eef51
[Core] introduce _no_split_modules to ModelMixin ( #6396 )
...
* introduce _no_split_modules.
* unnecessary spaces.
* remove unnecessary kwargs and style
* fix: accelerate imports.
* change to _determine_device_map
* add the blocks that have residual connections.
* add: CrossAttnUpBlock2D
* add: testin
* style
* line-spaces
* quality
* add disk offload test without safetensors.
* checking disk offloading percentages.
* change model split
* add: utility for checking multi-gpu requirement.
* model parallelism test
* splits.
* splits.
* splits
* splits.
* splits.
* splits.
* offload folder to test_disk_offload_with_safetensors
* add _no_split_modules
* fix-copies
2024-04-30 08:46:51 +05:30
Aritra Roy Gosthipaty
b02e2113ff
[Tests] reduce the model size in the amused fast test ( #7804 )
...
* chore: reducing model sizes
* chore: shrinks further
* chore: shrinks further
* chore: shrinking model for img2img pipeline
* chore: reducing size of model for inpaint pipeline
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-04-30 08:11:26 +05:30
Aritra Roy Gosthipaty
21f023ec1a
[Tests] reduce the model size in the ddpm fast test ( #7797 )
...
* chore: reducing unet size for faster tests
* review suggestions
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-04-30 08:11:03 +05:30
Aritra Roy Gosthipaty
31d9f9ea77
[Tests] reduce the model size in the ddim fast test ( #7803 )
...
chore: reducing model size for ddim fast pipeline
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-04-30 07:54:38 +05:30
Clint Adams
f53352f750
Set main_input_name in StableDiffusionSafetyChecker to "clip_input" ( #7500 )
...
FlaxStableDiffusionSafetyChecker sets main_input_name to "clip_input".
This makes StableDiffusionSafetyChecker consistent.
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-04-29 11:39:59 -10:00
RuiningLi
83ae24ce2d
Added get_velocity function to EulerDiscreteScheduler. ( #7733 )
...
* Added get_velocity function to EulerDiscreteScheduler.
* Fix white space on blank lines
* Added copied from statement
* back to the original.
---------
Co-authored-by: Ruining Li <ruining@robots.ox.ac.uk >
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com >
2024-04-29 10:32:13 -10:00
jschoormans
8af793b2d4
Adding TextualInversionLoaderMixin for the controlnet_inpaint_sd_xl pipeline ( #7288 )
...
* added TextualInversionMixIn to controlnet_inpaint_sd_xl pipeline
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com >
2024-04-29 09:00:53 -10:00
Dhruv Nair
eb96ff0d59
Safetensor loading in AnimateDiff conversion scripts ( #7764 )
...
* update
* update
2024-04-29 17:36:50 +05:30
