* add componentspec and configspec
* up
* up
* move methods to blocks
* Modular Diffusers Guiders (#11311)
* cfg; slg; pag; sdxl without controlnet
* support sdxl controlnet
* support controlnet union
* update
* update
* cfg zero*
* use unwrap_module for torch compiled modules
* remove guider kwargs
* remove commented code
* remove old guider
* fix slg bug
* remove debug print
* autoguidance
* smoothed energy guidance
* add note about seg
* tangential cfg
* cfg plus plus
* support cfgpp in ddim
* apply review suggestions
* refactor
* rename enable/disable
* remove cfg++ for now
* rename do_classifier_free_guidance->prepare_unconditional_embeds
* remove unused
* [modular diffusers] introducing ModularLoader (#11462)
* cfg; slg; pag; sdxl without controlnet
---------
Co-authored-by: Aryan <aryan@huggingface.co>
* make loader optional
* remove lora step and ip-adapter step -> no longer needed
* rename pipeline -> components, data -> block_state
* seperate controlnet step into input + denoise
* refactor controlnet union
* reefactor pipeline/block states so that it can dynamically accept kwargs
* remove controlnet union denoise step, refactor & reuse controlnet denoisee step to accept aditional contrlnet kwargs
* allow input_fields as input & update message
* update input formating, consider kwarggs_type inputs with no name, e/g *_controlnet_kwargs
* refactor the denoiseestep using LoopSequential! also add a new file for denoise step
* change warning to debug
* fix get_execusion blocks with loopsequential
* fix auto denoise so all tests pass
* update imports on guiders
* remove modular reelated change from pipelines folder
* made a modular_pipelines folder!
* update __init__
* add notes
* add block state will also make sure modifed intermediates_inputs will be updated
* move block mappings to its own file
* make inputs truly immutable, remove the output logic in sequential pipeline, and update so that intermediates_outputs are only new variables
* decode block, if skip decoding do not need to update latent
* fix imports
* fix import
* fix more
* remove the output step
* make generator intermediates (it is mutable)
* after_denoise -> decoders
* add a to-do for guider cconfig mixin
* refactor component spec: replace create/create_from_pretrained/create_from_config to just create and load method
* refactor modular loader: 1. load only load (pretrained components only if not specific names) 2. update acceept create spec 3. move the updte _componeent_spec logic outside register_components to each method that create/update the component: __init__/update/load
* update components manager
* up
* [WIP] Modular Diffusers support custom code/pipeline blocks (#11539)
* update
* update
* remove the duplicated components_manager file I forgot to deletee
* fix import in block mapping
* add a to-do for modular loader
* prepare_latents_img2img pipeline method -> function, maybe do the same for others?
* update input for loop blocks, do not need to include intermediate
* solve merge conflict: manually add back the remote code change to modular_pipeline
* add node_utils
* modular node!
* add
* refator based on dhruv's feedbacks
* update doc format for kwargs_type
* up
* updatee modular_pipeline.from_pretrained, modular_repo ->pretrained_model_name_or_path
* save_pretrained for serializing config. (#11603)
* save_pretrained for serializing config.
* remove pushtohub
* diffusers-cli rough
---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>
---------
Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* 1. add pre-computation of prompt embeddings when custom prompts are used as well
2. save model card even if model is not pushed to hub
3. remove scheduler initialization from code example - not necessary anymore (it's now if the base model's config)
4. add skip_final_inference - to allow to run with validation, but skip the final loading of the pipeline with the lora weights to reduce memory reqs
* pre encode validation prompt as well
* Update examples/dreambooth/train_dreambooth_lora_hidream.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Update examples/dreambooth/train_dreambooth_lora_hidream.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Update examples/dreambooth/train_dreambooth_lora_hidream.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* pre encode validation prompt as well
* Apply style fixes
* empty commit
* change default trained modules
* empty commit
* address comments + change encoding of validation prompt (before it was only pre-encoded if custom prompts are provided, but should be pre-encoded either way)
* Apply style fixes
* empty commit
* fix validation_embeddings definition
* fix final inference condition
* fix pipeline deletion in last inference
* Apply style fixes
* empty commit
* layers
* remove readme remarks on only pre-computing when instance prompt is provided and change example to 3d icons
* smol fix
* empty commit
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
