UmerHA
|
fda1531d8a
|
Fixing implementation of ControlNet-XS (#6772)
* CheckIn - created DownSubBlocks
* Added extra channels, implemented subblock fwd
* Fixed connection sizes
* checkin
* Removed iter, next in forward
* Models for SD21 & SDXL run through
* Added back pipelines, cleared up connections
* Cleaned up connection creation
* added debug logs
* updated logs
* logs: added input loading
* Update umer_debug_logger.py
* log: Loading hint
* Update umer_debug_logger.py
* added logs
* Changed debug logging
* debug: added more logs
* Fixed num_norm_groups
* Debug: Logging all of SDXL input
* Update umer_debug_logger.py
* debug: updated logs
* checkim
* Readded tests
* Removed debug logs
* Fixed Slow Tests
* Added value ckecks | Updated model_cpu_offload_seq
* accelerate-offloading works ; fast tests work
* Made unet & addon explicit in controlnet
* Updated slow tests
* Added dtype/device to ControlNetXS
* Filled in test model paths
* Added image_encoder/feature_extractor to XL pipe
* Fixed fast tests
* Added comments and docstrings
* Fixed copies
* Added docs ; Updates slow tests
* Moved changes to UNetMidBlock2DCrossAttn
* tiny cleanups
* Removed stray prints
* Removed ip adapters + freeU
- Removed ip adapters + freeU as they don't make sense for ControlNet-XS
- Fixed imports of UNet components
* Fixed test_save_load_float16
* Make style, quality, fix-copies
* Changed loading/saving API for ControlNetXS
- Changed loading/saving API for ControlNetXS
- other small fixes
* Removed ControlNet-XS from research examples
* Make style, quality, fix-copies
* Small fixes
- deleted ControlNetXSModel.init_original
- added time_embedding_mix to StableDiffusionControlNetXSPipeline .from_pretrained / StableDiffusionXLControlNetXSPipeline.from_pretrained
- fixed copy hints
* checkin May 11 '23
* CheckIn Mar 12 '24
* Fixed tests for SD
* Added tests for UNetControlNetXSModel
* Fixed SDXL tests
* cleanup
* Delete Pipfile
* CheckIn Mar 20
Started replacing sub blocks by `ControlNetXSCrossAttnDownBlock2D` and `ControlNetXSCrossAttnUplock2D`
* check-in Mar 23
* checkin 24 Mar
* Created init for UNetCnxs and CnxsAddon
* CheckIn
* Made from_modules, from_unet and no_control work
* make style,quality,fix-copies & small changes
* Fixed freezing
* Added gradient ckpt'ing; fixed tests
* Fix slow tests(+compile) ; clear naming confusion
* Don't create UNet in init ; removed class_emb
* Incorporated review feedback
- Deleted get_base_pipeline / get_controlnet_addon for pipes
- Pipes inherit from StableDiffusionXLPipeline
- Made module dicts for cnxs-addon's down/mid/up classes
- Added support for qkv fusion and freeU
* Make style, quality, fix-copies
* Implemented review feedback
* Removed compatibility check for vae/ctrl embedding
* make style, quality, fix-copies
* Delete Pipfile
* Integrated review feedback
- Importing ControlNetConditioningEmbedding now
- get_down/mid/up_block_addon now outside class
- renamed `do_control` to `apply_control`
* Reduced size of test tensors
For this, added `norm_num_groups` as parameter everywhere
* Renamed cnxs-`Addon` to cnxs-`Adapter`
- `ControlNetXSAddon` -> `ControlNetXSAdapter`
- `ControlNetXSAddonDownBlockComponents` -> `DownBlockControlNetXSAdapter`, and similarly for mid/up
- `get_mid_block_addon` -> `get_mid_block_adapter`, and similarly for mid/up
* Fixed save_pretrained/from_pretrained bug
* Removed redundant code
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
|
2024-04-16 21:56:20 +05:30 |
|
UmerHA
|
e192ae08d3
|
Add ControlNet-XS support (#5827)
* Check in 23-10-05
* check-in 23-10-06
* check-in 23-10-07 2pm
* check-in 23-10-08
* check-in 231009T1200
* check-in 230109
* checkin 231010
* init + forward run
* checkin
* checkin
* ControlNetXSModel is now saveable+loadable
* Forward works
* checkin
* Pipeline works with `no_control=True`
* checkin
* debug: save intermediate outputs of resnet
* checkin
* Understood time error + fixed connection error
* checkin
* checkin 231106T1600
* turned off detailled debug prints
* time debug logs
* small fix
* Separated control_scale for connections/time
* simplified debug logging
* Full denoising works with control scale = 0
* aligned logs
* Added control_attention_head_dim param
* Passing n_heads instead of dim_head into ctrl unet
* Fixed ctrl midblock bug
* Cleanup
* Fixed time dtype bug
* checkin
* 1. from_unet, 2. base passed, 3. all unet params
* checkin
* Finished docstrings
* cleanup
* make style
* checkin
* more tests pass
* Fixed tests
* removed debug logs
* make style + quality
* make fix-copies
* fixed documentation
* added cnxs to doc toc
* added control start/end param
* Update controlnetxs_sdxl.md
* tried to fix copies..
* Fixed norm_num_groups in from_unet
* added sdxl-depth test
* created SD2.1 controlnet-xs pipeline
* re-added debug logs
* Adjusting group norm ; readded logs
* Added debug log statements
* removed debug logs ; started tests for sd2.1
* updated sd21 tests
* fixed tests
* fixed tests
* slightly increased error tolerance for 1 test
* make style & quality
* Added docs for CNXS-SD
* make fix-copies
* Fixed sd compile test ; fixed gradient ckpointing
* vae downs = cnxs conditioning downs; removed guess
* make style & quality
* Fixed tests
* fixed test
* Incorporated review feedback
* simplified control model surgery
* fixed tests & make style / quality
* Updated docs; deleted pip & cursor files
* Rolled back minimal change to resnet
* Update resnet.py
* Update resnet.py
* Update src/diffusers/models/controlnetxs.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Update src/diffusers/models/controlnetxs.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Incorporated review feedback
* Update docs/source/en/api/pipelines/controlnetxs_sdxl.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/api/pipelines/controlnetxs.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/api/pipelines/controlnetxs.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/api/pipelines/controlnetxs.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update src/diffusers/models/controlnetxs.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update src/diffusers/models/controlnetxs.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update src/diffusers/pipelines/controlnet_xs/pipeline_controlnet_xs.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/api/pipelines/controlnetxs.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update src/diffusers/pipelines/controlnet_xs/pipeline_controlnet_xs_sd_xl.py
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Incorporated doc feedback
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
|
2023-12-06 23:33:47 +01:00 |
|
