* [hooks] Support Flux2 in MagCache
MagCache could not be enabled on `Flux2Transformer2DModel` (Flux2 and Flux2 Klein):
1. `Flux2TransformerBlock` and `Flux2SingleTransformerBlock` were not registered in
`TransformerBlockRegistry`, so `apply_mag_cache` raised `ValueError: Model class ... not registered`.
2. Once registered, the tail hook raised a shape error. The head hook records the image-only input of
`transformer_blocks[0]`, but the Flux2 model concatenates the text and image streams before the
single-stream blocks and calls them with `encoder_hidden_states=None`, so the last single block returns
the fused `[text; image]` sequence. The residual `tail_output - head_input` could not be formed.
Register the two Flux2 blocks (the single block with `return_encoder_hidden_states_index=None`, matching
how the model calls it) and add `_align_to_hidden_states`, which keeps only the trailing image tokens of
the fused tensor when computing the residual. The cached residual is therefore image-only, which is what
the reference MagCache4FLUX implementation does, and it matches the head input on skipped steps. This also replaces
a dead "fallback" branch in the tail hook that subtracted mismatched shapes anyway.
Tests: `MagCacheTesterMixin` for `Flux2Transformer2DModel`, `Flux2Pipeline` and `Flux2KleinPipeline`.
FirstBlockCache has the same head/tail structure and still fails on Flux2; left for a follow-up.
* [hooks] Slice the fused tail output in the existing mismatch branch
Implement the Flux2 residual fix inside the tail hook's sequence-length mismatch branch instead of a
separate `_align_to_hidden_states` helper. The surrounding structure stays as it was: the `diff == 0`
case still covers a batch-size mismatch via broadcasting, and only the mismatched-length case changes
to subtract the trailing image tokens.
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
* feat: add torchtpu
* feat:draft TorchTPU support
* fix: wan overflow issue + compile mode error on sdxl
* doc: enhance with TorchTPU doc
* doc: enhance with TorchTPU doc
* style: remove unused imports flagged by ruff
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: remove Debug Eager and Fused Eager sections from tpu.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add TP for TPU
* style: fix import sorting in TPU test scripts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: ruff format TPU test scripts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: style
* fix: test for native 4 devices
* fix: propagate TPU device fixes to Flux/Flux2/Wan-family copies; register tpu.md; drop redundant execution_device check
- Propagate the text_encoder.device-based fix (introduced for TPU CPU-offload
support) from FluxPipeline/Flux2KleinPipeline/WanPipeline into their
`# Copied from` copies (flux/*, flux2_klein_inpaint, visualcloze, anyflow,
chronoedit, lucy_edit, skyreels_v2/*). SDXL-family copies of
StableDiffusionXLPipeline.encode_prompt are intentionally left untouched;
they'll be handled in a follow-up PR that fixes device placement for every
pipeline component (not just text encoders).
- Register docs/source/en/optimization/tpu.md in _toctree.yml (was breaking
the docs build: "not present in the table of contents").
- Remove the redundant "prefer non-CPU, non-meta component" loop from
DiffusionPipeline._execution_device: PR #14383 already fixed this in
DiffusionPipeline.device, which _execution_device falls back to. Verified
on TPU hardware that _execution_device still resolves correctly for a
split-placement pipeline after the removal.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* tests: cleanup
* fix: fix compile mode
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* test: remove flux2 e2e test
* doc: apply suggestions
* doc: apply suggestions
* review: remove monkey patch
* review: revert neuron-specific changes in the tests
* review: apply suggestions
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* doc: add tpu to tp doc
* review: remove unnecessary for tpu
* test: assert every _tp_plan parameter is sharded after a TP load
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* removal: delete redundant tp shard helpers
* removal: drop TPU workarounds no longer needed after recent main changes
* doc: keep the FLUX.2-dev text encoder off a single TPU chip in the TP example
* doc: encode the prompt on CPU in the TPU TP example so it runs end to end
* test: tighten TPU TP tolerance and simplify the TPU TP worker
* doc: use enable_model_cpu_offload() in the TPU eager example
* doc: shard both FLUX.2-dev text encoder and transformer in the TPU TP example
* doc: run the text encoders on TPU in the compiled example
* test: drop the redundant TPU sync in the TP worker
* test: trim comments in the TPU TP tests
* test: run the TPU TP test with mp.spawn like the CUDA one
* test: pin the TPU TP test to 4 chips
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* doc: use DistributedConfig and dtype, and drop the torch_tpu import in the TPU examples
* test: skip the TPU TP test for models without a _tp_plan
* doc: correct the warning of recompilation
* fix: clone the SD3 timestep so torch.compile doesn't recompile every step, thus we can use it for the tpu.md compile mode snippet
* doc: update advice for compile mode when one chip doesn't fit
* test: pass the TPU TP tolerances as test arguments
* review: apply suggestions to the doc
* review: view compilation patch for tpu-only
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: JingyaHuang <huang_jingya@outlok.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Add optional FP32 sampling state for Cosmos3
* Add cache execution contexts to Cosmos3 pipelines
* Add SeaCache support for Cosmos3
* Cap consecutive SeaCache reuses
* Align SeaCache defaults with Cosmos3 inference
* Enable SeaCache by default for Cosmos3
* Document SeaCache for Cosmos3
* Use FP32 sampling state by default for Cosmos3
* Fix SeaCache reconfiguration, transfer state isolation, and FP32 sampling
* Remove SeaCache stats and ablation instrumentation
* Move SeaCache to transformer-level and disable by default
* Align SeaCache and tests with model-level cache patterns
* Make Cosmos3 sampling state always FP32
* Add pipe definition to the docs snippet
* Remove redundant hook registry cache resets
* Use raw vision latents for SeaCache indicators
* Make Cosmos3 SeaCache safe for regional compilation
* Fix SeaCache with Cosmos3 model parallelism
* Support single-stream models in SeaCache
* Add Wan T2V support to SeaCache
* pass denoising step info to cache hooks via cache_context
Replaces SeaCache's pipeline-state callbacks: the pipeline attaches the loop
coordinates (step_index, sigma, num_inference_steps) to the denoiser call with
`cache_context(name, ...)`, carried by a frozen CacheContext dataclass and stored
on each stateful hook's StateManager (the registry no longer holds context).
Co-Authored-By: Dhruv Nair <dhruv.nair@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix Wan cache metadata and Cosmos modular cleanup
Pass denoising step metadata to Wan T2V conditional and unconditional cache contexts.
Restore stateful cache reset after Cosmos modular pipeline calls, restore the required Literal import, and update regression tests and documentation.
* Refactor Cosmos3 SeaCache decoder hook
* Apply CI formatting updates
* Fix distilled system prompt fallback
* Preserve Cosmos3 fp32 sampling state
---------
Co-authored-by: yiyixuxu <yixu310@gmail.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
docs: fix typos in docs, comments and docstrings
Prose, comments and docstrings only. No executable line changed.
docs/:
- optimization/memory.md: maxmium -> maximum; heigh -> height (in the
channels_last layout tuple, which read "(batch size, heigh, width,
channels)")
- api/pipelines/bria_fibo.md: proffesional -> professional
- api/pipelines/wan.md: involed -> involved
src/ comments and docstrings:
- guiders/magnitude_aware_guidance.py: supression -> suppression
- models/attention.py: spliting -> splitting
- models/attention_dispatch.py: abritrary -> arbitrary (x2)
- models/transformers/transformer_ltx2.py: timstamps -> timestamps (x2)
- pipelines/ltx2/pipeline_ltx2{,_image2video}.py: corrct -> correct
- pipelines/deprecated/unidiffuser/modeling_uvit.py: tbe -> the
- pipelines/visualcloze/pipeline_visualcloze_{combined,generation}.py:
"whe they are lists" -> "when they are lists" in a user-facing
ValueError message
Deliberately not touched:
- pipeline_kandinsky*.py "promt"/"scren" live inside prompt_template,
which is fed to the text encoder verbatim. Changing them would change
generation output.
- ANE (Apple Neural Engine), MoT (Mixture-of-Transformers): initialisms.
- "racoon": appears in prompt examples whose reference images were
generated with that exact prompt.
- "Shuting Wang": a paper author's name.
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* misc: update examples link
* misc: update examples link
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* docs: introduce cache-dit to diffusers
* Refine documentation for CacheDiT features
Updated the wording for clarity and consistency in the documentation. Adjusted sections on cache acceleration, automatic block adapter, patch functor, and hybrid cache configuration.
* Add Pruna optimization framework documentation
- Introduced a new section for Pruna in the table of contents.
- Added comprehensive documentation for Pruna, detailing its optimization techniques, installation instructions, and examples for optimizing and evaluating models
* Enhance Pruna documentation with image alt text and code block formatting
- Added alt text to images for better accessibility and context.
- Changed code block syntax from diff to python for improved clarity.
* Add installation section to Pruna documentation
- Introduced a new installation section in the Pruna documentation to guide users on how to install the framework.
- Enhanced the overall clarity and usability of the documentation for new users.
* Update pruna.md
* Update pruna.md
* Update Pruna documentation for model optimization and evaluation
- Changed section titles for consistency and clarity, from "Optimizing models" to "Optimize models" and "Evaluating and benchmarking optimized models" to "Evaluate and benchmark models".
- Enhanced descriptions to clarify the use of `diffusers` models and the evaluation process.
- Added a new example for evaluating standalone `diffusers` models.
- Updated references and links for better navigation within the documentation.
* Refactor Pruna documentation for clarity and consistency
- Removed outdated references to FLUX-juiced and streamlined the explanation of benchmarking.
- Enhanced the description of evaluating standalone `diffusers` models.
- Cleaned up code examples by removing unnecessary imports and comments for better readability.
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Enhance Pruna documentation with new examples and clarifications
- Added an image to illustrate the optimization process.
- Updated the explanation for sharing and loading optimized models on the Hugging Face Hub.
- Clarified the evaluation process for optimized models using the EvaluationAgent.
- Improved descriptions for defining metrics and evaluating standalone diffusers models.
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
