* tests: split block/leaf group offloading and derive the offloaded components
`GroupOffloadTesterMixin.test_group_offloading_inference` ran both offload
levels in one test body, so a pipeline that failed at one level had to skip
both. Split it into `test_group_offloading_inference_block_level` and
`test_group_offloading_inference_leaf_level`, sharing the helpers the test
body used to define inline, and compare against the class-scoped
`base_pipe_output` rather than rebuilding a baseline per level.
The set of components to offload was a hardcoded list of eight names, so a
pipeline with a component under any other name had it silently left on CPU
for the forward pass to trip over. Derive the set instead: every `nn.Module`
component is offloaded unless the config lists it in
`group_offloading_leaf_level_exclude_modules`, the new
`group_offloading_exclude_modules`, or `group_offloading_onload_component_names`.
A name in either exclusion list that matches no component on the pipeline
fails as the typo it is.
Ideogram4's `unconditional_transformer` was one of the silently dropped
components, which is why its group offload test was skipped; it now needs no
declaration at all, and `text_encoder` picks up block-level coverage it never
had. LTX2's skip goes the same way, with `audio_vae` declared alongside the
other VAEs the tests keep on the accelerator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* style: fix `create_pipe` indentation in the group offload tester
The method body was indented one level too deep, which `ruff format` rewrites.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* tests: give group offloading one exclusion list per level
`group_offloading_onload_component_names` named an implementation detail —
components the tests keep resident — and justified itself with a mechanism
that cannot occur: lazy prefetch mis-tracing a tiled VAE decode needs
`use_stream=True`, which no test on this mixin sets, and most of the
pipelines it covers never enable tiling either.
What it actually encodes is a level-specific incapability, the mirror of
`group_offloading_leaf_level_exclude_modules`. Leaf-level offloading onloads
each leaf on its own `forward`, so it breaks on compute that reads a leaf's
`.weight` directly. Block-level onloads a group when the group's leader runs
its `forward`, so it breaks on compute that re-enters submodules without
going through that leader — which is what a VAE decode path does. Rename it
to `group_offloading_block_level_exclude_modules` and scope it to the level
it describes. `group_offloading_exclude_modules` is then redundant, since a
component that fails at both levels goes in both lists, and it never had a
user. `vqvae` leaves the default: no pipeline in the new harness has one.
Measured across the suite, offloading the `vae` at block level breaks 28 of
86 test classes, while all 86 offload it at leaf level with no failures --
coverage the component-scoped tests were skipping and the pipeline-level
test has always had. Sweep before and after: the same 9 pre-existing
failures, 782 -> 810 passing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Move group offloading changes to shared LTX-2 test mixins
* tests: cover the group offloading stream prefetch path
`use_stream=True` is the only way into `LazyPrefetchGroupOffloadingHook`,
which traces the group execution order on the first forward pass, wires each
group to its successor and flips `non_blocking` for the passes after. Nothing
in the pipeline suite reached that code, which is how a stale comment about
streams mis-tracing a tiled VAE decode survived unexamined for so long.
Add `test_group_offloading_inference_block_level_streaming` and
`..._leaf_level_streaming` alongside the synchronous pair, as separate methods
rather than a parametrized test so a pipeline can skip one level without
losing the other. Both route through `_run_group_offload_inference`, so they
compare against the same `base_pipe_output` and honour the same per-level
exclusion lists.
`apply_group_offloading` raises rather than degrading when there is nowhere to
put a stream, and `@require_torch_accelerator` still passes on MPS, so the two
tests also check for CUDA or XPU before running.
Guard against the flag quietly ceasing to have an effect: after enabling
offloading with `use_stream=True`, assert at least one group across the
pipeline actually carries a stream. The count is pipeline-wide because a
component with no `ModuleList`/`Sequential` children has nothing to prefetch
and legitimately streams nothing, and it asserts rather than skips because a
regression that drops `use_stream` would zero every count and a skip would
turn the whole streaming suite green by omission.
Sweep over `tests/pipelines -k group_offloading`: 31 failed, 1145 passed,
identical to the run without these tests. All 13 streaming failures have a
non-streaming twin that fails the same way, so the prefetch path adds no
failure of its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* tests: port the group offload overrides orphaned by the level split
Splitting `test_group_offloading_inference` into a block-level and a
leaf-level test removed the name six other test files override, so their
markers guarded a method the mixin no longer defines while the real tests ran
unprotected. The `xfail(strict=True)` ones hid it: `super()` raised
`AttributeError`, which counts as the expected failure, so they reported green
while testing nothing.
Rechecking each reason rather than renaming mechanically, three of the six
turned out to describe behaviour that no longer exists:
`LTX2DFRPipelineTesterConfig` subclasses `BasePipelineTesterConfig` rather
than `LTX2BaseTesterConfig`, so the family's `audio_vae` block-level exclusion
never reached it and its audio VAE was being offloaded. It needs the
declaration, not a skip, and its group offload tests now pass.
AudioLDM2 and Motif Video only fail at block level. `get_text_features()` and
the vision tower bypass the `forward` that block-level offloading gates the
group on, while leaf level onloads each leaf on its own `forward` and passes,
so the skip and the xfail are scoped to block level and the leaf-level tests
now run.
The Kandinsky prior xfails stay at both levels but their reason was describing
the hardcoded component list this branch removed. The actual cause is that the
pipeline calls `PriorTransformer.post_process_latents()` after the denoising
loop, reading `clip_mean` / `clip_std` that group offloading onloads only for
the duration of `forward`.
Also drop the class-count measurement from the block-level exclusion comment:
it was already stale one merge later. The mechanism it was evidence for —
`vae.decode()` never running `vae.forward()` — is the durable part.
Sweep over `tests/pipelines -k group_offloading`: 31 failed -> 13, all of them
failing identically on `main`, except `kolors`'s pipeline-level test, which
fails in isolation on `main` too and only passes there because of the test
order the split changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Clarify the Ideogram 4 block-level group offloading module exclusion comment
* tests: parametrize `use_stream` instead of splitting the streaming tests
The streaming tests were separate methods so a pipeline could skip or xfail
them independently of their synchronous twins. Nothing has ever needed that:
across three full sweeps every streaming failure had a synchronous twin at the
same level and vice versa, and none of the sixteen downstream overrides marked
a streaming variant differently from its twin. The two *levels* do get scoped
apart — AudioLDM2 and Motif Video are block-level only — so those keep their
own methods.
Fold `use_stream` into a parameter on each level's test. Four mixin tests
become two and sixteen override methods become eight, with the same coverage:
a full sweep of the parametrized and the separate-method versions on the same
tree gives 15 failed, 1263 passed, 152 skipped, 22 xfailed, 1 xpassed on both,
with identical failure sets once `[stream]` and the `_streaming` suffix are
normalised.
Marks do not carry through a subclass override, so an override that forgets to
re-declare the parametrize collapses to one un-parametrized test that errors on
the missing argument — which an `xfail` then reports as green, the same failure
mode the orphaned overrides had. `_USE_STREAM` carries a note saying so, and the
six overrides re-declare it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* tests: point the PNDM marker comment at the split group offload tests
`test_group_offloading_inference` no longer exists — it became one test per
offload level — so the comment listing which tests share the class-level xfail
named a test that is not there, and undercounted them by one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: record the group offload streaming parameter in the testing guide
The shared memory mixin now runs each offload level with and without
`use_stream`, which is the only path that reaches group offloading's lazy
prefetch. Say so next to the exclusion-list guidance, along with the reason it
is a parameter rather than a separate test, and the requirement that a subclass
overriding either test re-declare the `parametrize` — marks do not carry
through an override, and forgetting leaves a test that errors on the missing
argument and is reported as green by an `xfail`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* tests: inline the single-caller group offload helpers
`_split_group_offload_components` and `_count_streamed_groups` each had one
caller, so per `code_style.md` they read better at the call site. The reasons
they existed survive as comments: that a component is offloaded unless the
level's list names it, and that several submodules share one `ModuleGroup` so
the stream count has to dedupe on group identity.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: condense the group offload guidance to one bullet
The three group offload bullets in the testing guide had grown to explain
the hook mechanism at a level of detail that answers a reviewer's questions
rather than telling a test author what to write. Collapse them into one
bullet keeping the parts that change what a PR does: which remedy to reach
for, which exclusion list matches which hazard, that an unmatched name fails
as a typo, and that an override must re-declare the `use_stream` parametrize.
The worked examples below the bullet already carry the concrete diagnosis,
and the parametrize trap is explained in full at the code site.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Explicitly declares group offloading leaf level excluded components for LTX-2
* Simplify comment regarding _USE_STREAM in GroupOffloadTesterMixin
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* refactor c* pipeline tests.
Migrate the remaining `tests/pipelines/` directories starting with C to the config class +
composable pytest mixins introduced in #14113: chroma, chronoedit, cogview3, consisid,
consistency_models, controlnet, controlnet_flux, controlnet_hunyuandit, controlnet_sd3 and
cosmos. cogvideo and cogview4 were already migrated.
Notable points beyond the mechanical conversion:
- Expected slices switch from the `"np"` layout (`[0, -3:, -3:, -1]`) to the `"pt"` one
(`[0, -1, -3:, -3:]`); the values themselves are unchanged.
- The Cosmos `safety_checker` overrides (save/load, variants, dtype dict, encode_prompt)
were duplicated across five files and now live in a shared
`CosmosSafetyCheckerTesterMixin` in `tests/pipelines/cosmos/testing_utils.py`.
- chroma and controlnet_flux import `FluxIPAdapterTesterMixin` from `..flux.testing_utils`
rather than the old `..test_pipelines_common` one.
- `PipelineTesterMixin.test_save_load_float16` now passes the non-deprecated `dtype=` to
`from_pretrained` instead of `torch_dtype=`, which the Cosmos2.5 `from_pretrained`
wrappers read when casting the safety checker they inject.
- Dropped along the way, matching #14619: `PipelineLatentTesterMixin`,
`PipelineKarrasSchedulerTesterMixin`, the attention-slicing and xformers tests, and the
`test_ip_adapter` overrides that only supplied an `expected_pipe_slice`.
`pytest tests/pipelines/{chroma,chronoedit,cogview3,consisid,consistency_models,controlnet,controlnet_flux,controlnet_hunyuandit,controlnet_sd3,cosmos}`
gives 14 failed, 1014 passed, 82 skipped. All 14 failures are in cosmos and reproduce
unchanged on main: `DummyCosmosSafetyChecker` is a parameter-less `ModelMixin` with a single
non-persistent buffer, so accelerate does not hook it for sequential offload and a reloaded
pipeline gets a fresh fp32 one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor a* pipeline tests.
* [tests] fix expected slice in the sparsectrl dict/tuple test
The `test_dict_tuple_outputs_equivalent` override ported in e10f8d08e carried a
nine-value expected slice that is not the one the pre-migration file used, so CI
failed on CPU with `Shape mismatch: actual torch.Size([6]) vs expected
torch.Size([9])`.
Six is correct: the base test slices `output_tuple[0, -3:, -3:, -1, -1]` and this
pipeline's dummy inputs use `num_frames=2`, so the frame axis yields 2 rather
than 3. Restored the original values, unchanged since #14120.
The literal is only read when `torch_device == "cpu"`, which is why local GPU
runs did not catch it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor zimage and krea2 pipeline tests
* remove group offloading skip and propaget the refactors to the rest of zimage
* add negative prompt
---------
Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>
* skip hunyuan framepack group offloading tests
* exclude only the SigLIP image encoder from leaf-level group offloading
`test_pipeline_level_group_offloading_inference` was skipped outright for
HunyuanVideoFramepack because `image_encoder` is a `SiglipVisionModel`, whose
attention pooling head wraps a `torch.nn.MultiheadAttention`. That hands
`self.out_proj.weight` to `torch.nn.functional.multi_head_attention_forward`
instead of calling `self.out_proj`, so the leaf-level onload hook on `out_proj`
never fires and its weights stay on the offload device.
Add a `group_offloading_leaf_level_exclude_modules` knob to the old-style
`PipelineTesterMixin` and the new-style `BasePipelineTesterConfig` (empty by
default, so no behavior change elsewhere), pass it through to
`enable_group_offload(exclude_modules=...)` in both implementations of the test,
and set it to `["image_encoder"]` for framepack instead of skipping. Block-level
offloading is unaffected — the whole head is onloaded as one unmatched module —
hence the level in the name.
The test now passes and covers leaf-level offloading of the transformer, VAE and
both text encoders. The VAE is coverage nothing else provided:
`test_group_offloading_inference` deliberately excludes `vae` and
`image_encoder`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: note how to handle a component that breaks leaf-level offloading
Records the rule behind `group_offloading_leaf_level_exclude_modules`: leaf-level
offloading hooks only the supported leaf types and onloads each on its own
forward, so any code that reads a leaf's `.weight` instead of calling the leaf
bypasses that hook.
Routes the fix by who owns the component. A diffusers model declares the gap with
`_supports_group_offloading = False` on the `ModelMixin` subclass, which both
offload mixins honor. A third-party component that can't be annotated goes in
`group_offloading_leaf_level_exclude_modules`, which keeps offload coverage for
every other component — where a hand-written skip would drop it for the whole
pipeline, the VAE included, since the component-scoped
`test_group_offloading_inference` deliberately excludes it.
`torch.nn.MultiheadAttention` is called out as the common instance rather than as
the definition, with `HunyuanDiTAttentionPool` as a case that fails the same way
with no MHA module involved, so the guidance still applies when a future
component fails for a different reason.
Also notes that a failure should be reproduced before a skip or exclusion is
added: of the five pipelines currently skipping the pipeline-level test, only
framepack and motif_video still fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop old style test info and simplify leaf level group offloading testing.md note.
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(torchao): route root-level tensors around safetensors reconstruction
Models with parameters at the root of the module tree (e.g. Wan's
`scale_shift_table`) crashed torchao's `unflatten_tensor_state_dict`
when loading serialized checkpoints, since flattened tensor names are
assumed to carry a module prefix. Filter such tensors and their
metadata entries out of the reconstruction and merge them back
unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(bnb): defer 8-bit weights split from their SCB stats across shards
Sharded serialization can place an 8-bit weight and its `SCB`
statistics in different shard files, in which case the shard-by-shard
loader failed with "Missing quantization component `SCB`". Hold the
incomplete half of the pair back until its counterpart arrives with a
later shard, mirroring the torchao pending mechanism, and disable
parallel shard loading for prequantized 8-bit checkpoints.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [tests] extend model-level quantization tester mixins
Migrate remaining model-level coverage from tests/quantization into the
tester mixins so it runs for every wired model:
- base: buffer-placement assertions in the device-map test, and an
opt-in sharded-serialization test enabled by setting
`sharded_serialization_config`
- bnb: serialization across all configs (sharded included), dtype
assignment and adapter training for 8-bit, device moves preserving
the memory footprint, corrupted-state-dict loading error, and a fixed
modules-to-not-convert test (BitsAndBytesConfig only exposes
llm_int8_skip_modules; the old test passed an unsupported kwarg and
only survived by being skipped)
- torchao: custom device maps with cpu/disk offload, generalized from
the Flux-specific test
- gguf: the diffusers-format single-file loading path, wired into the
Flux model tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [tests] fix model-level quantization test wiring
- SD3.5: the quantized testers reused the random-init dummy inputs
(4 latent channels, fp32) while the tiny Hub checkpoint has
in_channels=8 and the quantizers load the model in half precision;
give them matching inputs and relax the 4-bit memory expectation for
the tiny checkpoint.
- QwenImage / Flux2: the quantized testers had no Hub checkpoint wired
at all, so every test errored; point them at
hf-internal-testing/tiny-qwenimage-pipe and tiny-flux2 with matching
inputs.
- NucleusMoE: no tiny checkpoint exists on the Hub yet; comment the
testers out like the LTX ones.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [tests] split tests/quantization into pipeline-level and backend-level tiers
Pipeline-level quantization tests (pipeline quality slices, cpu offload,
LoRA loading, compile, PipelineQuantizationConfig) move to
tests/pipelines/testing_utils/quantization.py, marked per backend so the
nightly CI can select them with `pytest -m`. tests/quantization keeps
only backend-level tests that fit neither tier: config validation,
utility warnings, and GGUF CUDA kernel correctness. Tests already
covered by the model-level mixins are dropped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [ci] select nightly quantization tests by marker across test tiers
Each backend job now runs `pytest -m <marker>` over tests/models,
tests/quantization, and tests/pipelines/testing_utils/quantization.py,
giving the model-level mixin tests a nightly home with the backend
dependencies installed. The torchao job additionally installs mslk.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: sayak@huggingface.co <sayak@ip-10-53-90-45.ec2.internal>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Fix Wan and Motif video pipeline fast test failures
Description
This PR fixes several failing fast tests for Motif Video and Wan pipelines.
Changes
Updated test_motif_video.py to use a Motif-specific fp16 save/load tolerance.
T5Gemma2Encoder rebuilds non-persistent RoPE buffers after save/load, which introduces small fp16 numerical drift. The test now keeps coverage while allowing this expected drift.
Updated transformer_wan.py to keep Wan RoPE modules in fp32 during mixed precision handling.
This avoids fp16 precision drift in Wan save/load paths.
Updated memory.py group offloading coverage to include transformer_2 and skip None optional components.
This fixes Wan 2.2 memory/offloading tests where optional secondary transformers may be absent.
Updated stale expected slices in test_wan_image_to_video.py for:
TestWanImageToVideoPipeline::test_inference
TestWanFLFToVideoPipeline::test_inference
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
* revert change in test_wan_image_to_video.py
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
---------
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* up
* up
* quantization level refactors.
* style
* up
* up
* use mixins and remove stale attributes
* fixture for the deprecated pipelines
* remove qkv
* remove lergacy for attention processor.
* use pytorch tensors exclusively.
* rejig dodgy test.
* improve how base pipeline output fixture is used.
* rename params.
* up
* use pipeline load api for ip adapter.
* move to flux specific.
* redesign config for caching
* remove eval.
* up
* up
* add a todo note on check_qkv_fusion_matches_attn_procs_length
* nan test in layerwise casting
* proper skipping
* skip properly.
* up
* up
