* fix(torchao): route root-level tensors around safetensors reconstruction
Models with parameters at the root of the module tree (e.g. Wan's
`scale_shift_table`) crashed torchao's `unflatten_tensor_state_dict`
when loading serialized checkpoints, since flattened tensor names are
assumed to carry a module prefix. Filter such tensors and their
metadata entries out of the reconstruction and merge them back
unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(bnb): defer 8-bit weights split from their SCB stats across shards
Sharded serialization can place an 8-bit weight and its `SCB`
statistics in different shard files, in which case the shard-by-shard
loader failed with "Missing quantization component `SCB`". Hold the
incomplete half of the pair back until its counterpart arrives with a
later shard, mirroring the torchao pending mechanism, and disable
parallel shard loading for prequantized 8-bit checkpoints.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [tests] extend model-level quantization tester mixins
Migrate remaining model-level coverage from tests/quantization into the
tester mixins so it runs for every wired model:
- base: buffer-placement assertions in the device-map test, and an
opt-in sharded-serialization test enabled by setting
`sharded_serialization_config`
- bnb: serialization across all configs (sharded included), dtype
assignment and adapter training for 8-bit, device moves preserving
the memory footprint, corrupted-state-dict loading error, and a fixed
modules-to-not-convert test (BitsAndBytesConfig only exposes
llm_int8_skip_modules; the old test passed an unsupported kwarg and
only survived by being skipped)
- torchao: custom device maps with cpu/disk offload, generalized from
the Flux-specific test
- gguf: the diffusers-format single-file loading path, wired into the
Flux model tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [tests] fix model-level quantization test wiring
- SD3.5: the quantized testers reused the random-init dummy inputs
(4 latent channels, fp32) while the tiny Hub checkpoint has
in_channels=8 and the quantizers load the model in half precision;
give them matching inputs and relax the 4-bit memory expectation for
the tiny checkpoint.
- QwenImage / Flux2: the quantized testers had no Hub checkpoint wired
at all, so every test errored; point them at
hf-internal-testing/tiny-qwenimage-pipe and tiny-flux2 with matching
inputs.
- NucleusMoE: no tiny checkpoint exists on the Hub yet; comment the
testers out like the LTX ones.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [tests] split tests/quantization into pipeline-level and backend-level tiers
Pipeline-level quantization tests (pipeline quality slices, cpu offload,
LoRA loading, compile, PipelineQuantizationConfig) move to
tests/pipelines/testing_utils/quantization.py, marked per backend so the
nightly CI can select them with `pytest -m`. tests/quantization keeps
only backend-level tests that fit neither tier: config validation,
utility warnings, and GGUF CUDA kernel correctness. Tests already
covered by the model-level mixins are dropped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [ci] select nightly quantization tests by marker across test tiers
Each backend job now runs `pytest -m <marker>` over tests/models,
tests/quantization, and tests/pipelines/testing_utils/quantization.py,
giving the model-level mixin tests a nightly home with the backend
dependencies installed. The torchao job additionally installs mslk.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: sayak@huggingface.co <sayak@ip-10-53-90-45.ec2.internal>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* initial
* fix code quality
* dependency test fix
* apply review suggestions
* add extras and version check
* fix gating and prefer local cache for modular
* simplify transformers registration and make the version warning silenceable
* remove pipeline level test
* only model level testing
* layers to quantizze, change em dashes
* env flag for transformers
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Fix training gradient underflow in quantization tests
Change autocast dtype from float16 to bfloat16 in _test_quantization_training.
Float16's limited dynamic range causes gradients to underflow to zero when
passing through quantized tensor subclass operations.
* fix autocast dtype check
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
---------
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* remove str option for quantization config in torchao
* Apply style fixes
* minor fixes
* Added AOBaseConfig docs to torchao.md
* minor fixes for removing str option torchao
* minor change to add back int and uint check
* minor fixes
* minor fixes to tests
* Update tests/quantization/torchao/test_torchao.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Update docs/source/en/quantization/torchao.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Update tests/quantization/torchao/test_torchao.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* version=2 update to test_torchao.py
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* support device type device_maps to work with offloading.
* add tests.
* fix tests
* skip tests where it's not supported.
* empty
* up
* up
* fix allegro.
