mirror of https://github.com/huggingface/diffusers.git synced 2026-10-29 11:12:00 +03:00

Files

Vinh H. Pham 60ec6f7242 Add Nunchaku Lite single-file quantization (#14100 )

* Add Nunchaku Lite single-file quantization

* Support config-backed Nunchaku Lite loading

* Remove Nunchaku runtime manifest metadata loading

* Simplify Nunchaku compact config loading

* Add Nunchaku Lite quantization tests

* Document Nunchaku Lite checkpoint loading

* Refine Nunchaku Lite quantization docs

* Remove unused Nunchaku smooth factor original weights

* Update docs/source/en/quantization/nunchaku.md

Co-authored-by: sergereview[bot] <283583894+sergereview[bot]@users.noreply.github.com>

* Update src/diffusers/quantizers/nunchaku/utils.py

Co-authored-by: sergereview[bot] <283583894+sergereview[bot]@users.noreply.github.com>

* Update src/diffusers/quantizers/nunchaku/nunchaku_quantizer.py

Co-authored-by: sergereview[bot] <283583894+sergereview[bot]@users.noreply.github.com>

* Address Nunchaku Lite review feedback

* Update Nunchaku Lite quantizer

* Rename Nunchaku Lite fp4 precision to nvfp4

* Allow torch_dtype to override Nunchaku compute dtype

* Keep Nunchaku replacement modules on meta during loading

* Require kernels when importing Nunchaku runtime utils

* Use kernels requirement decorators for Nunchaku tests

* Validate Nunchaku CUDA capability requirements

* Apply suggestions from code review

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* docs: update nunchaku quantization guide

* test: remove nunchaku quantization tests

* fix: reject hopper for nunchaku lite

* Apply suggestion from @rootonchair

* docs: update nunchaku lite compile example

* Apply style fixes

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Vinh H. Pham  <phamvinh257@gmail.com>

* update doc to be more reasonable

* fix import that requires kernels

---------

Co-authored-by: sergereview[bot] <283583894+sergereview[bot]@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

2026-07-14 10:09:08 +05:30

1.5 KiB

Raw Blame History

Quantization

Quantization techniques reduce memory and computational costs by representing weights and activations with lower-precision data types like 8-bit integers (int8). This enables loading larger models you normally wouldn't be able to fit into memory, and speeding up inference.

Tip

Learn how to quantize models in the Quantization guide.

PipelineQuantizationConfig

autodoc quantizers.PipelineQuantizationConfig

BitsAndBytesConfig

autodoc quantizers.quantization_config.BitsAndBytesConfig

GGUFQuantizationConfig

autodoc quantizers.quantization_config.GGUFQuantizationConfig

NunchakuLiteQuantizationConfig

autodoc quantizers.quantization_config.NunchakuLiteQuantizationConfig

QuantoConfig

autodoc quantizers.quantization_config.QuantoConfig

TorchAoConfig

autodoc quantizers.quantization_config.TorchAoConfig

DiffusersQuantizer

autodoc quantizers.base.DiffusersQuantizer

1.5 KiB Raw Blame History