mirror of https://github.com/huggingface/diffusers.git synced 2026-10-24 20:59:27 +03:00

Files

Álvaro Somoza 8b33b47332 [quantization] SDNQ core loading (#14277 )

* initial

* fix code quality

* dependency test fix

* apply review suggestions

* add extras and version check

* fix gating and prefer local cache for modular

* simplify transformers registration and make the version warning silenceable

* remove pipeline level test

* only model level testing

* layers to quantizze, change em dashes

* env flag for transformers

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

2026-07-30 13:09:11 +05:30

1.6 KiB

Raw Permalink Blame History

Quantization

Quantization techniques reduce memory and computational costs by representing weights and activations with lower-precision data types like 8-bit integers (int8). This enables loading larger models you normally wouldn't be able to fit into memory, and speeding up inference.

Tip

Learn how to quantize models in the Quantization guide.

PipelineQuantizationConfig

autodoc quantizers.PipelineQuantizationConfig

BitsAndBytesConfig

autodoc quantizers.quantization_config.BitsAndBytesConfig

GGUFQuantizationConfig

autodoc quantizers.quantization_config.GGUFQuantizationConfig

NunchakuLiteQuantizationConfig

autodoc quantizers.quantization_config.NunchakuLiteQuantizationConfig

QuantoConfig

autodoc quantizers.quantization_config.QuantoConfig

SDNQConfig

autodoc quantizers.quantization_config.SDNQConfig

TorchAoConfig

autodoc quantizers.quantization_config.TorchAoConfig

DiffusersQuantizer

autodoc quantizers.base.DiffusersQuantizer

1.6 KiB Raw Permalink Blame History