The blog post announces native integration of Nunchaku 4-bit diffusion inference into Hugging Face Diffusers.

Nunchaku is based on SVDQuant, a quantization method that runs transformer layers with 4-bit weights and activations (W4A4), reducing memory and speeding up denoising.

Previously, Nunchaku checkpoints required a separate inference library; now they load with from_pretrained() and require no local CUDA compilation.

The post introduces Nunchaku Lite, a new integration path in Diffusers that patches nn.

Linear modules with runtime SVDQ/AWQ layers, using kernels from the Hub via the kernels package.

Two kernel families are used: svdq_w4a4 (for attention and MLP projections, in INT4 or NVFP4 precision) and awq_w4a16 (for modulation layers). NVFP4 requires Blackwell GPUs (RTX 50 series), while INT4 works on Turing/Ampere/Ada.

Nunchaku Lite delivers roughly 30% speedup and up to 50% VRAM reduction compared to BF16, though less than the original Nunchaku engine due to lack of architecture-specific fused kernels.

Benchmarks on an RTX PRO 6000 with ERNIE-Image-Turbo show: BF16 baseline 3.00 s full pipeline, 31.1 GB VRAM; Nunchaku Lite NVFP4 2.27 s, 20.6 GB (1.35x speedup); with torch.compile 1.68 s (1.8x).

Quantizing text encoders with bitsandbytes NF4 further reduces VRAM.

The post also explains how to quantize new models using diffuse-compressor: inspect, calibrate, package as a Diffusers pipeline, and push to Hub. It notes limitations: the generic path cannot handle structural rewrites (e.g.

, fusing QKV projections) that the original Nunchaku engine uses; such rewrites require model-specific target configs and runtime adapters. Ready-to-use checkpoints are available on the Hub.