quant-forge

ADR 0001: Cholesky Second-Order GPTQ (act_order) vs. Salient-Channel AWQ

Context

Emergent activation outliers in Transformer blocks (20x+ magnitude in <3% of channels) cause catastrophic output MSE degradation under naive Round-to-Nearest (RTN) 4-bit quantization.

Decision

We implement both MLSys 2024 Best Paper AWQ (grid-searched per-channel equivalence scaling $s = \bar{|X|}^\gamma$) and ICLR 2023 GPTQ using exact float64 Cholesky factorization of $H^{-1} = R^\top R$ paired with descending Hessian curvature ordering (act_order=True).

Consequences