AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization

Assigning MoE expert precision from weight-spectrum heavy-tailedness, without calibration data for the bit-allocation stage.

Wanqi Yang1,2,3 Yuexiao Ma4 Alexander Conzelmann1,2 Xiawu Zheng4 Michael W. Mahoney5,6,7 T. Konstantin Rusch1,2,3,8 Shiwei Liu1,2,3
1Max Planck Institute for Intelligent Systems
2ELLIS Institute Tübingen | 3Tübingen AI Center
4Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Xiamen University
5International Computer Science Institute | 6Lawrence Berkeley National Laboratory
7University of California, Berkeley | 8Liquid AI
Paper Code BibTeX
AlphaQ workflow.

AlphaQ measures heavy-tailedness, scales quantization noise, and solves constrained bit allocation.

Core Contributions

Spectral Importance

PL_Alpha_Hill from expert weight spectra provides a data-free signal for quantization sensitivity.

Global Bit Optimization

A budget-constrained solver assigns mixed precision across experts and layers under matched bit budgets.

Calibration-Free Allocation

AlphaQ avoids calibration-set dependence when assigning bits, reducing domain bias in MoE quantization.

The Motivation

Calibration data can bias bit allocation

Existing data-driven methods estimate expert importance from calibration samples. For frontier MoE LLMs, the training distribution is usually inaccessible, so a calibration set can be an imperfect surrogate.

Domain-dependent allocation

Different calibration domains induce different bit-width patterns.

Domain-level skew

Allocation can overfit to the calibration domain and degrade unseen tasks.

AlphaQ removes this dependency

The bit-allocation signal comes directly from model parameters.

Calibration-domain bias in data-driven MoE bit allocation.

Domain bias introduced by data-driven bit-width allocation in Mixtral-8x7B.

The Method

Weight spectra as a data-free importance signal

Core Insight

Experts with more heavy-tailed weight spectra are treated as structurally more important and receive higher precision, while less sensitive experts can be quantized more aggressively.

Heavy-Tailed Expert Diversity

PL Alpha Hill distribution across MoE models.

PL_Alpha_Hill varies across blocks, layers, and MoE architectures.

Measure Spectra

Compute PL_Alpha_Hill for expert-layer weight matrices, optionally with FARMS sampling.

Scale Noise

Combine alpha-derived importance with bit-width-dependent quantization noise.

Solve Globally

Optimize bit assignment under a single global average-bit budget.

Empirical Validation

Experimental Results

Main MoE Quantization Results

Main AlphaQ result table.

Results on DeepSeekV2-Lite, Qwen1.5-MoE , and Mixtral-8x7B.

Speedup Tradeoff

Mixtral-8x7B

Accuracy-speedup tradeoff.

Accuracy versus inference speedup relative to BF16.

Memory Footprint

Mixtral and Qwen1.5-MoE

Memory footprint.

AlphaQ reduces expert-weight memory under low-bit budgets.

Key Takeaways

Calibration-free bit allocation avoids coupling MoE quantization to a chosen calibration distribution.

PL_Alpha_Hill provides a global signal for expert and layer heterogeneity across MoE architectures.

Global optimization improves low-bit quality under matched bit budgets while reducing memory footprint.

Cite This Work

@inproceedings{
yang2026alphaq,
title={AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization},
author={Wanqi Yang and Yuexiao Ma and Alexander Conzelmann and Xiawu Zheng and Michael W. Mahoney and T. Konstantin Rusch and Shiwei Liu},
booktitle={ICLR 2026 2nd Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy},
year={2026},
url={https://openreview.net/forum?id=rbE8Pxs8vx}
}