analiza aiqbench
This research establishes a theoretical framework for optimal quantization that simultaneously minimizes mean squared error while enforcing specific output distribution constraints. By leveraging majorization theory, the derived method enables precise control over output entropy and mutual information, offering practical utility for communication channel matching and data anonymization. The findings provide a rigorous mathematical foundation for advancing information-theoretic limits in quantization tasks.
Minimum Distortion Quantization with Specified Output Distribution
arXiv:2606.10458v1 Announce Type: cross Abstract: We derive the optimal quantizer of a real-valued random variable $W$ with distribution $P_W$ such that 1) the distribution of the quantization output $X$ that can take $k$ values follows any specified distribution $P_X$ over $\{1,\ldots,k\}$, and 2) the minimum mean squared error (MMSE) of estimating $W$ from $X$ is minimized. It is shown that the optimal quantizer takes the form $X=\sigma\big(F_{\sigma^{-1}(X)}^{-1}(F_W(W))\big)$, where $\sigma$ is the optimal permutation of $\{1,\ldots,k\}$ among all permutations to minimize the MMSE, and $F$ is the cumulative distribution function. When $P_W$ is uniform over an interval or $P_X$ is uniform over $\{1,\ldots,k\}$, the quantizer takes a simple form $X=F_{X}^{-1}(F_W(W))$. The concept of majorization plays a key role in the optimality proof. Specifying the output distribution is useful for designing quantizers with explicitly controlled output entropy, maximized mutual information between input and output, tailored output distribution to match channel input requirements for communication, and data anonymization.
Abstract:We derive the optimal quantizer of a real-valued random variable $W$ with distribution $P_W$ such that 1) the distribution of the quantization output $X$ that can take $k$ values follows any specified distribution $P_X$ over $\{1,\ldots,k\}$, and 2) the minimum mean squared error (MMSE) of estimating $W$ from $X$ is minimized. It is shown that the optimal quantizer takes the form $X=\sigma\big(F_{\sigma^{-1}(X)}^{-1}(F_W(W))\big)$, where $\sigma$ is the optimal permutation of $\{1,\ldots,k\}$ among all permutations to minimize the MMSE, and $F$ is the cumulative distribution function. When $P_W$ is uniform over an interval or $P_X$ is uniform over $\{1,\ldots,k\}$, the quantizer takes a simple form $X=F_{X}^{-1}(F_W(W))$. The concept of majorization plays a key role in the optimality proof. Specifying the output distribution is useful for designing quantizers with explicitly controlled output entropy, maximized mutual information between input and output, tailored output distribution to match channel input requirements for communication, and data anonymization.
Information Theory (cs.IT); Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Statistics Theory (math.ST)
arXiv:2606.10458 [cs.IT]
(or arXiv:2606.10458v1 [cs.IT] for this version)
https://doi.org/10.48550/arXiv.2606.10458 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 254 cuvinte
Subiecte conexe
InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
arXiv · 22 oct. 2026
Test-Time Adaptation of Quantized ViTs via Single-Pass Quantizer-Aligned Recalibration
arXiv · 7 oct. 2026
RDQ: Residual Distribution Quantization for Large Language Models
arXiv · 29 iul. 2026
SoloQ: Calibration-Free Quantization for Diffusion Language Models
arXiv · 7 oct. 2026