On the Optimality of Sufficient Statistics-based Quantizers


IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.45, no.3, pp.3567-3573, 2023 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 45 Issue: 3
  • Publication Date: 2023
  • Doi Number: 10.1109/tpami.2022.3172282
  • Journal Name: IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, PASCAL, ABI/INFORM, Aerospace Database, Applied Science & Technology Source, Business Source Elite, Business Source Premier, Communication Abstracts, Compendex, Computer & Applied Sciences, EMBASE, INSPEC, MEDLINE, Metadex, zbMATH, Civil Engineering Abstracts
  • Page Numbers: pp.3567-3573
  • Keywords: convex analysis, Fisher information, hypothesis testing, parameter estimation, Quantization
  • Hacettepe University Affiliated: Yes


IEEELet X be a random variable taking values in a set $\calX$, and let $\{P_{\theta}; \theta\in \Theta\}$ be a family of distributions indexed by the parameter vector $\theta$ taking values in a set $\Theta$. A quantized random variable $\gamma(X)$ is obtained by employing a quantizer $\gamma : \calX \rightarrow \{1,\ldots,K\}$. It is shown that any extreme point of the set of all possible probability distributions of $\gamma(X)$ can be achieved by a deterministic quantizer that decides based only on the sufficient statistics. Using this fact, optimality properties of deterministic sufficient statistics-based quantizers are established for the problem of parameter estimation. It is proven that there always exists an optimal partitioning of sufficient statistics into K convex polytopes which maximizes the trace of the Fisher information matrix when $\{P_{\theta}; \theta\in \Theta\}$ belongs to the exponential family. Furthermore, the optimality of likelihood ratio statistic for simple hypothesis testing follows as a consequence of this result, thereby demonstrating a link between parameter estimation and hypothesis testing.