Are Guessing, Source Coding and Tasks Partitioning Birds of A Feather?

• Published in: Entropy (Basel, Switzerland) Vol. 24; no. 11; p. 1695
• Main Authors: Ashok Kumar, M., Sunny, Albert, Thakre, Ashish, Kumar, Ashisha, Dinesh Manohar, G.
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 19-11-2022; MDPI
• Subjects: Asymptotic properties; Codes; Coding; Entropy; Entropy (Information theory); guessing; Inequality; Information theory; Kullback–Leibler divergence; Optimization; Partitioning; Probability; Probability distribution; Random variables; Rényi entropy; Shannon entropy; source coding; tasks partitioning
• ISSN: 1099-4300; 1099-4300
• DOI: 10.3390/e24111695

This paper establishes a close relationship among the four information theoretic problems, namely Campbell source coding, Arikan guessing, Huleihel et al. memoryless guessing and Bunte and Lapidoth tasks’ partitioning problems in the IID-lossless case. We first show that the aforementioned problems are mathematically related via a general moment minimization problem whose optimum solution is given in terms of Renyi entropy. We then propose a general framework for the mismatched version of these problems and establish all the asymptotic results using this framework. The unified framework further enables us to study a variant of Bunte–Lapidoth’s tasks partitioning problem which is practically more appealing. In addition, this variant turns out to be a generalization of Arıkan’s guessing problem. Finally, with the help of this general framework, we establish an equivalence among all these problems, in the sense that, knowing an asymptotically optimal solution in one problem helps us find the same in all other problems.