Universal codes for finite sequences of integers drawn from a monotone distribution

• Published in: IEEE transactions on information theory Vol. 48; no. 6; pp. 1713 - 1720
• Main Authors: Foster, D.P., Stine, R.A., Wyner, A.J.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-06-2002; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Alphabets; Codes; Entropy; Inequalities; Information; Information theory; Integers; Mathematical analysis; Memoryless systems; Minimax technique; Redundancy; Sequences; Statistics; Theory
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2002.1003851

We offer two noiseless codes for blocks of integers X/sup n/ = (X/sub 1/, ..., X/sub n/). We provide explicit bounds on the relative redundancy that are valid for any distribution F in the class of memoryless sources with a possibly infinite alphabet whose marginal distribution is monotone. Specifically, we show that the expected code length L (X/sup n/) of our first universal code is dominated by a linear function of the entropy of X/sup n/. Further, we present a second universal code that is efficient in that its length is bounded by nH/sub F/ + o(nH/sub F/), where H/sub F/ is the entropy of F which is allowed to vary with n. Since these bounds hold for any n and any monotone F we are able to show that our codes are strongly minimax with respect to relative redundancy (as defined by Elias (1975)). Our proofs make use of the elegant inequality due to Aaron Wyner (1972).