Real-Time Coding With Limited Lookahead

• Published in: IEEE transactions on information theory Vol. 59; no. 6; pp. 3582 - 3606
• Main Authors: Asnani, Himanshu, Weissman, Tsachy
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01-06-2013; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Actions; Algorithms; Applied sciences; average cost optimality equation (ACOE); beliefs; Bellman equation; Bernoulli Hypothesis; Channel coding; Coding theory; Coding, codes; constrained Markov decision process; controlled Markov chains; Decoding; Equations; Exact sciences and technology; expected average distortion; finite-state decoders; Information theory; Information, signal and communications theory; Lagrangian; lookahead; Markov analysis; Markov processes; Mathematical model; optimal cost; policy; Real time; Real-time systems; side information; Signal and communications theory; Telecommunications and information theory; value iteration; vending machine; Markov process; Markov decision; side information; Feedback regulation; Iterative method; expected average distortion; Sequential decoding; finite-state decoders; Markov chain; Credal approach; Information source; vending machine; controlled Markov chains; policy; Availability; Binary channel; Hamming distance; Bellman equation; Lagrangian; constrained Markov decision process; Actions; optimal cost; Channel coding; Real time system; beliefs; average cost optimality equation (ACOE); lookahead; Memoryless channel; Causality; Memoryless source; value iteration
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2013.2245396

A real-time coding system with lookahead consists of a memoryless source, a memoryless channel, an encoder, which encodes the source symbols sequentially with knowledge of future source symbols up to a fixed finite lookahead d , with or without feedback of the past channel output symbols and a decoder, which sequentially constructs the source symbols using the channel output. The objective is to minimize the expected per-symbol distortion. For a fixed finite lookahead d\geq 1 , we invoke the theory of controlled Markov chains to obtain an average cost optimality equation (ACOE), the solution of which, denoted by D(d) , is the minimum expected per-symbol distortion. With increasing d , D(d) bridges the gap between causal encoding, d=0 , where symbol-by-symbol encoding-decoding is optimal and the infinite lookahead case, d=\infty , where Shannon Theoretic arguments show that separation is optimal. We extend the analysis to a system with finite-state decoders, with or without noise-free feedback. For a Bernoulli source and binary symmetric channel, under Hamming loss, we compute the optimal distortion for various source and channel parameters, and thus obtain computable bounds on D(d) . We also identify regions of source and channel parameters where symbol-by-symbol encoding-decoding is suboptimal. Finally, we demonstrate the wide applicability of our approach by applying it in additional coding scenarios, such as the case where the sequential decoder can take cost-constrained actions affecting the quality or availability of side information about the source.