Fixed Delay Joint Source Channel Coding for Finite Memory Systems Aditya Mahajan and Demosthenis Teneketzis Dept. of EECS , University of Michigan, Ann Arbor, MI–48109 ISIT 2006–July 13, 2006
Fixed Delay & Fixed Complexity
Motivation I • Classical Information Theory does not take delay and com- plexity into account. S I • Why consider delay and complexity? T • Delay: 2 − QoS (end--to--end delay) in communication networks 0 − Control over communication channels. 0 − Decentralized detection in sensor networks. 6 • Complexity: (size of lookup table) − cost − power consumption 1 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Finite Delay Communication I • Separation Theorem: distortion d is feasible is S Rate Distortion of Source < Channel Capacity I R ( d ) < C T • For finite delay system Separation Theorem does not hold. 2 0 • What is equivalent of rate distortion and channel capacity? 0 • Find a metric to check whether distortion level d is feasible 6 or not. • Metric will depend on the source and the channel. 2 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Problem Formulation I Objective S Evaluate optimal performance R − 1 ( C ) for the simplest non-- I trivial system T − Markov Source 2 − memoryless noisy channel 0 − additive distortion 0 Constraints 6 − Use stationary encoding and decoding schemes. − Fixed memory available at the encoder and the decoder. 3 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Model I • Markov Source: S − Source Output { X 1 , X 2 , . . . } , X n ∈ X . I − Transition probability matrix P T • Finite State Encoder: 2 − Input X n , State S n , Output Z n 0 Z n = f ( X n , S n − 1 ) , Z n ∈ Z 0 S n = h ( X n , S n − 1 ) , S n ∈ S 6 • Memoryless Channel: � � Z n , Y n − 1 � � = Pr ( Y n | Z n ) = Q ( Y n , Z n ) Pr Y n 4 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Model I • Finite State Decoder: S − Input Y n , State M n , Output � X n I T � � X n = g ( Y n , M n − 1 ) , X n ∈ X M n = h ( Y n , M n − 1 ) , M n ∈ M 2 0 • Distortion Metric: 0 ρ : X × X → [ 0, K ] , K < ∞ 6 • D step delay ρ ( X n − D , � X n ) 5 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Problem Formulation I S n − 1 M n − 1 � S X n Z n Y n X n Markov Finite State Memoryless Finite State Source Encoder Channel Decoder I P f, h Q g, l T Problem (P1) 2 Given source ( X , P ) , channel ( Z , Y , Q ) , memory ( S , M ) and dis- 0 tortion ( ρ, D ) , determine encoder ( f, h ) and decoder ( g, l ) so as 0 to minimize 6 � � N � � � � 1 � � X n − D , � J ( f, h, g, l ) � lim sup E ρ X n � f, h, g, l � � N N →∞ n = D + 1 where � N = N − D + 1 6 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Literature Overview I • Transmitting Markovian source through finite--state ma- chines as encoders and decoders. S I • Problem considered by Gaarder and Slepian in mid 70 ’s. T • N. T. Gaarder and D. Slepain 2 On optimal finite--state digital communication systems, ISIT , Grignano, Italy, 1979 0 TIT , vol. 28 , no. 2 , pp. 167 – 186 , 1982 . 0 6 7 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Our Approach I • Start with a simpler (to analyze) problem − Finite horizon − time--varying design S − zero delay I T • dynamic team problem—solved using Stochastic Optimiza- tion Techniques 2 • finite delay problem 0 • infinite horizon problem 0 6 • Find conditions under which time invariant (stationary) de- signs are optimal. • Low complexity algorithms to obtain optimal performance and optimal design. 8 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Finite Horizon Problem
Finite Horizon Case — Model I • Encoder and Tx Memory Update S Z n = f n ( X n , S n − 1 ) f � ( f 1 , . . . , f N ) I S n = h n ( X n , S n − 1 ) h � ( h 1 , . . . , h N ) T • Decoder and Rx Memory Update 2 � X n = g n ( Y n , M n − 1 ) g � ( g 1 , . . . , g N ) 0 M n = l n ( Y n , M n − 1 ) l � ( l 1 , . . . , l N ) 0 6 • Delay D = 0 9 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Finite Horizon Problem Formulation I S n − 1 M n − 1 � S Markov X n Finite State Z n Memoryless Y n Finite State X n Source Encoder Channel Decoder I P f, h Q g, l T Problem (P2) 2 Given source ( X , P ) , channel ( Z , Y , Q ) , memory ( S , M ) , distor- 0 tion ( ρ, D = 0 ) and horizon N , determine encoder ( f, h ) and 0 decoder ( g, l ) so as to minimize � 6 � � N � � � � � X n , � J N ( f, h, g, l ) � E ρ X n � f, h, g, l � n = 1 where f � ( f 1 , . . . , f N ) , and so on for h, g, l . 10 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Solution Concept in Seq. Stoch. Opt I • One Step Optimization � S � � N � � � ρ ( X n , � � f N , h N , g N , l N I X n ) min E � f 1 ,f 2 ,...,f N T n = 1 h 1 ,h 2 ,...,h N g 1 ,g 2 ,...,g N l 1 ,l 2 ,...,l N 2 • 4N Step Optimization—Sequential Decomposition 0 0 � � � � min min min min · · · g 1 6 f 1 l 1 h 1 � � ���� � � � �� · · · min min min min · · · f N g N l N h N 11 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Dynamic Team Problems I • Team Decision Theory: distributed agents with common objective S I − Marshak and Radner T − Witsenhausen 2 • Decentralized of information—encoder and decoder have different view of the world. 0 0 • Non--classical information pattern 6 • Non--convex functional optimization problem • Most important step is identifying information state 12 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Information State If ϕ n − 1 is the information state at n − (and γ = ( f, h, g, l ) ) I S • State in the sense of I T n − 1 ( γ n ) T n ( γ n + 1 ) − → ϕ n − 1 − − − − − − → ϕ n − − − − − − → ϕ n + 1 − → T • Absorbs the effect of past decision rules on future perfor- 2 mance. 0 � � � N � 0 � � ρ ( X i , � � γ N E X i ) � 1 6 i = n � � � N � � � ρ ( X i , � � π 0 n − 1 , γ N = E X i ) � n i = n 13 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Find an information state for Problem (P2)
Find an information state for Problem (P2) Guess & Verify
Information State for (P2) I • Definition S π 1 n � Pr ( X n , Y n , S n − 1 , M n − 1 ) I π 2 n � Pr ( X n , S n − 1 , M n ) T π 0 n � Pr ( X n , M n ) S n , 2 0 0 6 14 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Information State for (P2) I • Definition S π 1 n � Pr ( X n , Y n , S n − 1 , M n − 1 ) I π 2 n � Pr ( X n , S n − 1 , M n ) T π 0 n � Pr ( X n , S n , M n ) 2 Lemma 0 0 For all n = 1, . . . , N , 6 there exist linear transforms T 0 , T 1 , T 2 such that • T 0 n − 1 ( f n ) T 1 T 2 n ( l n ) n ( h n ) → π 0 → π 1 → π 2 → π 0 − − − − − − − − − − − − − − − − − n − → n − 1 n n 14 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Information State for (P2) I • Definition S π 1 n � Pr ( X n , Y n , S n − 1 , M n − 1 ) I π 2 n � Pr ( X n , S n − 1 , M n ) T π 0 n � Pr ( X n , M n ) S n , 2 Lemma (cont. . .) 0 0 For all n = 1, . . . , N , 6 • the expected instantaneous cost can be written as � � � � f n , h n , g n , l n � ρ ( X n , � ρ ( π 1 = � E X n ) n , g n ) 14 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Solution of (P2) I Dynamic Program S • For n = 1, . . . , N I � � � T V 0 n − 1 ( π 0 V 1 T 0 ( f n ) π 0 � n − 1 ) = min , n n − 1 f n � � � V 1 n ( π 1 n ) = V n ( π 1 � V 2 T 1 ( l n ) π 1 2 n ) + min , n n l n 0 V n ( π 1 ρ ( π 1 � � � n ) = min n , g n ) , 0 g n � � � 6 V 2 n ( π 2 � V 0 T 2 ( h n ) π 2 n ) = min , n n h n and V 0 N ( π 0 N ) � 0. 15 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Solution of (P2) I • The arg min at each step determines the corresponding op- timal design rule. S I • The optimal performance is given by T N = V 0 0 ( π 0 J ∗ 0 ) 2 • Computations: Numerical methods from Markov decision 0 theory can be used. 0 6 16 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Next steps . . .
Finite Delay Problem I • Delay D � = 0 S • Sliding window transformation of the source I T X n = ( X n − D , . . . , X n ) ρ ( X n , � X n ) = ρ ( X n − D , � X n ) 2 0 • Reduces to problem (P2). 0 6 17 Mahajan Teneketzis: Fixed Delay Joint Source Channel Coding
Recommend
More recommend
