Multiple Description Coding with Many Channels Raman Venkataramani Harvard University, Cambridge, MA raman@deas.harvard.edu DIMACS workshop on network information theory Joint work with Gerhard Kramer (Bell Labs) and Vivek Goyal (Digital Fountain)
DIMACS 2003 Packet Networks Packets Packet loss J 1 J 2 Source E D Output J 3 Encoder Decoder Model : Packets are either lost completely or received error-free . 1
DIMACS 2003 How do we deal with packet loses? • Request a retransmission. – Good for loss-less transmission. – Not feasible for real time data such as voice and video . Alternate Approach : • Reconstruct using available packets. – Requires adding redundancy to packets ( coding ). – Advantage : Graceful degradation of output quality when packet losses increase. The second approach is called Multiple Description Coding . 2
DIMACS 2003 Example: Coding with 3 Descriptions Descriptions (packets) Packet loss J 1 J 2 ˆ X N X N E D J 3 IID Source Encoder Decoder • The source is IID vector X N (length N ). • The encoder produces L “descriptions” J 1 ,. . . , J L of X N . X N from the available descriptions. • The decoder produces an output ˆ 3
DIMACS 2003 Side decoder J 1 X N D 1 1 X N D 12 12 Source Encoder Side decoder J 2 X N X N E D 2 2 Channel X N D 13 13 Side decoder J 3 X N D 3 3 X N D 23 23 Central decoder X N D 123 123 | {z } 4 Decoder
DIMACS 2003 Review of Rate-Distortion Theory J ˆ X N X N E D achievable Source Encoder Decoder Distortion D 1 N H ( J ) ≤ R not N d ( X N , ˆ X N ) ≤ D 1 achievable Rate R The RD region is the convex set R ≥ R ( D ) Theorem 1. [Shannon] where I ( X ; ˆ E d ( X, ˆ R ( D ) = min X ) X ) ≤ D s.t. ˆ X minimized over all ˆ X jointly distributed with X . Gaussian Source X ∼ N (0 , 1) : R ( D ) = 1 2 log( 1 D ) 5
DIMACS 2003 Multiple Description (MD) coding Source: Length N vector X N of i.i.d. random variables. Encoder: X N → { J 1 , . . . , J L } which are the L “descriptions” of X N at rates R 1 ,. . . R L per source symbol. J l = f l ( X N ) , H ( J l ) ≤ NR l , Descriptions: l = 1 , . . . L . Decoder: Consists of 2 L − 1 decoders: one for each non-empty subset of the available descriptions. Decoder Outputs: X N S = g S ( { J l : l ∈ S} ) where S ⊆ { 1 , . . . , L } , S � = ∅ . In the last example ( L = 3 ): S = { 1 } , { 2 } , { 3 } , { 1 , 2 } , { 1 , 3 } , { 2 , 3 } , or { 1 , 2 , 3 } 6
DIMACS 2003 Problem Statement • Problem: What is the Rate-Distortion (RD) region? • The rates ( L parameters) are R 1 , . . . , R L . • The distortions ( (2 L − 1) parameters) are D S = 1 N E d ( X N , X N S ) , S ⊆ { 1 , . . . , L } , S � = ∅ • The RD region is ( L + 2 L − 1) -dimensional. • Remark: For L = 1 , it is Shannon’s RD region. 7
DIMACS 2003 L = 2 case The RD region is the set of possible rates and distortions as N → ∞ : N E d ( X N , X N R 1 = 1 D 1 = 1 N H ( J 1 ) 1 ) N E d ( X N , X N R 2 = 1 D 2 = 1 N H ( J 1 ) 2 ) N E d ( X N , X N D 12 = 1 12 ) X N D 1 1 J 1 X N X N E D 12 12 J 2 Source Encoder D 2 X N 2 Decoders Outputs 8
DIMACS 2003 Review of Past Research El Gamal and Cover (1982) found an achievable region for L = 2 : R 2 R 1 ≥ I ( X ; X 1 ) achievable R 2 ≥ I ( X ; X 2 ) R 1 + R 2 ≥ I ( X ; X 1 X 2 X 12 ) + I ( X 1 ; X 2 ) D S ≥ E d ( X, X S ) , S = 1 , 2 , 12 0 R 1 where X 1 , X 2 , X 12 are any r.v’s jointly Rate region for fixed X 1 , X 2 , X 12 distributed with the source X . such that D S ≥ E d ( X, X S ) . Remark 1: The convex hull of this region is achievable by time-sharing. Remark 2: Gives the RD region for the Gaussian source . 9
DIMACS 2003 • Ozarow (1980) computed an outer bound on the RD region for L = 2 for the Gaussian source. The bound meets the inner bound by El Gamal and Cover. • Zhang and Berger (1987) provided a stronger achievable result than El Gamal and Cover for L = 2 . For the binary symmetric source with Hamming distortion measure, their result provides a strict improvement . • Wolf, Wyner and Ziv (1980), Witsenhausen and Wyner (1981), Zhang and Berger (1983) provided some results for the binary symmetric source. 10
DIMACS 2003 An Achievable Region for L > 2 Theorem 2. The RD region contains the rates and distortions satisfying � R l ≥ ( |S| − 1) I ( X ; X ∅ ) − H ( X U : U ∈ 2 S | X ) l ∈S H ( X T | X U : U ∈ 2 T − T ) � + T ⊆S D S ≥ E d S ( X, X S ) for every ∅ � = S ⊆ L = { 1 , . . . , L } and some joint distribution between outputs { X S } and the source X . Remark: This result generalizes the results of El Gamal and Cover , and of Zhang and Berger . 11
DIMACS 2003 Gaussian Source: Outer Bound on the RD Region • Gaussian source : X ∼ N (0 , 1) . • Squared-error distortion : d ( x, y ) = | x − y | 2 . • An outer bound on the RD region: Theorem 3. The RD region is contained in � M � � � m =1 ( D K m + λ ) � � ∀K ∈ 2 L − 2 ≤ exp R k min inf D K , ( D K + λ )(1 + λ ) M − 1 λ ≥ 0 {K m } M m =1 k ∈K minimized over all partitions {K m } of K . 12
DIMACS 2003 Special Case: L Channels and L + 1 Decoders X N D 1 1 X N D 2 2 Outputs : X N 1 , X N 2 ,. . . , X N L and X N 0 = X N 12 ...L X N E Rates : R 1 , R 2 ,. . . , R L Source Encoder Distortions : D 1 , D 2 ,. . . , D L X N D L L and D 0 X N 0 = X N D 0 12 ...L Decoders Keep only side and central decoders. Ignore all other decoders. 13
DIMACS 2003 Inner and Outer Bounds on the RD region • Inner Bound: Computable from our achievable region (Theorem 2). • Outer Bound: Compuatable from Theorem 3 for the Gaussian source • Tightness of Bounds: The inner and outer bounds meet for over some range of rates and distortions for the Gaussian source. 14
DIMACS 2003 Example: 3-Channel 4-Decoder Problem Take L = 3 , D 1 = D 2 = D 3 = 1 / 2 and D 0 = 1 / 16 . Outer Bound : R l ≥ 0 . 5 , l = 1 , 2 , 3 Outer bound R 3 R 1 + R 2 + R 3 ≥ 2 . 1755 Provably Achievable Achievable Rates : R l ≥ 0 . 5 , l = 1 , 2 , 3 R 1 + R 2 + R 3 = 2 . 1755 R 2 0 R l + R m ≥ 1 . 1258 , l < m Rate region for D 1 = D 2 = D 3 = 1 / 2 and D 0 = 1 / 16 R 1 Remark: Excess rate = 2 . 1755 − R ( D 0 ) = 0 . 1755 bits. 15
DIMACS 2003 1.3 1.2 1.1 R 3 bits/symbol 1 0.9 0.8 0.7 0.6 0.5 0.5 1.5 1 1 1.5 0.5 R 2 bits/symbol R 1 bits/symbol Blue Region: Inner and outer bounds meet on a hexagon. Green Region: Inner bound (does not meet outer bound). 16
DIMACS 2003 Another Example: L = 3 , D 1 = D 2 = 1 / 2 , D 3 = 3 / 4 , and D 0 = 1 / 16 : 1.6 1.4 1.2 R 3 bits/symbol 1 0.8 0.6 0.4 0.2 0.5 1.5 1 1 1.5 0.5 R 2 bits/symbol R 1 bits/symbol The RD Region 17
DIMACS 2003 Gaussian Sources are Successively Refinable Chains : R 1 ≥ R ( D 1 ) R 1 + R 2 ≥ R ( D 12 ) R 1 R 2 R 3 X X 1 X 2 X 123 . . . R 1 + R 2 + R 3 ≥ R ( D 123 ) Gaussian Trees : R 1 ≥ R ( D 1 ) X 123 R 3 R 1 + R 2 ≥ R ( D 12 ) X 2 . . . R 2 R 1 R 4 R 1 + R 2 + R 3 ≥ R ( D 123 ) X X 1 X 124 Gaussian R 5 R 1 + R 2 + R 4 ≥ R ( D 124 ) X 15 R 1 + R 5 ≥ R ( D 15 ) 18
DIMACS 2003 Summary • Multiple Description Coding : – Motivation: coding for packet networks. • Results : – An achievable region – An outer bound for the Gaussian source. • Still unsolved : – The Gaussian problem for L ≥ 3 – Non-Gaussian sources 19
Recommend
More recommend
