Local-Optimality Guarantees for Optimal Decoding Based on Paths Nissim Halabi Guy Even School of Electrical Engineering, Tel-Aviv University August 29, 2012 1/23
Communication Over a Noisy Channel u ∈ { 0 , 1 } k Channel c ∈ C ⊂ { 0 , 1 } N λ ( y ) ∈ R N Channel ˆ c ∈ { 0 , 1 } N Noisy Channel noisy codeword Encoder Decoder codeword u ∈ { 0 , 1 } k ˆ MBIOS channel: memoryless, binary-input, output-symmetric Log-Likelihood-Ratio (LLR): � Pr( y i | c i = 0) � λ i ( y i ) � ln Pr( y i | c i = 1) Linear Code: C ⊆ { 0 , 1 } N is subspace of F N 2 of dimension k . Optimal decoding: Maximum Likelihood decoding. Input: y . Output: ml ( y ) . ml ( y ) � arg max Pr { y | c = x } x ∈C = arg min � λ ( y ) , x � x ∈C 2/23
Tanner Codes Defined by Tanner Graphs G = ( V ∪ J , E ) x 1 v 1 Tanner code C ( G, C J ) represented x 2 v 2 by bipartite graph C 1 C 1 x ∈ C ( G, C J ) iff x ∈ C j for every x 3 v 3 j ∈ { 1 , . . . , J } C 2 C 2 x 4 v 4 In general: x 5 v 5 C 3 C 3 degrees: can be regular, irregular, x 6 v 6 bounded, or arbitrary can allow arbitrary linear local C 4 C 4 x 7 v 7 codes x 8 v 8 C 5 C 5 Examples: LDPC codes x 9 v 9 [Gallager’63], Expander codes x 10 v 10 [Sipser-Spielman’96] V J Variable Nodes Local-Code Nodes 3/23
Linear Programming (LP) Decoding conv( X ) ⊆ R N - the convex hull a set of points X ⊆ R N . ML-decoding can be rephrased: ml ( y ) � arg min � λ ( y ) , x � x ∈ conv( C ) Generalized fundamental polytope of a Tanner code C ( G, C J ) - relaxation of conv( C ) [following Feldman-Wainwright-Karger’05] P ( G, C J ) � � conv( C j ) C j ∈C J LP-decoding: lp ( y ) � arg min � λ ( y ) , x � x ∈P ( G, C J ) 4/23
LP Decoding with ML Certificate LP-decode ( λ ) x lp ← arg min x ∈P ( G, C J ) � λ, x � . solve LP: ˆ x lp ∈ { 0 , 1 } N then if ˆ x lp is an ML codeword return ˆ else return fail end if Polynomial time algorithm Applies to any MBIOS channel! Integral solution ⇒ ML-certificate 5/23
Goal: Analysis of Finite Length Codes Problem (Finite Length Analysis) Design: Constant rate code C ( G, C J ) and an efficient decoding algorithm dec . Analyze: If SNR > t , then Pr ( dec ( λ ) � = x | c = x ) � exp ( − N α ) for some 0 < α . Goal: Minimize t (lower bound on SNR). Remarks: Not an asymptotic problem Code is not chosen randomly from an ensemble Successful decoding � = ML decoding 6/23
Unified Analysis Framework via Local-Optimality Advances in analysis of finite-length codes via local-optimality: [Koetter-Vontobel’06], [Arora-Daskalakis-Steurer’09], [H-Even’10], [Vontobel’10], [H-Even’11] Based on complicated combinatorial structures embedded in the Tanner graph of the codes and non-trivial analyses of random processes Today Demonstrate the proof technique - use simple characterization of local-optimality based on paths Simpler proofs obtained via local-optimality based on paths for the case of repeat-accumulate codes [Feldman-Karger’02], [Goldenberg-Burshtein’11] 7/23
Certificate for ML-Optimality / LP-Optimality Problem (Optimality Certificate) Input: Channel observation λ and a codeword x ∈ C Question 1: Is x ML-optimal with respect to λ ? is it unique? (NP-Hard) Question 2: Is x LP-optimal with respect to λ ? is it unique? Relax: one-sided error test A positive answer = certificate for the optimality of x w.r.t. λ A negative answer = don’t know if optimal or not (allow one sided error) “Local-Optimality” criterion: efficient test via local computations 8/23
Definition of Local-Optimality [Feldman’03] For x ∈ { 0 , 1 } N and f ∈ [0 , 1] N ⊆ R N , define the relative point x ⊕ f by ( x ⊕ f ) i � | x i − f i | Consider a finite set of “deviations” � B ⊂ [0 , 1] N Definition (following [Arora-Daskalakis-Steurer’09]) A codeword x ∈ C is locally-optimal w.r.t. λ ∈ R N if for all vectors β ∈ B , � λ, x ⊕ β � > � λ, x � Goal Find a set B of locally-structured devia- λ tions such that: ML ( λ ) 1 x ∈ lo ( λ ) ⇒ x = ml ( λ ) & unique LP ( λ ) 2 x ∈ lo ( λ ) ⇒ x = lp ( λ ) & unique LO ( λ ) 3 Pr λ { x ∈ lo ( λ ) | c = x } = 1 − o (1) 9/23
Even Tanner Codes Definition (Even Tanner codes) Variables nodes have even degree All local codewords have even weight Example LDPC codes with even left degrees Irregular repeat accumulate codes where the repetition factors are even Expander codes with even variable node degrees and even weighted local codes 10/23
Deviations Based on Paths p is a path of length h : h can be greater than girth, p may be not simple Each path p defines a “characteristic” vector χ G ( p ) ∈ R N 1 � { v | v ∈ p } [ χ G ( p )] v � � � deg G ( v ) · � . B ( h ) ⊂ [0 , 1] N is the set of deviations � χ G ( p ) � � B ( h ) � � � p is a backtrackless path of length h � h + 1 Example a p = ( a, X, b, Z, a, Y, c, Z, b ) X b h = 8 Y χ G ( p ) = { 2 3 , 2 3 , 1 c 3 , 0 } Z χ G ( p ) h +1 = { 2 27 , 2 27 , 1 27 , 0 } d 11/23
Local-Optimality based on h -Paths Set of deviations B ( h ) = normalized characteristic vectors of h -paths. � χ G ( p ) � � B ( h ) � � � p is a backtrackless path of length h � h + 1 Definition A codeword x ∈ C is h -locally optimal w.r.t. λ ∈ R N if for all vectors β ∈ B ( h ) , � λ, x ⊕ β � > � λ, x � 12/23
Local Optimality ⇒ Unique ML-codeword Theorem If x is h -locally optimal w.r.t. λ , then x is the unique ML-codeword w.r.t. λ . Proof method: Lemma (Decomposition Lemma) Every codeword is a conical combination of h -paths in G x = α · E β ∈ ρ B ( h ) [ β ] Proof of decomposition lemma Every codeword is a conical combination of simple cycles in G 1 Every cycle is a conical combination h -paths in G 2 Following [ADS’09]: decomposition lemma ⇒ unique ML 13/23
Verifying Local Optimality Hard: Is x the unique ML-codeword? Easy: Is x is locally optimal? Codeword can be efficiently verified to be locally-optimal w.r.t. λ (dynamic programming / ∼ Floyd’s algorithm) 14/23
Local Optimality ⇒ Unique LP optimality Theorem If x is a h -locally optimal codeword w.r.t. λ , then x is also the unique optimal LP solution given λ . Proof method: reduction to “ML” using graph covers. z ∗ = ML � λ ↑ M � M -Covering Graph ˜ In graph covers, realization of LP-Opt and ML codeword are the same [Vontobel-Koetter’05] z ∗ = LP Opt. Base Graph 15/23
Local Optimality ⇒ Unique LP optimality Theorem If x is a h -locally optimal codeword w.r.t. λ , then x is also the unique optimal LP solution given λ . Proof method: reduction to “ML” using graph covers. x � x ↑ M is locally-optimal w.r.t. λ ↑ M M -Covering Graph ˜ Lemma: Local-optimality is invariant w.r.t. lifting to covering graphs Base Graph x is locally-optimal w.r.t. λ 16/23
Local Optimality ⇒ Unique LP optimality Theorem If x is a h -locally optimal codeword w.r.t. λ , then x is also the unique optimal LP solution given λ . Proof method: reduction to “ML” using graph covers. Thm: Local-Opt ⇒ ML Opt. z ∗ = ML x � x ↑ M is locally-optimal w.r.t. λ ↑ M � λ ↑ M � = M -Covering Graph ˜ ˜ Base Graph 17/23
Local Optimality ⇒ Unique LP optimality Theorem If x is a h -locally optimal codeword w.r.t. λ , then x is also the unique optimal LP solution given λ . Proof method: reduction to “ML” using graph covers. Thm: Local-Opt ⇒ ML Opt. z ∗ = ML x � x ↑ M is locally-optimal w.r.t. λ ↑ M � λ ↑ M � = M -Covering Graph ˜ ˜ Lemma: Local-optimality is invariant [Vontobel-Koetter’05] w.r.t. lifting to covering graphs z ∗ = LP Opt. = Base Graph x is locally-optimal w.r.t. λ 18/23
Probabilistic Analysis Symmetry of local-optimality implies: ∃ β ∈ B ( h ) s . t . � λ, β � � 0 � � c = 0 N � Pr { LP decoding fails } � Pr � . Let D � d max · d max L R Bounds rely on: girth ( G ) > log D ( N ) . [Gallager’63] gives an explicit construction of such graphs. Theorem Consider BSC with crossover probability p . d min ) · ( ǫ + 3 2 + 1 − 2 · (1+ L 2 log D (2)) , then d max For every ǫ > 0 , if p < D L Pr { lp ( λ ) � = x | c = x } � N − ǫ Analogous theorem derived for the BI-AWGN channel Obtain same results as in [Feldman-Karger’02], [Goldenberg-Burshtein’11] for RA (2) and RA (2 q ) . 19/23
Local-Optimality - Proof Technique The proof technique in [KV’06], [ADS’09], [HE’10], [HE’11] is based on the following steps: Define a set of deviations. A deviation is induced by 1 combinatorial structures in the Tanner graph or the computation tree. Define local-optimality. Loosely speaking, a codeword x is 2 locally-optimal if its cost is smaller than the cost of every relative point. Local-optimality ⇒ Unique ML-codeword. 3 Decomposition Lemma: Every codeword is a conical sum of deviations. Local-optimality ⇒ Unique LP-codeword. 4 Lifting Lemma: Local-optimality is invariant under liftings of codewords to covering graphs. Analyze the probability that there does not exist a 5 locally-optimal codeword. 20/23
Recommend
More recommend
