On the Construction of Polar Codes for Channels with Moderate Input - PowerPoint PPT Presentation

On the Construction of Polar Codes for Channels with Moderate Input Alphabet Sizes Ido Tal 1 / 19

Problem: Construction of polar (LDPC) codes, for a channel with moderate input alphabet size q . Say, q ≥ 16. Punchline: Provably hard ∗†‡§ . ∗ For a specific channel † under a certain construction model ‡ deterministically § some more assumptions 2 / 19

Given: ◮ Underlying channel W : X → Y und ◮ |X| = q ◮ Uniform input distribution is capacity achieving ◮ Codeword length n = 2 m Goal: ◮ Assuming uniform input, calculate misdecoding probability of synthesized channels W ( m ) : X → Y i , 0 ≤ i < n i ◮ Unfreeze channels with very low probability of misdecoding 3 / 19

P U ( X ) � uniform distribution on input alphabet X Algorithm: Naive solution input : Underlying channel W , index i = � b 1 , b 2 , . . . , b m � 2 output : P e ( W ( m ) , P U ( X ) ) i W ← W for j = 1 , 2 , . . . , m do if b j = 0 then W ← W − else W ← W + return P e (W , P U ( X ) ) Problem: Y i grows exponentially with n . 4 / 19

P U ( X ) � uniform distribution on input alphabet X Algorithm: Degrading solution input : Underlying channel W , index i = � b 1 , b 2 , . . . , b m � 2 , bound on output alphabet size L output : Upper bound on P e ( W ( m ) , P U ( X ) ) i Q ← degrading merge ( W , L , P U ( X ) ) for j = 1 , 2 , . . . , m do if b j = 0 then W ← Q − else W ← Q + Q ← degrading merge (W , L , P U ( X ) ) return P e (Q , P U ( X ) ) Question: How good of an approximation to W is degrading merge (W , L , P U ( X ) )? 5 / 19

Notation: ◮ W : X → Y — generic memoryless channel ◮ q = |X| — input alphabet size ◮ P X — input distribution ◮ Q : X → Y ′ — degraded version of W ◮ L — bound on new output alphabet size, |Y ′ | ≤ L ◮ X — input to W or Q ◮ Y — output of W ◮ Y ′ — output of Q Goal: degrading merge ( W , L , P X ) must find Q : X → Y ′ such that ◮ Q degraded with respect to W ◮ |Y ′ | ≤ L ◮ ∆ = I ( X ; Y ) − I ( X ; Y ′ ) is “small” 6 / 19

An implementation of degrading merge ( W , L , P X ) exists [TalSharovVardy] for which �� 1 � 1 / q � ∆ = I ( X ; Y ) − I ( X ; Y ′ ) ≤ O L Apropos: similar behaviour in upgraded case [PeregTal] Totally useless (at least in theory), for moderate q : L ≈ 10 32 q = 16 , ∆ ≤ 0 . 01 = ⇒ Good luck. . . 7 / 19

An inherent difficulty? What can be said about DC ( q , L ) � sup min ( I ( W ) − I ( Q )) . Q : Q ≺ W , W , P X | out ( Q ) |≤ L We already know that �� 1 � 1 / q � DC ( q , L ) ≤ O L Need: a lower bound on DC ( q , L ) 8 / 19

Cut to the end DC ( q , L ) � sup min ( I ( W ) − I ( Q )) Q : Q ≺ W , W , P X | out ( Q ) |≤ L We will shortly prove that 2 �� 1 � � q − 1 DC ≥ O L Above attained for ◮ Uniform input distribution P X = P U ( X ) ◮ Sequence W 1 , W 2 , . . . of “progressively hard channels” ◮ The capacity achieving input distribution of each W M is the uniform distribution P U ( X ) 9 / 19

Consequences: Try and build a polar code for W M . . . Algorithm: Degrading solution input : Underlying channel W , index i = � b 1 , b 2 , . . . , b m � 2 , bound on output alphabet size L output : Upper bound on P e ( W ( m ) , P U ( X ) ) i Q ← degrading merge ( W , L , P U ( X ) ) for j = 1 , 2 , . . . , m do if b j = 0 then W ← Q − else W ← Q + Q ← degrading merge (W , L , P U ( X ) ) return P e (Q , P U ( X ) ) 10 / 19

Consequences: Try and build a polar code for W M . . . ◮ Would like number of good channels to be ≈ n · I ( W M ) ◮ However, number of good channels is upper bounded by � � n · I degrading merge ( W M , L , P U ( X ) ) 2 � �� 1 �� � q − 1 ≥ n · I ( W M ) − O L For q = 16, in order to lose at most 0 . 01, need L ≈ 10 15 11 / 19

LDPC: Same problem when trying to design an LDPC code for W M ◮ Pick a code ensamble with rate close to I ( W M ) ◮ Use density evolution to asses code: 1. Initialize ◮ Assume all-zero codeword ◮ Quantize output letters: letters with close posteriors are grouped together 2. Main loop ◮ Already hopeless at this point: main loop is with respect to quantized channel, which has mutual information below design rate 12 / 19

The channel W M : For an integer M ≥ 1, define W M : X → Y M as follows: ◮ Input alphabet is X = { 1 , 2 , . . . , q } ◮ Output alphabet is q � � � Y M = � j 1 , j 2 , . . . , j q � : j 1 , j 2 , . . . , j q ≥ 0 , j x = M , x =1 where j x are non-negative integers summing to M ◮ Channel transition probabilities: q · j x W ( � j 1 , j 2 , . . . , j q �| x ) = � M + q − 1 � M q − 1 ◮ Input distribution unifrom = ⇒ all output letters equally likely 13 / 19

The channel W M : ◮ Posterior probabilities P ( X = x | Y = � j 1 , j 2 , . . . , j q � ) = j x M ◮ Shorthand: output letter is labelled by posterior probabilities vector � j 1 , j 2 , . . . , j q � � ( j 1 / M , j 2 / M , . . . , j q / M ) 14 / 19

Optimal degrading: Claim [KurkoskiYagi]: ◮ Let W : X → Y , P X , and L be given. ◮ Let Q : X → Z be an optimal degrading of W to a channel Q with |Z| ≤ L . ◮ That is, I ( X , Y ) − I ( X , Y ′ ) is minimized. ◮ Then, Q is gotten from W by defining a partition ( A i ) L i =1 of Y and mapping with probability 1 all symbols in A i to a single symbol z i ∈ Z Let ( A i ) L i =1 be such a partition with respect to W M 15 / 19

L 2 squared bound: Lemma: For A = A i as above, let ∆( A ) be the drop in mutual information incurred by merging all the letters in A i into a single letter. Then, ∆( A ) ≥ ˜ ∆( A ) , where 1 1 ˜ � p � 2 � � p − ¯ ¯ ∆( A ) = 2 , p = | A | p . � M + q − 1 � 2 q − 1 p ∈ A p ∈ A 16 / 19

Bounding in terms of | A | : Lemma: L L L q +1 � � ˜ � q − 1 + o (1) , ∆( A i ) ≥ ∆( A i ) ≥ const ( q ) · | A i | i =1 i =1 i =1 where the o (1) is a function of M alone and goes to 0 as M → ∞ Observation: Up to the o (1), expression is convex in | A i | . Thus, sum is lower bounded by setting | A i | = |Y M | / L . 17 / 19

Theorem: 2 2 q − 1 � 1 � � 1 � q − 1 q − 1 DC ( q , L ) ≥ 2( q + 1) · · , σ q − 1 · ( q − 1)! L where σ q − 1 is the constant for which the volume of a sphere in R q − 1 of radius r is σ q − 1 r q − 1 18 / 19

Backup ◮ Just how representative is W M ? ◮ What can be done? ◮ Channels W M “converges” to ◮ W ∞ : X → X × [0 , 1] q ◮ Given an input x , the channel picks ϕ 1 , ϕ 2 , . . . , ϕ q , non-negative reals summing to 1. All possible choices are equally likely, Dirichlet(1,1,. . . ,1) ◮ Then, the input x is transformed into x + i (with a modulo operation where appropriate) with probability ϕ i ◮ The transformed symbol along with the vector ( ϕ 1 , ϕ 2 , . . . , ϕ q ) are the output of the channel 19 / 19