Graceful degradation over the BEC via non-linear codes Hajir - PowerPoint PPT Presentation

Graceful degradation over the BEC via non-linear codes Hajir Roozbehani, Yury Polyanskiy LIDS Massachusetts Institute of Technology June 2020 2020 IEEE International Symposium on Information Theory H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 1

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k Channel is Y i ( ǫ ) = BEC ǫ ( X i ) H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k Channel is Y i ( ǫ ) = BEC ǫ ( X i ) k E [ d H ( S k , ˆ Interested in BER for systematic bits BER f,g ( ǫ ) = 1 S k ( ǫ ))] H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k Channel is Y i ( ǫ ) = BEC ǫ ( X i ) k E [ d H ( S k , ˆ Interested in BER for systematic bits BER f,g ( ǫ ) = 1 S k ( ǫ ))] H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k Channel is Y i ( ǫ ) = BEC ǫ ( X i ) k E [ d H ( S k , ˆ Interested in BER for systematic bits BER f,g ( ǫ ) = 1 S k ( ǫ ))] Both encoder/decoder are fixed H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k Channel is Y i ( ǫ ) = BEC ǫ ( X i ) k E [ d H ( S k , ˆ Interested in BER for systematic bits BER f,g ( ǫ ) = 1 S k ( ǫ ))] Both encoder/decoder are fixed Want an encoder/decoder pair with a smooth BER curve as ǫ varies BER C/R 1 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k Channel is Y i ( ǫ ) = BEC ǫ ( X i ) k E [ d H ( S k , ˆ Interested in BER for systematic bits BER f,g ( ǫ ) = 1 S k ( ǫ ))] Both encoder/decoder are fixed Want an encoder/decoder pair with a smooth BER curve as ǫ varies BER C/R 1 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k Channel is Y i ( ǫ ) = BEC ǫ ( X i ) k E [ d H ( S k , ˆ Interested in BER for systematic bits BER f,g ( ǫ ) = 1 S k ( ǫ ))] Both encoder/decoder are fixed Want an encoder/decoder pair with a smooth BER curve as ǫ varies BER C/R 1 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Graceful degradation over BEC Source generates iid bits S k ∼ Ber(1 / 2) k Channel is Y i ( ǫ ) = BEC ǫ ( X i ) k E [ d H ( S k , ˆ Interested in BER for systematic bits BER f,g ( ǫ ) = 1 S k ( ǫ ))] Both encoder/decoder are fixed Want an encoder/decoder pair with a smooth BER curve as ǫ varies Need two-point converses BER C/R 1 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 2

Two point converses BER C/R 1 Shannon converse H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 3

Two point converses BER Anchor point A C/R 1 Shannon converse H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 3

Two point converses BER Exists? Anchor point A C/R 1 Shannon converse H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 3

Two point converses BER Separation scheme Anchor point B C/R 1 Shannon converse H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 3

Two point converses BER Exists? Separation scheme Anchor point B C/R 1 Shannon converse H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 3

Two point converses BER Exists? Separation scheme Anchor point B C/R 1 Shannon converse We will show that non-trivial codes exist = ⇒ H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 3

Two point converses BER Exists? Separation scheme Anchor point B C/R 1 Shannon converse We will show that non-trivial codes exist = ⇒ Need to understand the two-point fundamental limits H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 3

Motivation: channel transforms Goal: transform a channel with high BER into one with low BER H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 4

Motivation: channel transforms Goal: transform a channel with high BER into one with low BER Channel H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 4

Motivation: channel transforms Goal: transform a channel with high BER into one with low BER Channel BER C/R 1 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 4

Motivation: channel transforms Goal: transform a channel with high BER into one with low BER Applications: coding for optical channels [ZK17, BK18], tornado-raptor codes, multi-user information theory, delay-sensitive applications (control, short-packet communication) Channel BER C/R 1 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 4

Motivation: channel transforms Goal: transform a channel with high BER into one with low BER Applications: coding for optical channels [ZK17, BK18], tornado-raptor codes, multi-user information theory, delay-sensitive applications (control, short-packet communication) Our main result today: proof that non-linear codes (namely, LDMCs) can provably outperform any linear-systematic code H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 4

Motivation: channel transforms Goal: transform a channel with high BER into one with low BER Applications: coding for optical channels [ZK17, BK18], tornado-raptor codes, multi-user information theory, delay-sensitive applications (control, short-packet communication) Our main result today: proof that non-linear codes (namely, LDMCs) can provably outperform any linear-systematic code How? new two-point converse bounds for linear-systematic codes H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 4

Low Density Majority Codes (LDMCs) Sparse graph codes: 0 1 2 3 4 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 5

Low Density Majority Codes (LDMCs) Sparse graph codes: 0 1 2 3 4 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 5

Low Density Majority Codes (LDMCs) Sparse graph codes: 0 1 2 3 4 H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 5

Low Density Majority Codes (LDMCs) Sparse graph codes: 0 1 2 3 4 Raptor codes[Sho06], non-linear compression [CMZ05], [MM08],[GV09] , Gallager codes [Mac05] H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 5

Low Density Majority Codes (LDMCs) Sparse graph codes: 0 1 2 3 4 Raptor codes[Sho06], non-linear compression [CMZ05], [MM08],[GV09] , Gallager codes [Mac05] Here f = majority H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 5

Low density majority codes Sample a few indices randomly+ take majorities: x �→ (maj( x 1 , x 2 , x 3 ) , maj( x 1 , x 3 , x 5 ) , · · · ) Example Suppose we have 1 equation left in three variables. Best possible distortion with XOR? y = x 1 + x 2 = ⇒ P ( x 1 = 1) = P ( x 1 = 0) = 1 / 2 Best linear code is x �→ x 1 . Average distortion is 1/3. But P ( x 1 = maj( x 1 , x 2 , x 3 )) = 3 / 4 Average distortion with majority is 1/4! H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 6

Animation H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 7

Classic codes vs non-linear codes 0.5 Regular systematic LDMC(5)-5 BP steps Regular LDMC(5)-5 BP steps LDPC-50 peeling steps Repetition 0.4 0.3 BER 0.2 0.1 0.0 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 C/R H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 8

Classic codes vs non-linear codes 0.5 Regular systematic LDMC(5)-5 BP steps Regular LDMC(5)-5 BP steps LDPC-50 peeling steps Repetition 0.4 0.3 BER 0.2 0.1 0.0 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 C/R Did we pick a bad LDPC? Can we do better than LDMCs with linear codes? = ⇒ need converse bounds. H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 8

Two point converses 0.5 Single-point Shannon converse (achievable ) Two-point converse Performance of a generic separation scheme 0.4 LDMC simulations 0.3 BER 0.2 0.1 A 0.0 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 C/R Orange line: converse for any code whose BER curve passes through or is below point A [KOP20, TKS13, KC16] Not strong enough! We need a better converse for linear codes H. Roozbehani, Y. Polyanskiy Graceful degradation over the BEC via non-linear codes 9