Data-Driven Ensembles for Deep and Hard-Decision Hybrid Decoding - PowerPoint PPT Presentation

Introduction Introduction Introduction Data-Driven Ensembles for Deep and Hard-Decision Hybrid Decoding International Symposium on Information Theory Tomer Raviv Nir Raviv Yair Be’ery School of Electrical Engineering Tel Aviv University, Israel June 2020

Outline I. Background II. On the Importance of Data III. Data-Driven Ensembles IV. Summary

Outline I. Background II. On the Importance of Data III. Data-Driven Ensembles IV. Summary

Error Correction Codes Error Correction Codes increases reliability of transmission by redundancy Assumptions Linear Codes AWGN channel Galois Field - GF(2) Encoding

Simplified Communication System

Parity Check Matrix Property of linear codes Known as parity check matrix Size Each row corresponds to a check constraint Each column to a variable

• Tanner Graph [1] Urbanke and Richardson ,“Modern Coding Theory” , 2008

Belief Propagation [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Belief Propagation [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Belief Propagation [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Belief Propagation [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Belief Propagation [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Belief Propagation [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Belief Propagation [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Belief Propagation [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Belief Propagation Hard decision rule: [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988

Suboptimal Performance Cycles degrade performance of BP… How to compensate for cycles?

Suboptimal Performance Cycles degrade performance of BP… How to compensate for cycles? Deep learning!

Weighted Belief Propagation Deep learning • Assign weights over the edges • Choose appropriate loss • Apply SGD Intuition Adjusting the weights compensates for small cycles [3] Nachmani et al. , “Learning to Decode Linear Codes Using Deep Learning”, Sep 16

WBP [3] Nachmani et al. , “Learning to Decode Linear Codes Using Deep Learning”, Sep 16

WBP Binary cross entropy multiloss: The BP and channel exhibit symmetry Training is done with zero codeword [3] Nachmani et al. , “Learning to Decode Linear Codes Using Deep Learning”, Sep 16

Outline I. Background II. On the Importance of Data III. Data-Driven Ensembles IV. Summary

ML Model Selection in Decoding • Incorporating domain knowledge into model • Constraints-free model

ML Model Selection in Decoding • Incorporating domain knowledge into model Choice of hypothesis class Learnable parameters assignment

ML Model Selection in Decoding • Constraints-free model Choice of any state-of-the-art architecture Less limitations on solution space [4 ] Tobias Gruber et al. “On deep learning - based channel decoding”. In: 51st Annual Conference on Information Sciences and Systems (CISS). 2017.

ML Model Selection in Decoding Currently, the model based approach dominates Curse of dimensionality Yet, the model isn’t all…

Data Importance in Deep Learning Training data is core in DL Not fully understood

Data Distinction Classical ML Decoding ML • Limited amount • Unlimited amount • Distribution is unknown • Distribution is specified by channel

Outline I. Background II. On the Importance of Data III. Data-Driven Ensembles IV. Summary

Ensembles A single decoder is limited.. Combination of decoders is superior Divide and conquer principle [5] Lior Rokach , “Pattern Classification using Ensemble Methods”, 2010

Ensembles for Decoding List-decoding is suboptimal ”there exists no clear evidence on which graph permutation performs best for a given input” [6] Elkelesh et al., “Belief Propagation List Decoding of Polar Codes”, 2018

Ensembles for Decoding Key points decoding performance complexity [6] Elkelesh et al., “Belief Propagation List Decoding of Polar Codes”, 2018

Data-Driven Ensemble

Data Specialized Decoders Based on Hamming distance Error pattern computation Region assignment Match to dataset Train on Based on syndrome ( won’t have time to cover)…

Gating Function Classical hard decision decoder (HDD) Outputs , calculate Gating types single-choice gating - for realizing all decoders gating - random-choice gating - randomly

Combiner Likelihood-maximizing function [7]: [7] Ralf Koetter et al., “Characterizations of pseudo -codewords of (low- density) parity check codes”, 2007

Experimental Results Results for BCH(63,45): CR-BCH(63,36) Code Waterfall - 0.3dB FER gain Error-floor – 1.25dB FER gain Compared to best previous results [8] 3-decoders [8] Ishay Be’ery et al., "Active Deep Decoding of Linear Codes," in IEEE Transactions on Communications, Feb. 2020.

Experimental Results Results for BCH(63,45): CR-BCH(63,45) Code Waterfall - 0.3dB FER gain Error-floor – 1dB FER gain Compared to best previous results [8] 3-decoders [8] Ishay Be’ery et al., "Active Deep Decoding of Linear Codes," in IEEE Transactions on Communications, Feb. 2020.

Outline I. Background II. On the Importance of Data III. Data-Driven Ensembles IV. Summary

Summary Domain knowledge is vital for ML in communication Data is important as the algorithm (!) How to effectively and efficiently utilize data is still insufficiently researched

Bibliography [1] Urbanke and Richardson ,“Modern Coding Theory” , 2008 [2] Judea Pearl , “ Probabilistic reasoning in intelligent systems: networks of plausible inference ” , 1988 [3] Nachmani et al. , “Learning to Decode Linear Codes Using Deep Learning”, Sep 16 [4 ] Tobias Gruber et al. “On deep learning - based channel decoding”. In: 51st Annual Conference on Information Sciences and Systems (CISS). 2017. [5] Lior Rokach , “Pattern Classification using Ensemble Methods”, 2010 [6] Elkelesh et al., “Belief Propagation List Decoding of Polar Codes”, 2018 [7] Ralf Koetter et al., “Characterizations of pseudo -codewords of (low- density) parity check codes”, 2007 [8] Ishay Be’ery et al., "Active Deep Decoding of Linear Codes," in IEEE Transactions on Communications, Feb. 2020.