Decoding of Linear Codes Arman Fazeli Alexander Vardy Hanwen Yao - PowerPoint PPT Presentation

Hardness of Successive-Cancellation Decoding of Linear Codes Arman Fazeli Alexander Vardy Hanwen Yao afazelic@ucsd.edu avardy@ucsd.edu hwyao@ucsd.edu Virtual Presentation at International Symposium on Information Theory 2020

Outline • Motivation • Background • Problem Definition • SCD-Linear is NP-hard • Connections to Polar Codes

Outline • Motivation • Background • Problem Definition • SCD-Linear is NP-hard • Connections to Polar Codes

Motivation • Gained Interest in successive-cancellation decoding since the advent of polar coding from nearly a decade ago. • It can be accomplished with decoding complexity . • The complexity is remained unknown for other families of codes.

Motivation • Gained Interest in successive-cancellation decoding since the advent of polar coding from nearly a decade ago. • It can be accomplished with decoding complexity . • The complexity is remained unknown for other families of codes. • Secondary motivation: – SC decoding of large-kernel polar codes

Conventional Polar Codes

Conventional Polar Codes

Large-Kernel Polar Codes

Large-Kernel Polar Codes Naïve decoding complexity is given by .

Motivation • Gained Interest in successive-cancellation decoding since the advent of polar coding from nearly a decade ago. • It can be accomplished with decoding complexity . • The complexity is remained unknown for other families of codes. • Secondary motivation: – SC decoding of large-kernel polar codes • Is it possible to SC decode arbitrary linear codes efficiently ?

Outline • Motivation • Background • Problem Definition • SCD-Linear is NP-hard • Connections to Polar Codes

NP-hardness • A problem is said to be NP-hard if every other problem in NP can be reduced to it by a deterministic Turing machine in a polynomial time. • A common strategy to prove NP-hardness is by the polynomial reduction from a known NP-hard problem. Picture is in courtesy of Wikipedia. • A few NP-hard problems in coding theory: – Maximum-likelihood Decoding (MLD) – Computation of the weight distribution

ML Decoding • ML decoding is equivalent to minimum distance decoding for common communication channels such as binary symmetric channel (BSC) with flip probability . • MLD-Linear is shown to be NP-Hard for the class of binary linear codes by Berlekamp, McEliece, and van Tilborg. • It can be properly formulated as

Outline • Motivation • Background • Problem Definition • SCD-Linear is NP-hard • Connections to Polar Codes

Successive-Cancellation Decoding The channel model for the i -th synthesized channel, i.e. successive-cancellation decoding at step i , is given as follows. : uncoded information vector : transmitted codeword : received vector

SCD-Linear • The choice of the generator matrix matters! It can even change the hardness of the problem. • For a fixed generator matrix, the SCD-Linear decision problem is formally stated as follows.

Ensemble SCD-Linear • But there are many generator matrices that can generate the same code! A natural question follows: – How hard is the SCD-Linear problem if we relax the constraint on G and let the solution choose from the ensemble of all generator matrices for the given linear code? • This problem can be formally stated as follows.

Outline • Motivation • Background • Problem Definition • SCD-Linear is NP-hard • Connections to Polar Codes

SCD-Linear is NP-hard • Let be a generator matrix. • At step i, the values of are known. • The output of the channel is given by . • For a given frozen vector , define • The decision problem is SCD-Linear is equivalent to

SCD-Linear is NP-hard (continued) • Given that all the codewords in (for ) are transmitted equally likely, we have where denotes the Hamming distance. • So, the decision problem in SCD-Linear can be further simplified as where .

SCD-Linear is NP-hard (continued) • Now, set . For any , we have • This means that both sides of are dominated by the terms with the smallest powers.

SCD-Linear is NP-hard (continued) • Hence, SC-Distance, formally stated as bellow, is a special case of SCD-Linear. • The proof goes by showing that MLD-Linear problem can be polynomially reduced to the SC-Distance problem.

SC-Distance NP-hard • Let be a generator matrix of the given binary code C and y be the received vector. Further let denote the rows in . • The goal is to invoke the algorithmic solution for SCD- Distance (oracle) k times in order to find a nearest codeword of y in C . • Define • In step 1, we ask the SC-Distance oracle to find out about where a tie is broken with a coin flip. • This allows us to reduce the search size by half.

SC-Distance NP-hard (continued) • Repeat the same method in each step. In step i , – We start from a linear shift of denoted by , which consists at least one of the closest codewords to y . – Ask the SC-Distance oracle about which allows us to cut the search size by half once again. – Depending on the oracle ’ s response, update the shift vector as • After k steps, the shift vector is the closest codeword to y .

Ensemble SC-Distance is NP-hard • Each step is there to reduce the search size in half. • The choice of the generator matrix has no significant role. • We can achieve the same if the SC-Distance problem is relaxed to make the comparison for a generator matrix of its choice. • Hence, the resulting computational problem is also NP-hard. • Corollary: The Ensemble SCD-Linear problem is also NP-Hard.

Outline • Motivation • Background • Problem Definition • SCD-Linear is NP-hard • Connections to Polar Codes

Extension to Matrices • We showed that one cannot devise an SC decoding algorithm that runs efficiently for an arbitrary generator matrix. • Note that it is possible to add rows on the top of any generator matrix and the SC decoding of the original matrix will be contained within the SC decoding of the extended square matrix. • But this statement does not remain true if the additional rows are placed after or within the original rows. • This is the reason why the SCD-Linear for is not equivalent to the SC decoding of an -polar code.

Polar Codes With Dynamically Frozen Values Originally proposed in and later generalized in it was shown that one can improve the distance properties of polar codes by setting the values of frozen bits dynamically as a linear combination of the previously decoded bits. That is where ‘ s are fixed and known to both encoder and decoder.

Polar Codes With Dynamically Frozen Values • This can be achieved by precoding the length-n vector with a non-singular upper-triangular matrix . • The encoding relation is hence given by where serves as an input to the polar encoder. • Given that there is a one-to-one relation between and , the same SC decoder that is capable of computing is also capable of computing which extends the number of efficiently SC-decodable matrices by a factor of

Linear Codes Are Polar Codes with Dynamically Frozen Values • Let be a generator matrix of the given binary code C . Then there exists a unique matrix such that The matrix is not necessarily upper-triangular. It is not even necessarily a square matrix. We can apply a set of elementary row operations on both sides of to arrive at where is in the reduced row echelon form. is also a generator matrix of C . In other words, every • linear code can be encoded as a polar code with dynamically frozen values. • A remark not to miss is that while is a generator matrix of the code, SC decoding of the corresponding polar code is not equivalent to SC decoding of C .