ECEN 5682 Theory and Practice of Error Control Codes Introduction - PowerPoint PPT Presentation

Introduction to Block Codes ECEN 5682 Theory and Practice of Error Control Codes Introduction to Block Codes Peter Mathys University of Colorado Spring 2007 Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes Basic Definitions Definition: A block code of length n and size M over an alphabet with q symbols is a set of M q -ary n -tuples called codewords. Example: Code #1 . Binary code of length n = 5 with M = 4 codewords given by C = { 00000 , 01011 , 10101 , 11110 } . Definition: The rate R of a q -ary block code of length n with M codewords is given by R = log q M . n Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes Definition: The redundancy r of a q -ary block code of length n with M codewords is given by r = n − log q M . Example: Code #1 has rate R = log 2 4 / 5 = 2 / 5 = 0 . 4 and redundancy r = 5 − log 2 4 = 3 bits. Example: Code #2 . 5-ary code of length n = 4 with M = 5 codewords given by C = { 0000 , 1342 , 2134 , 3421 , 4213 } . This code has rate R = log 5 5 / 4 = 1 / 4 = 0 . 25 and redundancy r = 4 − log 5 5 = 3 symbols. Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes The goal when using error control codes is to detect and/or correct transmission errors. Suppose code #1 is used and the (corrupted) codeword v = (00101) is received. Comparing v with all “legal” codewords and marking the discrepancies with * yields: 00000 01011 10101 11110 00101 00101 00101 00101 ----- ----- ----- ----- ..*.* .***. *.... **.*. The discrepancies are the result of transmission errors. If all error positions are marked with one and all other positions with zero, then the received codeword v = (00101) corresponds to the set of possible errors E = { 00101 , 01110 , 10000 , 11010 } , when code #1 is used. But which of these 4 errors is the “right” one? Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes To decide which error out of a set of errors is the “right” one, one needs to make additional assumptions about the likelihood of errors and error patterns. The two most common models for the occurrence of errors are: (i) Independent and identically distributed (iid) errors with probablity ǫ . This requires a memoryless transmission channel model. (ii) Burst errors of length L . If an error occurs, it is very likely that it is followed by L − 1 more errors. Burst errors occur for instance in mobile communications due to fading and in magnetic recording due to media defects. Burst errors can be converted to iid errors by the use of an interleaver . Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes More generally, and especially for non-binary codes, one also needs a model for the error amplitudes . Two possibilities are (i) Uniformly distributed non-zero error amplitudes . This is a good model for orthogonal signaling. (ii) Non-uniformly distributed non-zero error amplitudes with smaller error magnitudes more likely than larger ones. This is a good model for QAM signaling. Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes In addition to the models that describe which error pattern e is most likely, a transmission channel model is also needed that specifies how codewords c and error patterns e are combined to form the received codeword v = f ( c , e ). The most prevalent model is the additive model shown in the following figure. Note that addition is often assumed to be modulo q addition for a q -ary code. error e v = c + e c + codeword received vector addition codeword (often modulo q ) A concise graphical way to describe simple error models is in the form of a discrete channel model that shows all possible transitions from the channel input X to the channel output Y , together with the associated transition probabilities p Y | X ( y | x ). Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes Example: The simplest discrete channel model is the memoryless binary symmetric channel (BSC) shown in the following figure. 1 − ǫ 1 1 • • ǫ Input Output X Y ǫ 0 0 • • 1 − ǫ This channel is completely described by the set of four transition probabilities: p Y | X (0 | 0) = 1 − ǫ , p Y | X (1 | 0) = ǫ , p Y | X (0 | 1) = ǫ , p Y | X (1 | 1) = 1 − ǫ . Clearly P { Y � = X } = ǫ and thus the (uncoded) probability of a bit error is P b ( E ) = ǫ . Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes Thus, if ǫ < 0 . 5 on a memoryless BSC, fewer errors are more likely and the “right” error pattern is the one with the fewest number of 1’s in it. Note that, since all symbols are binary here, only errors of amplitude 1 are possible and no specification for the distribution of error amplitudes is needed. Example: Suppose q = 5, errors occur iid with Pr { Y � = X } < 0 . 5, and uniformly distributed amplitudes. The corresponding channel model is a memoryless 5-ary symmetric channel (5SC) with transition probabilities � ǫ/ 4 , if y � = x , x , y ∈ { 0 , 1 , 2 , 3 , 4 } , p Y | X ( y | x ) = 1 − ǫ , if y = x , x , y ∈ { 0 , 1 , 2 , 3 , 4 } . In this case the decoding rule assumes again that the “right” error pattern is the one with the fewest nonzero symbols in it. Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes Example: Suppose again q = 5 and errors occur iid with P { Y � = X } < 0 . 5. But now assume that only errors of magnitude 1 occur, with +1 and − 1 being equally likely. This leads to another memoryless 5SC with transition probabilities  ǫ/ 2 , if y = x ± 1 (mod 5) , x , y ∈ { 0 , 1 , 2 , 3 , 4 } ,  p Y | X ( y | x ) = 1 − ǫ , if y = x , x , y ∈ { 0 , 1 , 2 , 3 , 4 } , 0 , otherwise .  Now the decoder decides on the error pattern with the fewest number of ± 1 (mod 5) symbols as the “right” error. Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes Once an error and a channel model are defined, a distance measure between codewords can be defined. Then one can determine the minimum distance between any two distinct codewords and this in turn determines how many errors a code can detect and/or correct under the given error and channel models. For the iid error model with (discrete) uniform error amplitude distribution the most appropriate measure is Hamming distance which is defined as follows. Definition: The Hamming distance d ( H ) ( x , y ) (or simply d ( x , y )) between two q -ary n -tuples x and y is the number of places in which they differ. Example: d (10221 , 20122) = 3. Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes The Hamming distance is probably the most popular distance measure for error control codes. Another measure that is more suitable in cases where smaller error magnitudes are more likely than larger ones is the Lee distance which is defined next. Definition: The Lee distance d ( L ) ( x , y ) between two q -ary n -tuples x and y is defined as d ( L ) ( x , y ) = | x 0 − y 0 | + | x 1 − y 1 | + . . . + | x n − 1 − y n − 1 | , where the magnitude | v | of a q -ary symbol v is computed modulo q as | v | = min { v , q − v } . Definition: The minimum distance d min of a code C = { c i , i = 0 , 1 , . . . M − 1 } is the smallest distance between any two distinct codewords of the code, i.e., d min = min i � = j d ( c i , c j ) , c i , c j ∈ C . Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes Example: Code #2 has has the following Hamming distances between pairs of distinct codewords: d ( H ) ( x , y ) 0000 1342 2134 3421 4213 0000 − 4 4 4 4 1342 4 − 4 4 4 2134 4 4 − 4 4 3421 4 4 4 − 4 4213 4 4 4 4 − The Lee distances between pairs of distinct codewords are: d ( L ) ( x , y ) 0000 1342 2134 3421 4213 0000 − 6 6 6 6 1342 6 − 6 6 6 2134 6 6 − 6 6 3421 6 6 6 − 6 4213 6 6 6 6 − Thus, for code #2, d ( H ) min = 4 and d ( L ) min = 6. Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Basic Definitions Introduction to Block Codes Linear Block Codes Example: Code #3 . Binary code with n = 10 and the following set of M = 4 codewords C = { 0010010111 , 0100101110 , 1001011100 , 1110001001 } . For this code the Hamming distances between pairs of codewords are d ( H ) ( x , y ) 0010010111 0100101110 1001011100 1110001001 0010010111 − 6 6 6 0100101110 6 − 6 6 1001011100 6 6 − 6 1110001001 6 6 6 − That is, code #3 has minimum Hamming distance 6. Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes