An Explicit Formula for the Zero-Error Feedback Capacity of a Class - PowerPoint PPT Presentation

An Explicit Formula for the Zero-Error Feedback Capacity of a Class of Finite-State Additive Noise Channels Amir Saberi *, Farhad Farokhi and Girish Nair EEE Department, The University of Melbourne Email: a.saberi @ student.unimelb.edu.au ISIT 2020 June 5, 2020 A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 1 / 17

Discrete finite-state additive noise channel Z i X i ⊕ Y i = X i ⊕ Z i A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 2 / 17

Discrete finite-state additive noise channel Z i X i ⊕ Y i = X i ⊕ Z i Z i ⊥ X 1: i ⊕ is mod q addition X i , Y i , Z i ∈ X := { 0 , 1 , . . . , q − 1 } A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 2 / 17

Discrete finite-state additive noise channel Z i X i ⊕ Y i = X i ⊕ Z i Aljajy’95: C = C f = log q − H ( Z ) H ( Z ) : The noise process entropy rate A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 2 / 17

Outline Channel model and definitions 1 Results on zero-error capacities 2 Examples 3 Sketch of the proofs 4 A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 3 / 17

Finite-state machine A directed graph G = ( S , E ) with vertex set S = { 0 , 1 , . . . , |S| − 1 } denoting states edge set E ⊆ S × S denoting possible transitions between two states. S i = 0 S i = 1 A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 4 / 17

For a finite-state machine Topological entropy 1 � � � � h := lim N log � set of all state seq. of length N � � N →∞ � A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 5 / 17

For a finite-state machine Topological entropy 1 � � � � h := lim N log � set of all state seq. of length N � � N →∞ � h = log λ P , ( λ P : Perron value ) A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 5 / 17

For a finite-state machine Topological entropy 1 � � � � h := lim N log � set of all state seq. of length N � � N →∞ � h = log λ P , ( λ P : Perron value ) λ P = max | λ ( A ) | � � √ 1 1 S i = 0 S i = 1 ⇒ A = ⇒ = 1 + 5 1 0 2 A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 5 / 17

Finite-state additive noise channel Y i = X i ⊕ Z i , i ∈ N , X i , Y i , Z i ∈ X := { 0 , 1 , . . . , q − 1 } , ⊕ is mod q addition; correlated additive noise Z i is governed by a finite-state machine; each outgoing edge from a state s i corresponds to different values z i of the noise. An example channel at which no consecutive errors are allowed: Z i = 1 Z i = 0 S i = 0 S i = 1 Z i = 0 A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 6 / 17

ˆ X i Y i M M Encoder Channel Decoder Zero-error capacity log |F| C 0 := sup n n ∈ N , F∈ F F ⊆ X n is the set of all n − block codes yielding zero decoding errors; for any channel noise sequence and channel initial state; no state information is available at the encoder and decoder. A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 7 / 17

ˆ X i Y i M M Encoder Channel Decoder Delay Zero-error feedback capacity log |M| C 0 f := sup n n ∈ N , F 1: n ∈F M The zero-error capacity of the channel having a feedback from the output. x i ( m ) = f m,i ( y 1: i − 1 ) , i = 1 , . . . , n, m ∈ M F 1: n = { f m, 1: n | m ∈ M} , where f m,i is the encoding function. A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 8 / 17

For a finite-state additive noise channel Theorem The zero-error feedback capacity is either zero or C 0 f = log q − h ( Z ) Moreover, C 0 ≥ log q − 2 h ( Z ) A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 9 / 17

For a finite-state additive noise channel Theorem The zero-error feedback capacity is either zero or C 0 f = log q − h ( Z ) Moreover, C 0 ≥ log q − 2 h ( Z ) Theorem C 0 = 0 ⇐ ⇒ C 0 f = 0 . A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 9 / 17

Examples Noise finite-state machine Capacity ( q = 2) Capacity ( q > 2) 1 C 0 f = log q φ , 0 S i = 0 S i = 1 C 0 = C 0 f = 0 ≥ log q C 0 φ 2 0 0 0 q C 0 f = 0 . 594 , C 0 f = log 1 . 325 , S i = 0 1 S i = 1 S i = 2 q C 0 ≥ 0 . 189 C 0 ≥ log 1 . 755 0 √ φ := 1+ 5 2 A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 10 / 17

Examples p 1 − p 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 1 S = 0 S = 1 A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 11 / 17

Examples p 1 − p 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 1 S = 0 S = 1 Z i = 1 Z i = 0 ¯ ¯ S i = 0 S i = 1 Z i = 0 A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 11 / 17

Examples p 1 − p 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 1 S = 0 S = 1 Z i = 1 Z i = 0 ¯ ¯ S i = 0 S i = 1 Z i = 0 log 5 φ 2 ≤ C 0 ≤ C 0 f = log 5 φ A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 11 / 17

Sketch of the proof � � � � � ∝ λ k � � � � | Y ( k ) | = � { y 1: k | M = m } � = � set of all state seq. length k � � � � P A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 12 / 17

Sketch of the proof � � � � � ∝ λ k � � � � | Y ( k ) | = � { y 1: k | M = m } � = � set of all state seq. length k � � � � P Converse: # of distinguisable messages ≤ q k /λ k P A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 12 / 17

Sketch of the proof � � � � � ∝ λ k � � � � | Y ( k ) | = � { y 1: k | M = m } � = � set of all state seq. length k � � � � P Achievability: Parity check symbols: which output (from { 1 , . . . , | Y ( k ) |} ) was received. A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 12 / 17

Sketch of the proof � � � � � ∝ λ k � � � � | Y ( k ) | = � { y 1: k | M = m } � = � set of all state seq. length k � � � � P Achievability: Parity check symbols: which output (from { 1 , . . . , | Y ( k ) |} ) was received. C 0 f − δ = log | Y ( k ) | kh ≈ n − k n − k A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 12 / 17

Sketch of the proof C 0 f ≥ k C 0 f − δ n log q � C 0 f − δ + h log q A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 13 / 17

Sketch of the proof C 0 f ≥ k C 0 f − δ n log q � C 0 f − δ + h log q 1 � constant � � � ⇒ C 0 f ≥ log q − log λ − δ 1 − log q − C 0 f n A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 13 / 17

Theorem ⇒ ∀ d 1: n ∈ X n , n ∈ N , ∃ a walk on the coupled graph C 0 = C 0 f = 0 ⇐ with the label sequence d 1: n . A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 14 / 17

Theorem ⇒ ∀ d 1: n ∈ X n , n ∈ N , ∃ a walk on the coupled graph C 0 = C 0 f = 0 ⇐ with the label sequence d 1: n . Coupled graph G c = G × G (tensor product); ◮ Vertex set: V = S × S ; ◮ Vertices ( i, j ) � ( k, m ) iff edges from i � k and j � m ; Each edge label is E ik ⊖ E jm . A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 14 / 17

Theorem ⇒ ∀ d 1: n ∈ X n , n ∈ N , ∃ a walk on the coupled graph C 0 = C 0 f = 0 ⇐ with the label sequence d 1: n . Coupled graph G c = G × G (tensor product); ◮ Vertex set: V = S × S ; ◮ Vertices ( i, j ) � ( k, m ) iff edges from i � k and j � m ; Each edge label is E ik ⊖ E jm . 1 (0 , 0) (1 , 1) 0 S i = 0 S i = 1 0 (1 , 0) (0 , 1) A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 14 / 17

Theorem ⇒ ∀ d 1: n ∈ X n , n ∈ N , ∃ a walk on the coupled graph C 0 = C 0 f = 0 ⇐ with the label sequence d 1: n . (0 , 0) (1 , 1) y 1: n = f m, 1: n ( z 1: n − 1 ) ⊕ z 1: n 1 1 1 1 y ′ 1: n = f m ′ , 1: n ( z ′ 1: n − 1 ) ⊕ z ′ 1: n (1 , 0) (0 , 1) A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 15 / 17

Theorem ⇒ ∀ d 1: n ∈ X n , n ∈ N , ∃ a walk on the coupled graph C 0 = C 0 f = 0 ⇐ with the label sequence d 1: n . (0 , 0) (1 , 1) y 1: n = f m, 1: n ( z 1: n − 1 ) ⊕ z 1: n 1 1 1 1 y ′ 1: n = f m ′ , 1: n ( z ′ 1: n − 1 ) ⊕ z ′ 1: n (1 , 0) (0 , 1) d 1: n := f m ′ , 1: n ( z ′ 1: n − 1 ) ⊖ f m, 1: n ( z 1: n − 1 ) A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 15 / 17

Theorem ⇒ ∀ d 1: n ∈ X n , n ∈ N , ∃ a walk on the coupled graph C 0 = C 0 f = 0 ⇐ with the label sequence d 1: n . (0 , 0) (1 , 1) y 1: n = f m, 1: n ( z 1: n − 1 ) ⊕ z 1: n 1 1 1 1 y ′ 1: n = f m ′ , 1: n ( z ′ 1: n − 1 ) ⊕ z ′ 1: n (1 , 0) (0 , 1) d 1: n := f m ′ , 1: n ( z ′ 1: n − 1 ) ⊖ f m, 1: n ( z 1: n − 1 ) . . . . . . . . . . . . d 1: n = z 1: n ⊖ z ′ ⇒ y 1: n = y ′ 1: n ⇐ 1: n A. Saberi (U of Melbourne) C 0 f = 1 − h ( Z ) June 5, 2020 15 / 17