Comparison of Channels: Criteria for Domination by a Symmetric - PowerPoint PPT Presentation

Comparison of Channels: Criteria for Domination by a Symmetric Channel Anuran Makur and Yury Polyanskiy EECS Department, Massachusetts Institute of Technology LIDS Student Conference 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 1 / 17

Outline Introduction 1 Preliminaries Guiding Question Motivation Conditions for Domination by a Symmetric Channel 2 Less Noisy Domination and Log-Sobolev Inequalities 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 2 / 17

Preliminaries A channel is a set of conditional distributions W Y | X that is represented by a row stochastic matrix W ∈ R q × r sto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 3 / 17

Preliminaries A channel is a set of conditional distributions W Y | X that is represented by a row stochastic matrix W ∈ R q × r sto . P q = probability simplex of row vectors in R q . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 3 / 17

Preliminaries A channel is a set of conditional distributions W Y | X that is represented by a row stochastic matrix W ∈ R q × r sto . P q = probability simplex of row vectors in R q . Recall that KL divergence is defined as: ( P X ( x ) ) ∑ D ( P X || Q X ) � P X ( x ) log . Q X ( x ) x ∈X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 3 / 17

Preliminaries A channel is a set of conditional distributions W Y | X that is represented by a row stochastic matrix W ∈ R q × r sto . P q = probability simplex of row vectors in R q . Recall that KL divergence is defined as: ( P X ( x ) ) ∑ D ( P X || Q X ) � P X ( x ) log . Q X ( x ) x ∈X Definition (Less Noisy Preorder [KM77]) W ∈ R q × r sto is less noisy than V ∈ R q × s sto , denoted W ≽ ln V , iff: ∀ P X , Q X ∈ P q , D ( P X W || Q X W ) ≥ D ( P X V || Q X V ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 3 / 17

Guiding Question Definition ( q -ary Symmetric Channel) The q -ary symmetric channel is defined as:  δ δ  1 − δ · · · q − 1 q − 1 δ δ 1 − δ · · ·   q − 1 q − 1 ∈ R q × q W δ �   . . . ...   sto . . . . . .     δ δ · · · 1 − δ q − 1 q − 1 where δ ∈ [0 , 1] is the total crossover probability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 4 / 17

Guiding Question Definition ( q -ary Symmetric Channel) The q -ary symmetric channel is defined as:  δ δ  1 − δ · · · q − 1 q − 1 δ δ 1 − δ · · ·   q − 1 q − 1 ∈ R q × q W δ �   . . . ...   sto . . . . . .     δ δ · · · 1 − δ q − 1 q − 1 where δ ∈ [0 , 1] is the total crossover probability. Remark: For every channel V ∈ R q × s sto , W 0 ≽ ln V and V ≽ ln W ( q − 1) / q . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 4 / 17

Guiding Question Definition ( q -ary Symmetric Channel) The q -ary symmetric channel is defined as:  δ δ  1 − δ · · · q − 1 q − 1 δ δ 1 − δ · · ·   q − 1 q − 1 ∈ R q × q W δ �   . . . ...   sto . . . . . .     δ δ · · · 1 − δ q − 1 q − 1 where δ ∈ [0 , 1] is the total crossover probability. Remark: For every channel V ∈ R q × s sto , W 0 ≽ ln V and V ≽ ln W ( q − 1) / q . 0 , q − 1 [ ] What is the q -ary symmetric channel with the largest δ ∈ q that is less noisy than a given channel V ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 4 / 17

Motivation: Strong Data Processing Inequality Data Processing Inequality : For any channel V ∈ R q × s sto , ∀ P X , Q X ∈ P q , D ( P X || Q X ) ≥ D ( P X V || Q X V ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 5 / 17

Motivation: Strong Data Processing Inequality Strong Data Processing Inequality [AG76]: For any channel V ∈ R q × s sto , ∀ P X , Q X ∈ P q , η D ( P X || Q X ) ≥ D ( P X V || Q X V ) where η ∈ [0 , 1] is a channel dependent contraction coefficient. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 5 / 17

Motivation: Strong Data Processing Inequality Strong Data Processing Inequality [AG76]: For any channel V ∈ R q × s sto , ∀ P X , Q X ∈ P q , η D ( P X || Q X ) ≥ D ( P X V || Q X V ) where η ∈ [0 , 1] is a channel dependent contraction coefficient. Relation to Erasure Channels [PW16]: A q -ary erasure channel E 1 − η ∈ R q × ( q +1) erases its input with probability sto 1 − η , and keeps it the same with probability η . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 5 / 17

Motivation: Strong Data Processing Inequality Strong Data Processing Inequality [AG76]: For any channel V ∈ R q × s sto , ∀ P X , Q X ∈ P q , η D ( P X || Q X ) ≥ D ( P X V || Q X V ) where η ∈ [0 , 1] is a channel dependent contraction coefficient. Relation to Erasure Channels [PW16]: A q -ary erasure channel E 1 − η ∈ R q × ( q +1) erases its input with probability sto 1 − η , and keeps it the same with probability η . What is the q-ary erasure channel with the smallest η ∈ [0 , 1] that is less noisy than a given channel V ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 5 / 17

Motivation: Strong Data Processing Inequality Strong Data Processing Inequality [AG76]: For any channel V ∈ R q × s sto , ∀ P X , Q X ∈ P q , η D ( P X || Q X ) ≥ D ( P X V || Q X V ) where η ∈ [0 , 1] is a channel dependent contraction coefficient. Relation to Erasure Channels [PW16]: A q -ary erasure channel E 1 − η ∈ R q × ( q +1) erases its input with probability sto 1 − η , and keeps it the same with probability η . What is the q-ary erasure channel with the smallest η ∈ [0 , 1] that is less noisy than a given channel V ? Prop: E 1 − η ≽ ln V ⇔ ∀ P X , Q X ∈ P q , η D ( P X || Q X ) ≥ D ( P X V || Q X V ). SDPI = ≽ ln domination by erasure channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 5 / 17

Motivation: Wyner’s Wiretap Channel 𝑍 𝑜 Decoder 𝑁 𝑌 𝑜 𝑁 𝑄 𝑍,𝑎|𝑌 Encoder 𝑎 𝑜 Eavesdropper Channel P Y | X = V is the main channel . P Z | X = W δ is the eavesdropper channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 6 / 17

Motivation: Wyner’s Wiretap Channel 𝑍 𝑜 Decoder 𝑁 𝑌 𝑜 𝑁 𝑄 𝑍,𝑎|𝑌 Encoder 𝑎 𝑜 Eavesdropper Channel P Y | X = V is the main channel . P Z | X = W δ is the eavesdropper channel . Secrecy capacity C S = maximum rate that can be sent to the legal receiver such that P ( M ̸ = ˆ M ) and 1 n I ( M ; Z n ) asymptotically vanish. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 3 February 2017 6 / 17