Less Noisy Domination by Symmetric Channels Anuran Makur and Yury - PowerPoint PPT Presentation

Less Noisy Domination by Symmetric Channels Anuran Makur and Yury Polyanskiy EECS Department, Massachusetts Institute of Technology ISIT 2017 A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 1 / 28

Outline Introduction 1 Preliminaries Main Results Motivation: Strong Data Processing Inequality Main Question Less Noisy Channels in Networks Equivalent Characterizations of Less Noisy Preorder 2 Conditions for Less Noisy Domination by Symmetric Channels 3 Consequences of Less Noisy Domination by Symmetric Channels 4 A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 2 / 28

� � � � � � Preliminaries probability distributions – row vectors A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 3 / 28

� � � � � � Preliminaries probability distributions – row vectors channels (conditional distributions) – row stochastic matrices A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 3 / 28

Preliminaries probability distributions – row vectors channels (conditional distributions) – row stochastic matrices Definition (Less Noisy Preorder [K¨ orner-Marton 1977]) P Y | X = W is less noisy than P Z | X = V , denoted W � ln V , if and only if I ( U ; Y ) ≥ I ( U ; Z ) for every joint distribution P U , X such that U → X → ( Y , Z ). � � � � � � A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 3 / 28

Preliminaries probability distributions – row vectors channels (conditional distributions) – row stochastic matrices Definition (Less Noisy Preorder [K¨ orner-Marton 1977]) P Y | X = W is less noisy than P Z | X = V , denoted W � ln V , if and only if D ( P X W || Q X W ) ≥ D ( P X V || Q X V ) for every pair of input distributions P X and Q X . � � � � � � A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 4 / 28

Main Results 1 Test � ln using different divergence measure? A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 5 / 28

Main Results 1 Test � ln using different divergence measure? Yes , χ 2 -divergence A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 5 / 28

Main Results 1 Test � ln using different divergence measure? Yes , χ 2 -divergence 2 Sufficient conditions for � ln domination by symmetric channels? A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 5 / 28

Main Results 1 Test � ln using different divergence measure? Yes , χ 2 -divergence 2 Sufficient conditions for � ln domination by symmetric channels? Yes degradation criterion for general channels stronger criterion for additive noise channels A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 5 / 28

Main Results 1 Test � ln using different divergence measure? Yes , χ 2 -divergence 2 Sufficient conditions for � ln domination by symmetric channels? Yes degradation criterion for general channels stronger criterion for additive noise channels 3 Why � ln domination by symmetric channels? A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 5 / 28

Main Results 1 Test � ln using different divergence measure? Yes , χ 2 -divergence 2 Sufficient conditions for � ln domination by symmetric channels? Yes degradation criterion for general channels stronger criterion for additive noise channels 3 Why � ln domination by symmetric channels? just because we IT � ln domination ⇒ log-Sobolev inequality secrecy capacity A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 5 / 28

Motivation: Strong Data Processing Inequality Data Processing Inequality: For any channel V , ∀ P X , Q X , D ( P X || Q X ) ≥ D ( P X V || Q X V ) A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 6 / 28

Motivation: Strong Data Processing Inequality Strong Data Processing Inequality [Ahlswede-G´ acs 1976] : For any channel V , ∀ P X , Q X , η KL ( V ) D ( P X || Q X ) ≥ D ( P X V || Q X V ) where η KL ( V ) – contraction coefficient: D ( P X V || Q X V ) η KL ( V ) � sup ∈ [0 , 1] . D ( P X || Q X ) P X , Q X A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 6 / 28

Motivation: Strong Data Processing Inequality Strong Data Processing Inequality [Ahlswede-G´ acs 1976] : For any channel V , ∀ P X , Q X , η KL ( V ) D ( P X || Q X ) ≥ D ( P X V || Q X V ) where η KL ( V ) – contraction coefficient. Relation to Erasure Channels [Polyanskiy-Wu 2016] : Definition: q -ary erasure channel q - EC (1 − η ) erases input w.p. 1 − η , and reproduces input w.p. η . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 6 / 28

Motivation: Strong Data Processing Inequality Strong Data Processing Inequality [Ahlswede-G´ acs 1976] : For any channel V , ∀ P X , Q X , η KL ( V ) D ( P X || Q X ) ≥ D ( P X V || Q X V ) where η KL ( V ) – contraction coefficient. Relation to Erasure Channels [Polyanskiy-Wu 2016] : Definition: q -ary erasure channel q - EC (1 − η ) erases input w.p. 1 − η , and reproduces input w.p. η . Prop [Polyanskiy-Wu 2016] : q - EC (1 − η ) � ln V ⇔ ∀ P X , Q X , η D ( P X || Q X ) ≥ D ( P X V || Q X V ) . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 6 / 28

Motivation: Strong Data Processing Inequality Strong Data Processing Inequality [Ahlswede-G´ acs 1976] : For any channel V , ∀ P X , Q X , η KL ( V ) D ( P X || Q X ) ≥ D ( P X V || Q X V ) where η KL ( V ) – contraction coefficient. Relation to Erasure Channels [Polyanskiy-Wu 2016] : Definition: q -ary erasure channel q - EC (1 − η ) erases input w.p. 1 − η , and reproduces input w.p. η . Prop [Polyanskiy-Wu 2016] : q - EC (1 − η ) � ln V ⇔ ∀ P X , Q X , η D ( P X || Q X ) ≥ D ( P X V || Q X V ) . SDPI ⇔ � ln domination by erasure channel A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 6 / 28

1 − � 0 0 �/(� − 1) �/(� − 1) �/(� − 1) 1 − � 1 1 �/(� − 1) �/(� − 1) �/(� − 1) 1 − � � − 1 � − 1 Main Question Given channel V , find q -ary symmetric channel W δ � � 0 , q − 1 with largest δ ∈ such that W δ � ln V ? q A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 7 / 28

Main Question Given channel V , find q -ary symmetric channel W δ � � 0 , q − 1 with largest δ ∈ such that W δ � ln V ? q Definition ( q -ary Symmetric Channel) Channel matrix: 1 − � 0 0 �/(� − 1)   δ δ �/(� − 1) 1 − δ · · · �/(� − 1) q − 1 q − 1 1 − � 1 1 δ δ 1 − δ · · ·   q − 1 q − 1 W δ �   . . . ...   . . . �/(� − 1) . . .     δ δ · · · 1 − δ �/(� − 1) q − 1 q − 1 �/(� − 1) where δ ∈ [0 , 1] – total crossover probability. 1 − � � − 1 � − 1 A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 7 / 28

Main Question Given channel V , find q -ary symmetric channel W δ � � 0 , q − 1 with largest δ ∈ such that W δ � ln V ? q Definition ( q -ary Symmetric Channel) Channel matrix: 1 − � 0 0 �/(� − 1)   δ δ �/(� − 1) 1 − δ · · · �/(� − 1) q − 1 q − 1 1 − � 1 1 δ δ 1 − δ · · ·   q − 1 q − 1 W δ �   . . . ...   . . . �/(� − 1) . . .     δ δ · · · 1 − δ �/(� − 1) q − 1 q − 1 �/(� − 1) where δ ∈ [0 , 1] – total crossover probability. 1 − � � − 1 � − 1 For every channel V , W 0 � ln V and V � ln W ( q − 1) / q . A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 7 / 28

Main Question Given channel V , find q -ary symmetric channel W δ � � 0 , q − 1 with largest δ ∈ such that W δ � ln V ? q Definition ( q -ary Symmetric Channel) Channel matrix: 1 − � 0 0 �/(� − 1)   δ δ �/(� − 1) 1 − δ · · · �/(� − 1) q − 1 q − 1 1 − � 1 1 δ δ 1 − δ · · ·   q − 1 q − 1 W δ �   . . . ...   . . . �/(� − 1) . . .     δ δ · · · 1 − δ �/(� − 1) q − 1 q − 1 �/(� − 1) where δ ∈ [0 , 1] – total crossover probability. 1 − � � − 1 � − 1 For every channel V , W 0 � ln V and V � ln W ( q − 1) / q . ∀ ǫ, δ ∈ (0 , 1) , W δ �� ln q - EC ( ǫ ). A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 7 / 28

Less Noisy Channels in Networks Consider general Bayesian network: � � � � � � � � � � � � � � A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 8 / 28

Less Noisy Channels in Networks Consider general Bayesian network: � � � � � � � � � � � � � � Conjecture: Replace P Z 5 | Z 2 with less noisy channel ⇒ P Y | X becomes less noisy. A. Makur & Y. Polyanskiy (MIT) Symmetric Channel Domination 29 June 2017 8 / 28