information capacity of the bsc permutation channel
play

Information Capacity of the BSC Permutation Channel Anuran Makur - PowerPoint PPT Presentation

Feb 21, 2023 •119 likes •1.14k views

Information Capacity of the BSC Permutation Channel Anuran Makur EECS Department, Massachusetts Institute of Technology Allerton Conference 2018 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 1 / 21 Outline Introduction

  1. Information Capacity of the BSC Permutation Channel Anuran Makur EECS Department, Massachusetts Institute of Technology Allerton Conference 2018 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 1 / 21

  2. Outline Introduction 1 Motivation: Coding for Communication Networks The Permutation Channel Model Capacity of the BSC Permutation Channel Achievability 2 Converse 3 Conclusion 4 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 2 / 21

  3. Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

  4. Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

  5. Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: 2 L input symbols Alphabet symbols = all possible L -bit packets ⇒ A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

  6. Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: Alphabet symbols = all possible L -bit packets multipath routed network or evolving network topology A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

  7. Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: Alphabet symbols = all possible L -bit packets multipath routed network ⇒ packets received with transpositions A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

  8. Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: Alphabet symbols = all possible L -bit packets multipath routed network ⇒ packets received with transpositions packets are impaired (e.g. deletions, substitutions) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

  9. Motivation: Point-to-point Communication in Networks SENDER RECEIVER NETWORK Model communication network as a channel: Alphabet symbols = all possible L -bit packets multipath routed network ⇒ packets received with transpositions packets are impaired ⇒ model using channel probabilities A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 3 / 21

  10. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER NETWORK A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  11. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  12. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets Random deletion channel: Delete each symbol/packet of codeword independently with probability p ∈ (0 , 1) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  13. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets Random deletion channel: Delete each symbol/packet of codeword independently with probability p ∈ (0 , 1) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  14. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets Random deletion channel: Delete each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  15. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER RANDOM RANDOM DELETION PERMUTATION NETWORK Abstraction: n -length codeword = sequence of n packets Random deletion channel: Delete each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  16. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION ? ? NETWORK Abstraction: n -length codeword = sequence of n packets Equivalent Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  17. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION 1 ? 3 1 2 3 3 1 ? NETWORK Abstraction: n -length codeword = sequence of n packets Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword Coding: Add sequence numbers (packet size = L + log( n ) bits, alphabet size = n 2 L ) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  18. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION 1 ? 3 1 2 3 3 1 ? NETWORK Abstraction: n -length codeword = sequence of n packets Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword Coding: Add sequence numbers and use standard coding techniques A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  19. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION 1 ? 3 1 2 3 3 1 ? NETWORK Abstraction: n -length codeword = sequence of n packets Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword Coding: Add sequence numbers and use standard coding techniques More refined coding techniques simulate sequence numbers, e.g. [Mitzenmacher 2006], [Metzner 2009] A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  20. Example: Coding for Random Deletion Network Consider a communication network where packets can be dropped: SENDER RECEIVER ERASURE RANDOM CHANNEL PERMUTATION ? ? NETWORK Abstraction: n -length codeword = sequence of n packets Erasure channel: Erase each symbol/packet of codeword independently with probability p ∈ (0 , 1) Random permutation block: Randomly permute packets of codeword How do you code in such channels without increasing alphabet size? A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 4 / 21

  21. The Permutation Channel Model � � � � 𝑁 𝑌 � 𝑎 � 𝑍 𝑁 RANDOM � CHANNEL ENCODER DECODER PERMUTATION Sender sends message M ∼ Uniform( M ) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 5 / 21

  22. The Permutation Channel Model � � � � 𝑁 𝑌 � 𝑎 � 𝑍 𝑁 RANDOM � CHANNEL ENCODER DECODER PERMUTATION Sender sends message M ∼ Uniform( M ) Possibly randomized encoder f n : M → X n produces codeword X n 1 = ( X 1 , . . . , X n ) = f n ( M ) (with block-length n ) A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 5 / 21

  23. The Permutation Channel Model � � � � 𝑁 𝑌 � 𝑎 � 𝑍 𝑁 RANDOM � CHANNEL ENCODER DECODER PERMUTATION Sender sends message M ∼ Uniform( M ) Possibly randomized encoder f n : M → X n produces codeword X n 1 = ( X 1 , . . . , X n ) = f n ( M ) (with block-length n ) Discrete memoryless channel P Z | X with input and output alphabets X and Y produces Z n 1 : n � 1 ( z n 1 | x n P Z n 1 ) = P Z | X ( z i | x i ) 1 | X n i =1 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 5 / 21

  24. The Permutation Channel Model � � � � 𝑁 𝑌 � 𝑎 � 𝑍 𝑁 RANDOM � CHANNEL ENCODER DECODER PERMUTATION Sender sends message M ∼ Uniform( M ) Possibly randomized encoder f n : M → X n produces codeword X n 1 = ( X 1 , . . . , X n ) = f n ( M ) (with block-length n ) Discrete memoryless channel P Z | X with input and output alphabets X and Y produces Z n 1 : n � 1 ( z n 1 | x n P Z n 1 ) = P Z | X ( z i | x i ) 1 | X n i =1 Random permutation generates Y n 1 from Z n 1 A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 5 / 21

Recommend

Bounds on Permutation Channel Capacity Anuran Makur Department of Electrical Engineering and

Bounds on Permutation Channel Capacity Anuran Makur Department of Electrical Engineering and

Bounds on Permutation Channel Capacity Anuran Makur Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology IEEE International Symposium on Information Theory 2020 Anuran Makur (MIT) Bounds on

1.47k views • 87 slides
Information Theory Lecture 10 Network Information Theory (CT15); a focus on channel capacity

Information Theory Lecture 10 Network Information Theory (CT15); a focus on channel capacity

Information Theory Lecture 10 Network Information Theory (CT15); a focus on channel capacity results The (two-user) multiple access channel (15.3) The (two-user) broadcast channel (15.6) The relay channel (15.7) Some remarks on

598 views • 13 slides
Part V. AWGN Channel Capacity AWGN Capacity Formula; Sphere Packing; Resources in AWGN Channel

Part V. AWGN Channel Capacity AWGN Capacity Formula; Sphere Packing; Resources in AWGN Channel

Part V. AWGN Channel Capacity AWGN Capacity Formula; Sphere Packing; Resources in AWGN Channel 61 Channel coding theorem For every memoryless channel, there is a definite number C (computable) such that: If the data rate R < C ,

910 views • 12 slides
Lecture 1 Capacity of the Gaussian Channel Basic concepts in information theory: Appendix B

Lecture 1 Capacity of the Gaussian Channel Basic concepts in information theory: Appendix B

Lecture 1 Capacity of the Gaussian Channel Basic concepts in information theory: Appendix B Capacity of the Gaussian channel: Appendix B, Ch. 5.13 Mikael Skoglund, Theoretical Foundations of Wireless 1/23 Entropy and Mutual

307 views • 12 slides
A General Formula for Channel Capacity 1 Definitions Information variable { 1 , . . .

A General Formula for Channel Capacity 1 Definitions Information variable { 1 , . . .

A General Formula for Channel Capacity 1 Definitions Information variable { 1 , . . . , M } , p ( i ) = Pr( = i ) Channel input X X and output Y Y , finite alphabets Codewords { x N 1 ( i ) : i = 1 , . . . , M } , x n

198 views • 6 slides
Chapter 7 Channel Capacity Peng-Hua Wang Graduate Inst. of Comm. Engineering National Taipei

Chapter 7 Channel Capacity Peng-Hua Wang Graduate Inst. of Comm. Engineering National Taipei

Chapter 7 Channel Capacity Peng-Hua Wang Graduate Inst. of Comm. Engineering National Taipei University Chapter Outline Chap. 7 Channel Capacity 7.1 Examples of Channel Capacity 7.2 Symmetric Channels 7.3 Properties of Channel Capacity 7.4

1.2k views • 62 slides
Achieving channel capacity ... Shannon says this is possible ... how ? Make use of the gap

Achieving channel capacity ... Shannon says this is possible ... how ? Make use of the gap

Introduction Turbo Principle SISO (Soft Input Soft Output) Example of a product code Achieving channel capacity ... Shannon says this is possible ... how ? Make use of the gap between the source rate and the channel capacity: coding scheme

729 views • 14 slides
Computing and Communications 2. Information Theory -Channel Capacity Ying Cui Department of

Computing and Communications 2. Information Theory -Channel Capacity Ying Cui Department of

1896 1920 1987 2006 Computing and Communications 2. Information Theory -Channel Capacity Ying Cui Department of Electronic Engineering Shanghai Jiao Tong University, China 2017, Autumn 1 Outline Communication system Examples of

512 views • 36 slides
2. Information Theory -Channel Capacity Ying Cui Department of Electronic Engineering Shanghai

2. Information Theory -Channel Capacity Ying Cui Department of Electronic Engineering Shanghai

1896 1920 1987 2006 Computing and Communications 2. Information Theory -Channel Capacity Ying Cui Department of Electronic Engineering Shanghai Jiao Tong University, China 2018, Autumn 1 Outline Communication system Examples of

756 views • 36 slides
ECS 289M Lecture 23 May 24, 2006 Measuring Capacity Intuitively, difference between

ECS 289M Lecture 23 May 24, 2006 Measuring Capacity Intuitively, difference between

ECS 289M Lecture 23 May 24, 2006 Measuring Capacity Intuitively, difference between unmodulated, modulated channel Normal uncertainty in channel is 8 bits Attacker modulates channel to send information, reducing uncertainty to 5

611 views • 20 slides
Further Properties of Wireless Channel Capacity Fengyou Sun and Yuming Jiang Norwegian University

Further Properties of Wireless Channel Capacity Fengyou Sun and Yuming Jiang Norwegian University

Wireless Channel Capacity Analysis of Cumulative Capacity Analysis of Maximum and Minimum Cumulative Capacity Further Properties of Wireless Channel Capacity Fengyou Sun and Yuming Jiang Norwegian University of Science and Technology (NTNU)

260 views • 22 slides
Compound channel with feedback: Opportunistic capacity and error exponents Aditya Mahajan and

Compound channel with feedback: Opportunistic capacity and error exponents Aditya Mahajan and

Compound channel with feedback: Opportunistic capacity and error exponents Aditya Mahajan and Sekhar Tatikonda Yale University March 17, 2010 CISS Compound channel Channel model | , =

791 views • 57 slides
Lecture 5 Channel Coding over Continuous Channels I-Hsiang Wang Department of Electrical

Lecture 5 Channel Coding over Continuous Channels I-Hsiang Wang Department of Electrical

Mutual Information and Differential Entropy Channel Coding with Input Cost Gaussian Channel Capacity Summary Lecture 5 Channel Coding over Continuous Channels I-Hsiang Wang Department of Electrical Engineering National Taiwan University

600 views • 34 slides
Previously... Joint typical sequences Covering and Packing Lemmas Channel Coding Theorem

Previously... Joint typical sequences Covering and Packing Lemmas Channel Coding Theorem

Lecture 12 Review Previously... Joint typical sequences Covering and Packing Lemmas Channel Coding Theorem Capacity of Gaussian channel Capacity of additive white Gaussian channel Forward proof of Channel Coding Theorem S. Cheng (OU-Tulsa)

2.44k views • 134 slides
Discrete Memoryless Channel y x Discrete Channel W Input alphabet y

Discrete Memoryless Channel y x Discrete Channel W Input alphabet y

Polar Codes: Speed of Polarization & Polynomial Gap to Capacity Venkatesan Guruswami Carnegie Mellon University (currently visiting Microsoft Research New England) Based on joint work with Patrick Xia Charles River Science of Information

798 views • 32 slides
Lecture 2: Wireless Physical Lecture 2: Wireless Physical Layer, Channel, and Capacity Layer,

Lecture 2: Wireless Physical Lecture 2: Wireless Physical Layer, Channel, and Capacity Layer,

Lecture 2: Wireless Physical Lecture 2: Wireless Physical Layer, Channel, and Capacity Layer, Channel, and Capacity Mythili Vutukuru CS 653 Spring 2014 Jan 9, Thursday Radio waves and spectrum Physical layer of mobile systems wireless

327 views • 20 slides
Capacity of a Broadcast Channel with Luo, Gohary, Yanikomeroglu Gaussian Jamming and a Friendly

Capacity of a Broadcast Channel with Luo, Gohary, Yanikomeroglu Gaussian Jamming and a Friendly

Capacity of a Broadcast Channel with Gaussian Jamming and a Friendly Eavesdropper 1 Capacity of a Broadcast Channel with Luo, Gohary, Yanikomeroglu Gaussian Jamming and a Friendly Channel Eavesdropper Model Relaying Schemes Kevin

507 views • 18 slides
Information Theory Lecture 4 Discrete channels, codes and capacity: CT7 Channels:

Information Theory Lecture 4 Discrete channels, codes and capacity: CT7 Channels:

Information Theory Lecture 4 Discrete channels, codes and capacity: CT7 Channels: CT7.12 Capacity and the coding theorem: CT7.37 and CT7.9 Combining source and channel coding: CT7.13 Mikael Skoglund, Information Theory 1/19

310 views • 10 slides
Permutation-based decoding of Reed-Muller codes in binary erasure channel Kirill Ivanov, R

Permutation-based decoding of Reed-Muller codes in binary erasure channel Kirill Ivanov, R

Permutation-based decoding of Reed-Muller codes in binary erasure channel Kirill Ivanov, R udiger Urbanke Ecole Polytechnique F ed erale de Lausanne, Switzerland July 12, 2018 kirill.ivanov@epfl.ch (EPFL) Decoding of RM codes in

235 views • 6 slides
Achieving Channel Capacity With Lattice Codes: From Fermat to Shannon Cong Ling Imperial College

Achieving Channel Capacity With Lattice Codes: From Fermat to Shannon Cong Ling Imperial College

Outline Statement of Coding Problems in Information Theory Lattices and Algebraic Number Theory Coding for Gaussian, Fading Achieving Channel Capacity With Lattice Codes: From Fermat to Shannon Cong Ling Imperial College London

928 views • 60 slides
The Approximate Sum Capacity of the Symmetric Gaussian K -User Interference Channel Or Ordentlich

The Approximate Sum Capacity of the Symmetric Gaussian K -User Interference Channel Or Ordentlich

The Approximate Sum Capacity of the Symmetric Gaussian K -User Interference Channel Or Ordentlich Joint work with Uri Erez and Bobak Nazer July 5th, ISIT 2012 MIT, Cambridge, Massachusetts Ordentlich, Erez, Nazer Approx. Sum Capacity of the

746 views • 59 slides
Capacity Region of the Gaussian Arbitrarily-Varying Broadcast Channel Fatemeh Hosseinigoki and

Capacity Region of the Gaussian Arbitrarily-Varying Broadcast Channel Fatemeh Hosseinigoki and

Capacity Region of the Gaussian Arbitrarily-Varying Broadcast Channel Fatemeh Hosseinigoki and Oliver Kosut A RIZONA S TATE U NIVERSITY ISIT 2020 1 / 12 Gaussian Arbitrarily-Varying Broadcast Channel Transmitter power constraint X 2

557 views • 44 slides
An example of a non-Abelian group code achieving channel capacity Jorge P. Arpasi Universidade

An example of a non-Abelian group code achieving channel capacity Jorge P. Arpasi Universidade

An example of a non-Abelian group code achieving channel capacity Jorge P. Arpasi Universidade Federal do Pampa - UNIPAMPA September 2018 Group Codes Given a channel ( X , Y , p ( y | x )) and a group G matched to X , a group code C is a

377 views • 12 slides
Multiuser Channel Capacity Mohammad Rezaeian Research supervisor Dr. Alex Grant Institute for

Multiuser Channel Capacity Mohammad Rezaeian Research supervisor Dr. Alex Grant Institute for

Multiuser Channel Capacity Mohammad Rezaeian Research supervisor Dr. Alex Grant Institute for Telecommunications Research University of South Australia April 10, 2002 Overview Multiuser channels Capacity for

717 views • 23 slides

More recommend