Two by Gelfand and Pinsker Amos Lapidoth ETH Zurich 2012 IEEE - PowerPoint PPT Presentation

Two by Gel’fand and Pinsker Amos Lapidoth ETH Zurich 2012 IEEE European School of Information Theory, Antalya, Turkey April 19, 2012 Joint work with Ligong Wang.

Two Results of Gel’fand and Pinsker from 1980

A Channel with Random Parameters • Channel law W ( y | x , s ) , { S k } ∼ IID P S . • The encoder knows the state sequence noncausally: f : M × S n → X n . • M is the message set 1 , . . . , 2 nR � � M = . • R is the rate, and n is the blocklength. • Decoder ignorant of state sequence: φ : Y n → M .

The highest rate of reliably communication Gel’fand and Pinsker: C = max I ( U ; Y ) − I ( U ; S ) where the maximum is over all PMFs P S ( s ) P U | S ( u | s ) P X | S , U ( x | s , u ) W ( y | x , s ) . And there is NLG in choosing P X | S , U deterministic: � � P S ( s ) P U | S ( u | s ) I x = g ( s , u ) W ( y | x , s ) C = P U | S , g : S×U→X I ( U ; Y ) − I ( U ; S ) max

Achievability R ) sequences IID P U : • Generate 2 n ( R +˜ � 1 , . . . , 2 n ˜ R � u ( m , ℓ ) , m ∈ M , ℓ ∈ . • To send Message m after observing s , look for some ℓ such � � that u ( m , ℓ ) , s are j.t. w.r.t. P S , U . • If none found, “encoding failure.” • The probability of encoding failure vanishes if ˜ R > I ( U ; S ) . • Decoder searches for a unique pair ( m ′ , ℓ ′ ) such that � u ( m ′ , ℓ ′ ) , y � is j.t. w.r.t. P U , Y . • The probability of success tends to one if R + ˜ R < I ( U ; Y ) .

The Converse nR ≤ I ( M ; Y n ) + n ǫ n � I ( M ; Y i | Y i − 1 ) + n ǫ n = i � � M , S n � Y i − 1 � � S n � M , Y i − 1 � � � � = I i +1 ; Y i − I i +1 ; Y i + n ǫ n i i � � Y i − 1 � � M , S n � Y i − 1 ; S i � � M , S n � � � = I i +1 ; Y i − I + n ǫ n i +1 i i � � Y i − 1 � � M , Y i − 1 , S n M , S n � � � � = I i +1 ; Y i − I i +1 ; S i + n ǫ n i i � M , Y i − 1 , S n � M , Y i − 1 , S n � � � � ≤ i +1 ; Y i − i +1 ; S i + n ǫ n I I i i � = I ( U i ; Y i ) − I ( U i ; S i ) + n ǫ n . i

It only remains to check that M , Y i − 1 , S n � � � � ⊸ − − X i , S i ⊸ − − Y i . i +1

What Is a Broadcast Channel? • One transmitter and two receivers. • Transmitted symbol: X ∈ X . • Received symbols: Y ∈ Y and Z ∈ Z . • Message m y ∈ M y for Receiver Y , and m z ∈ M z for Z . • Channel is used n times (“the blocklength”). • The rates are R y = log # M y R z = log # M z , . n n • The encoder: ∈ X n . � � � � �→ x ( m y , m z ) = x 1 ( m y , m z ) , . . . , x n ( m y , m z ) m y , m z • The decoders: φ y : Y n → M y , φ z : Z n → M z .

The Probability of Error A memoryless BC of law W ( y , z | x ): n � Pr[ Y = y , Z = z | X = x ] = W ( y k , z k | x k ) . k =1 The probabilities of error: 1 1 � � Pr[ φ y ( Y ) � = m y | M y = m y , M z = m z ] # M y # M z m y ∈M y m z ∈M z and 1 1 � � Pr[ φ z ( Z ) � = m z | M y = m y , M z = m z ] . # M y # M z m y ∈M y m z ∈M z

Capacity Region • ( R y , R z ) is achievable if for every ǫ > 0 and δ > 0 we are guaranteed that for all sufficiently large blocklengths n we can find encoder/decoders of rates ( R y − δ, R z − δ ) for which both error probabilities are smaller than ǫ . • Some special cases for which the capacity is known: • The degraded BC • Less Noisy • More capable • The deterministic BC • The semideterministic BC.

The Deterministic Broadcast Channel Y = f y ( X ) , Z = f z ( X ) for some f y : X → Y , f z : X → Z . Gel’fand, Marton, and Pinsker: The capacity region is the convex closure of the union over all PMFs P X of the (sets of) rate pairs R y ≤ H ( Y ) R z ≤ H ( Z ) R y + R z ≤ H ( Y , Z ) where the entropies are computed for the joint PMF � � � � P XYZ ( x , y , z ) = P X ( x ) 1 y = f y ( x ) z = f z ( x ) 1 .

The Converse for the Deterministic BC The converse is easy: n � I ( M y ; Y ) ≤ H ( Y k ) , k =1 n � I ( M z ; Z ) ≤ H ( Z k ) , k =1 and n � I ( M y , M z ) ≤ H ( Y k , Z k ) . k =1 To bound R y we ignore the fact that H ( Y | M y ) is typically not zero (because of M z ). Likewise for R z . And to bound R y + R z we pretend that the receivers can cooperate.

Deterministic BC—the Direct Part • Choose P X , inducing a joint P X P Y | X P Z | X of marginal P Y , Z . • In two independent assignments, assign to each y ∈ Y n a random index I ∈ { 1 , . . . , 2 nR y } and to each z ∈ Z n a random index J ∈ { 1 , . . . , 2 nR z } . • Let B ( i , j ) comprise the pairs ( y , z ) that are mapped to ( i , j ). • If ( y , z ) are jointly typical w.r.t. P Y , Z , then there must exist some x ∈ X n that produces the outputs ( y , z ), because joint typicality implies � � Pr[ Y = y , Z = z ] > 2 − n H ( Y , Z )+ ǫ > 0 , and the only way this probability can be positive is if some x induces these outputs. • To send ( m y , m z ) look for a pair ( y , z ) in B ( m y , m z ) that is jointly typical, and transmit the sequence x that produces it. • If there is no j.t. ( y , z ) in B ( m y , m z ), ⇒ “encoding failure.”

The Semideterministic Broadcast Channel Only Y is deterministic given x : Y = f y ( x ) , Pr[ Z = z | X = x ] = W ( z | x ) . Gel’fand and Pinsker: The capacity is the convex hull of the union over all P X of the sets of rate pairs ( R y , R z ) R y < H ( Y ) R z < I ( U ; Z ) R y + R z < H ( Y ) + I ( U ; Z ) − I ( U ; Y ) over all joint distribution on ( X , Y , Z , U ) under which, conditional on X , the channel outputs Y and Z are drawn according to the channel law independently of U : � � P XYZU ( x , y , z , u ) = P X , U ( x , u ) 1 y = f y ( x ) W ( z | x ) . Achievability follows from Marton’s Inner Bound (More later).

State-Dependence and Prescience • A state sequence S 1 , . . . , S n is generated IID ∼ P S . The channel law is W ( y , z | s , x ) . • A prescient encoder knows S 1 , . . . , S n before transmission begins: x = x ( m y , m z , s ) .

State-Dependence and Prescience • A state sequence S 1 , . . . , S n is generated IID ∼ P S . The channel law is W ( y , z | s , x ) . • A prescient encoder knows S 1 , . . . , S n before transmission begins: x = x ( m y , m z , s ) . At least as hard as the BC without a state. . . .

The Steinberg-Shamai Inner Bound Achievability of ( R 1 , R 2 ) is guaranteed whenever R 1 ≤ I ( U 0 , U 1 ; Y ) − I ( U 0 , U 1 ; S ) R 2 ≤ I ( U 0 , U 2 ; Z ) − I ( U 0 , U 2 ; S ) � + � R 1 + R 2 ≤ − max { I ( U 0 ; Y ) , I ( U 0 ; Z ) } − I ( U 0 ; S ) + I ( U 0 , U 1 ; Y ) − I ( U 0 , U 1 ; S ) + I ( U 0 , U 2 ; Z ) − I ( U 0 , U 2 ; S ) − I ( U 1 ; U 2 | U 0 , S ) , for some PMF of marginal P S ; that satisfies ( U 0 , U 1 , U 2 ) ⊸ − − ( X , S ) ⊸ − − ( Y , Z ); with the conditional of ( Y , Z ) given ( X , S ) being W ( y , z | x , s ).

The Semideterministic State-Dependent BC with a Prescient Transmitter • Y is a deterministic function of ( x , s ) but Z possibly not: Y = f ( s , x ) , Pr[ Z = z | X = x , S = s ] = W ( z | x , s ) . • The transmitter has noncausal state-information: � � � � m y , m z , s �→ x ( m y , m z , s ) = x 1 ( m y , m z , s ) , . . . , x n ( m y , m z , s ) .

Two Special Cases • State is null = ⇒ (classical) semideterministic BC. (Gel’fand and Pinsker’80b) .

Two Special Cases • State is null = ⇒ (classical) semideterministic BC. (Gel’fand and Pinsker’80b) . • Y is null = ⇒ the single-user “Gel’fand-Pinsker problem” (Gel’fand and Pinsker’80a) : C = max − Z I ( U ; Z ) − I ( U ; S ) U ⊸ − − ( X , S ) ⊸ − where the maximization is over PMFs of the form P S ( s ) P U | S ( u | s ) P X | S , U ( x | s , u ) W ( z | x , s ) , and P X | S , U can be taken to be deterministic.

Who Is S.I Gel’fand?

Who Is S.I Gel’fand? Sergey Israilevich Gel’fand. Ph.D. 1968 Moscow State Univeristy Supervisor: A. A. Kirillov. Israil Moiseevich Gel’fand (father)

The Main Result The capacity region is convex closure of the union of rate-pairs ( R y , R z ) satisfying R y < H ( Y | S ) R z < I ( U ; Z ) − I ( U ; S ) R y + R z < H ( Y | S ) + I ( U ; Z ) − I ( U ; S , Y ) over all joint distribution on ( X , Y , Z , S , U ) whose marginal P S is the given state distribution and under which, conditional on X and S , the channel outputs Y and Z are drawn according to the channel law independently of U : � � P XYZSU ( x , y , z , s , u ) = P S ( s ) P XU | S ( x , u | s ) 1 y = f ( x , s ) W ( z | x , s ) . Moreover, the capacity region is unchanged if the state sequence is revealed to the deterministic receiver.

If the State Is Null R y < H ( Y ✓ ❙ ✓ | S ) ❙ R z < I ( U ; Z ) − ✘✘✘✘ ✿ 0 I ( U ; S ) ✿ I ( U ; Y ) ✘ | S ) + I ( U ; Z ) − ✘✘✘✘✘ R y + R z < H ( Y ✓ ❙ ✓ I ( U ; S , Y ) ❙ ❍❍ ✟ s , u ) = ✟✟ SU ( x , y , z , ✁ y = f ( x , ✁ W ( z | x , ✁ � � P XYZ ✁ ❆ P S ( s ) P XU ✓ | S ( x , u | ✁ s ) 1 ❆ s ) ❆ ❆ s ) . ❍ ❆ ❙ That is, R y < H ( Y ) R z < I ( U ; Z ) R y + R z < H ( Y ) + I ( U ; Z ) − I ( U ; Y ) � � P XYZU ( x , y , z , u ) = P XU ( x , u ) 1 y = f ( x ) W ( z | x ) .