Finite-Blocklength Performance of Sequential Transmission over BSC - PowerPoint PPT Presentation

Finite-Blocklength Performance of Sequential Transmission over BSC with Noiseless Feedback Hengjie Yang and Richard D. Wesel UCLA 2020 IEEE International Symposium on Information Theory (ISIT) Los Angeles, CA, USA H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 1 / 17

Outline Introduction 1 Our contributions 2 New non-asymptotic upper bound on average blocklength Markovian analysis Comparison of results Summary 3 H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 2 / 17

Motivation Stop feedback vs. Full feedback : X t Y t ˆ θ θ Encoder DMC Decoder ACK/NACK H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 3 / 17

Motivation Stop feedback vs. Full feedback : X t Y t ˆ θ θ Encoder DMC Decoder Y t − 1 H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 3 / 17

Motivation Stop feedback vs. Full feedback : X t Y t ˆ θ θ Encoder DMC Decoder Y t − 1 Two important literatures : • Polyanskiy et al. showed that the stop-feedback code can obtain a much higher achievable rate than fixed-length code without feedback. Y. Polyanskiy, H. V. Poor, and S. Verdu, “Channel coding rate in the finite blocklength regime,” IEEE Trans. Inf. Theory , vol. 56, no. 5, pp. 2307-2359, May 2010. H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 3 / 17

Motivation Stop feedback vs. Full feedback : X t Y t ˆ θ θ Encoder DMC Decoder Y t − 1 Two important literatures : • Polyanskiy et al. showed that the stop-feedback code can obtain a much higher achievable rate than fixed-length code without feedback. Y. Polyanskiy, H. V. Poor, and S. Verdu, “Channel coding rate in the finite blocklength regime,” IEEE Trans. Inf. Theory , vol. 56, no. 5, pp. 2307-2359, May 2010. • Naghshvar et al. proposed a novel scheme known as small-enough-difference (SED) encoder to attain the capacity and optimal error exponent of the BSC. M. Naghshvar, T. Javidi, and M. Wigger, Extrinsic Jensen-Shannon divergence: Applications to variable-length coding, IEEE Trans. Inf. Theory , vol. 61, no. 4, pp. 2148-2164, April 2015. H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 3 / 17

Motivation Stop feedback vs. Full feedback : X t Y t ˆ θ θ Encoder DMC Decoder Y t − 1 Two important literatures : • Polyanskiy et al. showed that the stop-feedback code can obtain a much higher achievable rate than fixed-length code without feedback. Y. Polyanskiy, H. V. Poor, and S. Verdu, “Channel coding rate in the finite blocklength regime,” IEEE Trans. Inf. Theory , vol. 56, no. 5, pp. 2307-2359, May 2010. • Naghshvar et al. proposed a novel scheme known as small-enough-difference (SED) encoder to attain the capacity and optimal error exponent of the BSC. M. Naghshvar, T. Javidi, and M. Wigger, Extrinsic Jensen-Shannon divergence: Applications to variable-length coding, IEEE Trans. Inf. Theory , vol. 61, no. 4, pp. 2148-2164, April 2015. Issue : The non-asymptotic upper bound on average blocklength of full-feedback codes by Naghshvar et al. is above that of stop-feedback codes by Polyanskiy. H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 3 / 17

Variable-length coding over the BSC ˆ θ X t Y t θ BSC ( p ) Encoder Decoder Y t − 1 System parameters : target prob. of error ǫ H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 4 / 17

Variable-length coding over the BSC ˆ θ X t Y t θ BSC ( p ) Encoder Decoder Y t − 1 System parameters : target prob. of error ǫ An ( M, ǫ ) variable-length code : 1 Message set Ω = { 1 , 2 , . . . , M } , H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 4 / 17

Variable-length coding over the BSC ˆ θ X t Y t θ BSC ( p ) Encoder Decoder Y t − 1 System parameters : target prob. of error ǫ An ( M, ǫ ) variable-length code : 1 Message set Ω = { 1 , 2 , . . . , M } , 2 Encoding function e t : Ω × Y t − 1 → X , X t = e t ( θ, Y t − 1 ) , t = 1 , 2 , . . . H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 4 / 17

Variable-length coding over the BSC ˆ θ X t Y t θ BSC ( p ) Encoder Decoder Y t − 1 System parameters : target prob. of error ǫ An ( M, ǫ ) variable-length code : 1 Message set Ω = { 1 , 2 , . . . , M } , 2 Encoding function e t : Ω × Y t − 1 → X , X t = e t ( θ, Y t − 1 ) , t = 1 , 2 , . . . 3 Decoding function d : Y τ → Ω , ˆ θ = d ( Y τ ) where τ is the random stopping time. H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 4 / 17

Variable-length coding over the BSC ˆ θ X t Y t θ BSC ( p ) Encoder Decoder Y t − 1 System parameters : target prob. of error ǫ An ( M, ǫ ) variable-length code : 1 Message set Ω = { 1 , 2 , . . . , M } , 2 Encoding function e t : Ω × Y t − 1 → X , X t = e t ( θ, Y t − 1 ) , t = 1 , 2 , . . . 3 Decoding function d : Y τ → Ω , ˆ θ = d ( Y τ ) where τ is the random stopping time. Goal : minimize τ,e t ( · ) ,d ( · ) E [ τ ] subject to P e � Pr { ˆ θ � = θ } ≤ ǫ . H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 4 / 17

Naghshvar et al.’s scheme Naghshvar et al. proposed the following scheme • The belief state : ρ ( t ) � [ ρ 1 ( t ) , ρ 2 ( t ) , . . . , ρ M ( t )] , t = 1 , 2 , . . . where ρ i ( t ) � Pr { θ = i | Y t } , i ∈ Ω and ρ i (0) = 1 /M by default. H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 5 / 17

Naghshvar et al.’s scheme Naghshvar et al. proposed the following scheme • The belief state : ρ ( t ) � [ ρ 1 ( t ) , ρ 2 ( t ) , . . . , ρ M ( t )] , t = 1 , 2 , . . . where ρ i ( t ) � Pr { θ = i | Y t } , i ∈ Ω and ρ i (0) = 1 /M by default. • The SED encoding rule : at time t , upon receiving y t − 1 thanks to the noiseless feedback, update ρ ( t − 1) by Bayes rule. Then partition Ω into two subsets S 0 ( t − 1) and S 1 ( t − 1) such that 0 ≤ π 0 ( t − 1) − π 1 ( t − 1) ≤ i ∈ S 0 ( t − 1) ρ i ( t − 1) , min where � � π 0 ( t − 1) � π 1 ( t − 1) � ρ i ( t − 1) , ρ i ( t − 1) . i ∈ S 0 ( t − 1) i ∈ S 1 ( t − 1) H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 5 / 17

Naghshvar et al.’s scheme Naghshvar et al. proposed the following scheme • The belief state : ρ ( t ) � [ ρ 1 ( t ) , ρ 2 ( t ) , . . . , ρ M ( t )] , t = 1 , 2 , . . . where ρ i ( t ) � Pr { θ = i | Y t } , i ∈ Ω and ρ i (0) = 1 /M by default. • The SED encoding rule : at time t , upon receiving y t − 1 thanks to the noiseless feedback, update ρ ( t − 1) by Bayes rule. Then partition Ω into two subsets S 0 ( t − 1) and S 1 ( t − 1) such that 0 ≤ π 0 ( t − 1) − π 1 ( t − 1) ≤ i ∈ S 0 ( t − 1) ρ i ( t − 1) , min where � � π 0 ( t − 1) � π 1 ( t − 1) � ρ i ( t − 1) , ρ i ( t − 1) . i ∈ S 0 ( t − 1) i ∈ S 1 ( t − 1) • Decoding (or stopping) rule : τ = min { t : max i ∈ Ω ρ i ( t ) ≥ 1 − ǫ } ˆ θ = arg max ρ i ( τ ) . i ∈ Ω H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 5 / 17

Two phases in the variable-length coding problem ∃ j ∈ Ω , ρ j ( t ) ≥ 1 / 2 Communication Phase Confirmation Phase ∀ i ∈ Ω , ρ i ( t ) < 1 / 2 Assume θ = i ∈ Ω henceforth. Communication phase : ∀ j ∈ Ω , ρ j ( t ) < 1 / 2 . Confirmation phase : ∃ j ∈ Ω , ρ j ( t ) ≥ 1 / 2 . H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 6 / 17

Average step size provided by the SED encoder Consider the log-likelihood ratio of the true message θ = i ∈ Ω , ρ i ( t ) U i ( t ) � log 1 − ρ i ( t ) . H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 7 / 17

Average step size provided by the SED encoder Consider the log-likelihood ratio of the true message θ = i ∈ Ω , ρ i ( t ) U i ( t ) � log 1 − ρ i ( t ) . Lemma (Naghshvar et al ., 2012) With the SED encoding rule over the BSC ( p ) , { U i ( t ) } ∞ t =1 forms a submartingale w.r.t. the filtration F t = σ { Y t } , satisfying E [ U i ( t + 1) |F t ] ≥ U i ( t ) + C, if U i ( t ) < 0 E [ U i ( t + 1) |F t ] = U i ( t ) + C 1 , if U i ( t ) ≥ 0 | U i ( t + 1) − U i ( t ) | ≤ C 2 where C = max P X I ( X ; Y ) = 1 − H ( p ) , = p log p q + q log q � P Y | X = x � P Y | X = x ′ � ( q � 1 − p ) C 1 = max x,x ′ ∈X D p, y ∈Y log max x ∈X P Y | X ( y | x ) min x ∈X P Y | X ( y | x ) = log q C 2 = max p. H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 7 / 17

Previous non-asymptotic results Theorem (Naghshvar et al. , 2015) Naghshvar et al.’s scheme for symmetric binary-input channels satisfies + log 1 E [ τ ] ≤ log M + log log M + 96 · 2 2 C 2 ǫ + 1 ǫ . C C 1 CC 1 H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 8 / 17

Previous non-asymptotic results Theorem (Naghshvar et al. , 2015) Naghshvar et al.’s scheme for symmetric binary-input channels satisfies + log 1 E [ τ ] ≤ log M + log log M + 96 · 2 2 C 2 ǫ + 1 ǫ . C C 1 CC 1 Theorem (Polyanskiy’s VLF bound, Williamson et al. , 2015) For a given ǫ > 0 and a positive integer M , there exists a stop-feedback VLF code for BSC ( p ) , satisfying + log 1 E [ τ ] ≤ log( M − 1) + 2(1 − p ) ǫ . C C C H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 8 / 17

An illustration of bounds: BSC (0 . 05) 0 . 8 Capacity of BSC (0 . 05) 0 . 7 0 . 6 0 . 5 Rate 0 . 4 0 . 3 0 . 2 0 . 1 0 0 0 . 5 1 1 . 5 2 Average Blocklength · 10 4 H. Yang (UCLA) ISIT 2020, Los Angeles, CA, USA June 2020 9 / 17