Fundamental Limits of Distributed Encoding Nastaran Abadi - PowerPoint PPT Presentation

Fundamental Limits of Distributed Encoding Nastaran Abadi Khooshemehr Mohammad Ali Maddah-Ali Sharif University of Technology International Symposium on Information Theory (ISIT) 2020 June 2020

Classical Coding Source Channel Hamming approach Shannon approach Adversarial errors Probabilistic errors 2

Some fundamental lim limit its on the parameters of codes Singleton bound If 𝐵 𝑟 (𝑜, 𝑒) is the maximum number of possible codewords in a 𝑟 -ary block code of length 𝑜 and minimum Gilbert – Varshamov bound distance 𝑒 , then 𝐵 𝑟 𝑜, 𝑒 ≤ 𝑟 𝑜−𝑒+1 . Hamming approach If 𝐵 𝑟 (𝑜, 𝑒) is the maximum number of Adversarial errors possible codewords in a 𝑟 -ary block code of length 𝑜 and minimum distance 𝑒 , then 𝑟 𝑜 𝐵 𝑟 𝑜, 𝑒 ≥ 𝑟−1 𝑘 . 𝑒−1 𝑜 σ 𝑘=0 𝑘 Griesmer bound If 𝑂(𝑙, 𝑒) is the minimum length of a binary code of dimension 𝑙 and and minimum 𝑒 𝑙−1 distance 𝑒 , then 𝑂 𝑙, 𝑒 ≥ σ 𝑗=0 2 𝑗 . and many more … 3

Let’s focus on the 4

A closer look at encoder Source Channel In some applications, the encoder can be distributed. 5

Example of applications with distributed data sources IoT Blockchain Shard 1 Shard 2 Shard 3 ⋮ In these systems, the encoding is distributed as well as the data production. 6

Distributed encoding Encoder Source node 1 Source node 2 Source node 3 7 distributed source nodes

Distributed encoding Source node 1 Decoder connects to some encoding nodes. Source Decoder node 2 Source node 3 8

Distributed encoding with adversaries Source node 1 Source node 2 Source node 3 9

Just one adversarial source node can undermine the system. Source node 1 More variables than equations Source node 2 Impossible to decode Source node 3 10

We study distributed encoding system, where some source nodes are controlled by an adversary. An adversarial node sends up to a finite number of different messages to the encoding nodes. We characterize the fundamental limit of this system. 11

Why do we assume an upper limit for the number of adversarial messages? The adversary cannot inject too many different messages into the system. There are methods to restrain the adversaries in distributed systems. 12

Objective in an adversarial distributed encoding system Decoding the messages of the honest nodes correctly. We do not care about the messages of the adversaries in decoding! 13

Distributed encoding system with adversaries No information about and . Source No information about the adversaries node 1 and their behavior. Source Decoder node 2 We need the decoder to decode the messages of the honest nodes correctly. Source node 3 We don’t care about the messages of adversaries. 14

System Parameters 𝑂 = 5 # of encoding nodes 𝐿 = 3 # of source nodes 𝑳 : the number of source nodes 𝛾 = 1 # of adversaries 𝑶 : the number of encoding nodes 𝑤 = 3 # of adversarial messages 𝜸 : the number of adversaries 𝒘 : the maximum number of the messages of one adversarial source node 𝒖 : the number of encoding nodes that decoder needs to connect to. 15

The problem What is the fundamental limit of 𝑢 in an (𝑂, 𝐿, 𝛾, 𝑤) distributed encoding system? (Informally, at least how many encoding nodes does the decoder need?) 𝑢 ∗ : fundamental limit of 𝑢 16

Fundamental limit of 𝑢 Theorem In an 𝑂, 𝐿, 𝛾, 𝑤 distributed encoding system, • if 𝑂 ≥ 𝐿 + 𝛾 𝑤 − 1 + 1 𝑢 ∗ = 𝐿 + 𝛾 𝑤 − 1 + 1 • If 𝐿 ≤ 𝑂 ≤ 𝐿 + 𝛾 𝑤 − 1 𝑢 ∗ = 𝑂 Recall 17

Proof Achievability There is a coding scheme where • the decoder can connect to any 𝑢 ∗ encoding nodes, • and generate an estimate for the input messages where the messages of the honest nodes are correctly decoded. For achievability, we need a code, decoding process, and correctness proof. Converse There is no coding scheme in which • the decoder connects to less than 𝑢 ∗ encoding nodes, • and estimates the messages of the honest nodes correctly. 18

Achievability-code We use this nonlinear code to achieve 𝑢 ∗ . nice structure 𝐿 𝑦 𝑜1 … 𝑦 𝑜𝐿 𝑔 𝑜 𝑦 𝑜1 , … , 𝑦 𝑜𝐿 = ෍ 𝛽 𝑜𝑙 , 1 ≤ 𝑜 ≤ 𝑂 𝑦 𝑜𝑙 𝑙=1 𝛽 𝑜1 , … , 𝛽 𝑜𝐿 : chosen independently and uniformly at random from the field Using nonlinear code • Hard for the adversary to evaluate the contribution of its messages in the encoded symbols • Hard for the adversary to cause confusion in the decoder 19 • Having a set of nonlinear equations with possibly many solutions

Achievability-code 𝐿 𝑦 𝑜1 … 𝑦 𝑜𝐿 𝑢 ∗ = 𝐿 + 𝛾 𝑤 − 1 + 1 𝑔 𝑜 𝑦 𝑜1 , … , 𝑦 𝑜𝐿 = ෍ 𝛽 𝑜𝑙 , 1 ≤ 𝑜 ≤ 𝑂 𝑦 𝑜𝑙 𝑙=1 𝐿 − 𝛾 + 𝛾𝑤 is the number of the variables in the system. With connecting to just one more encoding node and using the equation of that node, decoder can be successful. 20

Achievability-decoding Decoder considers every possible scenario and finds feasible solutions. 21

Achievability- correctness We prove every feasible solution satisfies correctness. We consider a partitioning for the encoding nodes. all options for the messages of source nodes We form a set of nonlinear equations. In some steps, we transform it to another set of feasible and undesirable solutions nonlinear equations. We use Bezout theorem to bound the number of the feasible and undesirable solutions. 22

Converse For any code, if the decoder connects to less than 𝑢 ∗ nodes, there is a way that adversary can mislead the decoder. The decoder does not know the adversaries and their behavior. Decoder would be confused between two contradicting feasible solutions. 23

Could we achieve 𝑢 ∗ with a linear code? 24

Fundamental limit of 𝑢 - linear regime Theorem (linear code) In an 𝑂, 𝐿, 𝛾, 𝑤 distributed encoding system where 𝑔 1 , … , 𝑔 𝑂 are linear functions, • if 𝑂 ≥ 𝐿 + 2𝛾 𝑤 − 1 ∗ 𝑢 linear = 𝐿 + 2𝛾 𝑤 − 1 • If 𝐿 ≤ 𝑂 ≤ 𝐿 + 2𝛾 𝑤 − 1 − 1 ∗ 𝑢 linear = 𝑂 Theorem (general code) In an 𝑂, 𝐿, 𝛾, 𝑤 distributed encoding system, Linear code is not good enough! • if 𝑂 ≥ 𝐿 + 𝛾 𝑤 − 1 + 1 𝑢 ∗ = 𝐿 + 𝛾 𝑤 − 1 + 1 • If 𝐿 ≤ 𝑂 ≤ 𝐿 + 𝛾 𝑤 − 1 𝑢 ∗ = 𝑂 25

Conclusion • We introduced the problem of distributed encoding. • We assumed that some of the source nodes are adversaries and send inconsistent messages to the encoding nodes. • We characterized the fundamental limit of the distributed encoding system. • We established matching achievability and converse. • We introduced nonlinear coding in order to achieve the fundamental limit. • There are many more problems to solve • How to optimize the decoding complexity? • What if some of encoding nodes are adversaries as well? • What is the fundamental limit if encoding nodes use a particular coding? • … 26

Thank you 27