Function Computation in Networked Environments Vinay A. Vaishampayan City University of New York July 25, 2018 Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 1 / 57
Outline: I While physical layer communication technologies, especially in optical and wireless communication continue to be of great importance, the internet has brought a lot of importance to application layer systems and application layer performance. Today, many application are distributed in some sense, e.g. in a data center a single machine is not sufficient to handle an application, and in a wide area network, many geographically separated nodes need to collaborate. Thus it becomes important to understand the communication needs of specific applications that are to be implemented in a distributed manner. Information theoretically, a good starting point to develop this understanding is through the problems of distributed function computation and communication complexity. Problem originally formulated in late 1970’s. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 2 / 57
Outline: II This talk will serve as an introduction, describe some applications, and some of the authors own research. Organization of the talk: ◮ Problem Description and Formulation ◮ Some Theory ◮ Some Applications ◮ Specific Research Problem: Nearest Lattice Point Search ◮ Summary Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 3 / 57
Part I: Problem Definition Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 4 / 57
Communication Complexity of Distributed Function Computation xn x₂ x₁ Nodes+at+which+f+is+required x₃ Given function f : X 1 × X 2 . . . × X n → Z . Observations x 1 ∈ X i , i = 1 , 2 , . . . , n are available at physically separated locations in a network. Compute f ( x 1 , x 2 , . . . , x n ). Function value should be available at designated nodes. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 5 / 57
Distributed Function Computation: Problem Definition Communication is carried out using a pre-arranged protocol, Π R (Π): Number of bits communicated by participants in Π for computing f ( x 1 , x 2 , . . . , x n ). Communication complexity of f : C ( f ) = min R (Π) Π that compute f Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 6 / 57
Terminology party ≡ sensor. n = 2: two-party. n > 2: multiparty. ◮ For 2-party: X 1 = X , X 2 = Y . Where is the result required? ◮ At a single location (fusion center): Centralized . ◮ At all sensor nodes: Distributed Nature of the protocol: Non-interactive/ Interactive How is the rate measured? Worst-case, average Exact computation or approximate computation. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 7 / 57
Interactive Communication: Problem Setup ∗ A. C. Yao, Some Complexity Questions Related to Distributive Computing, ACM 1979. g(X,Y) g(X,Y) Alice has x , Bob has y , both wish to Enc compute g ( x , y ). Dec mN=f(X,m1,m2,...,mN-1) Communication in rounds following a Enc Time Dec pre-decided protocol. Enc m3=f(X,m1,m2) (R3 bits) Dec Definition Enc Dec m2=f(Y,m1) (R2 bits) Two-way communication complexity is the Enc minimum number of bits that must be Dec Enc m1=f(X) (R1 bits) exchanged for the worst-case input so that Dec Alice and Bob can compute f ( x , y ) X Y Alice Bob Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 8 / 57
Example: [Yao 1979] f A B C a 0 1 0 b 0 0 0 { a } { b } { c } { d,e } c 1 1 1 { A,C } { B } d 0 0 1 { A,B } { C } e 1 1 0 { d } { e } { d } { e } Interactive protocol, distributed. Worst case inputs require 4 bits. Average rate: 9 / 5 bits. Assumes equally likely and independent inputs. Observation: function must be constant on every leaf of the tree. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 9 / 57
Part II: Theory Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 10 / 57
Basic Results C ( f ) ≤ min(log 2 |X| , log 2 |Y| ) + log 2 | Range( f ) | ◮ Proof: Alice sends x ∈ X to Bob using log 2 |X| bits. Bob sends back f ( x , y ) using log 2 | Range( f ) | bits. ◮ This is the ‘obvious’ method. Idea is to improve on this. C ( f ) ≥ log 2 | Range( f ) | ◮ Proof: | Range( f ) | is the number of leaves of the code tree. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 11 / 57
Rectangles Definition (Rectangle) R = A × B : A ⊂ X , B ⊂ Y Definition (Monochromatic Rectangle) Rectangle R such that f ( x , y ) is constant on R . Leaf nodes of a protocol Π that compute f are monochromatic rectangles. Any protocol Π that computes f induces a partition of X × Y into monochromatic rectangles. Theorem If any monochromatic rectangular partition of X × Y has at least T rectangles, then C ( f ) ≥ log 2 T. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 12 / 57
Monochromatic Rectangles due to Algorithm Π Algorithm Π { a } { b } { c } { d,e } { A,C } { B } { A,B } { C } { d } { e } { d } { e } Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 13 / 57
Basic Results [KN:1997] Definition (Fooling Set S ) S ⊂ X × Y f ( x , y ) = z , ( x , y ) ∈ S . For each distinct pair ( x 1 , y 1 ) , ( x 2 , y 2 ) ∈ S , either f ( x 1 , y 2 ) � = z or f ( x 2 , y 1 ) � = z . Theorem If a function f has a fooling set of size T then C ( f ) ≥ log 2 T. Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 14 / 57
Rank Lower Bound Mehlhorn and Schmidt, “Las Vegas is Better than Determinism in VLSI and Distributed computing,” STOC, 1982. f A B C a 1 1 0 { a,b } { c,d } b 1 1 1 c 0 1 1 { A,B } { C } { A } { B,C } d 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 1 0 0 0 = + + 0 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 � �� � � �� � � �� � � �� � M f M 1 M 2 M 3 rank ( M f ) ≤ rank ( M 1 ) + rank ( M 2 ) + rank ( M 3 ) ≤ T ���� # leaves Theorem C ( f ) ≥ log 2 rank ( M f ) Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 15 / 57
Information Theory: Distributed Function Computation J. Korner and K. Marton, How to Encode the Modulo-2 Sum of Binary Sources, IEEE IT 1979 X Rate=Rg1 bits/sample Encoder g(X,Y) 1 Decoder Encoder 2 Y Rate=Rg2 bits/sample Fusion Center Pair of random variables X , Y . Function g : X × Y → R . Reproduce g ( X , Y ) exactly in F . Rg 1 > I ( U ; X | V ), Rg 2 > I ( V ; Y | U ), Rg 1 + Rg 2 > I ( UV , XY ). U , V auxiliary random variables that satisfy U − X − Y − V and H ( g ( X , Y ) | UV ) = 0. A graph-theoretic characterization based on maximally independent sets of a graph determined by g is in A. Orlitsky and J. Roche, Coding for Computing, IEEE IT, 2001 Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 16 / 57
Interactive Communication: Rate Region N. Ma and S. Ishwar, “Some Results on Distributed Source Coding for Interactive Function Computation,” IEEE IT 2011. R 1 ≥ I ( X ; U 1 | Y ) , U 1 − X − Y g(X,Y) g(X,Y) Enc R 2 ≥ I ( Y ; U 2 | X , U 1 ) , U 2 − ( Y , U 1 ) − X mN=f(X,m1,m2,...,mN-1) Dec Enc R 3 ≥ I ( X ; U 3 | Y , U 1 , U 2 ) , U 3 − ( X , U 1 , U 2 ) − Y Time Dec . . Enc m3=f(X,m1,m2) . . . . (R3 bits) Dec Enc Dec m2=f(Y,m1) (R2 bits) R 2 n ≥ I ( Y ; U 2 n | X , U 1 , . . . , U 2 n − 1 ) Enc Dec U 2 n − ( X , U 1 , U 2 , ..., U 2 n − 1 ) − Y Enc m1=f(X) (R1 bits) Dec H ( G | X , U 2 n 1 ) = 0 , H ( G | Y , U 2 n 1 ) = 0 X Y Alice Bob Vinay A. Vaishampayan (City University of New York) Function Computation in Networked Environments July 25, 2018 17 / 57
Recommend
More recommend
