sublinear algorithms
play

Sublinear Algorithms Lecture 6 Sofya Raskhodnikova Penn State - PowerPoint PPT Presentation

Sublinear Algorithms Lecture 6 Sofya Raskhodnikova Penn State University Thanks to Madhav Jha (Penn State) for help with creating these slides. 1 Communication Complexity A Method for Proving Lower Bounds [Blais Brody Matulef 11 ] Use known


  1. Sublinear Algorithms Lecture 6 Sofya Raskhodnikova Penn State University Thanks to Madhav Jha (Penn State) for help with creating these slides. 1

  2. Communication Complexity A Method for Proving Lower Bounds [Blais Brody Matulef 11 ] Use known lower bounds for other models of computation Partially based on slides by Eric Blais

  3. (Randomized) Communication Complexity π‘‡β„Žπ‘π‘ π‘“π‘’ π‘ π‘π‘œπ‘’π‘π‘› π‘‘π‘’π‘ π‘—π‘œπ‘• 1101000101110101110101010110 … Alice Bob 0100 11 001 β‹― 0011 π½π‘œπ‘žπ‘£π‘’: 𝑦 Input: 𝑧 Compute 𝐷 𝑦, 𝑧 Goal: minimize the number of bits exchanged. Communication complexity of a protocol is t he maximum number of bits β€’ exchanged by the protocol. Communication complexity of a function 𝐷 , denoted 𝑆(𝐷) , is the β€’ communication complexity of the best protocol for computing C. 3

  4. Example: Set Disjointness 𝐸𝐽𝑇𝐾 𝒍 1101000101110101110101010110 … Alice Bob π½π‘œπ‘žπ‘£π‘’: 𝑇 βŠ† [π‘œ] , 𝑇 = 𝑙 . Input: π‘ˆ βŠ† [π‘œ] , π‘ˆ = 𝑙 Compute 𝐸𝐽𝑇𝐾 𝑙 𝑇, π‘ˆ = 𝒃𝒅𝒅𝒇𝒒𝒖 if 𝑇 ∩ π‘ˆ = βˆ… π’”π’‡π’Œπ’‡π’…π’– otherwise Theorem [Hastad Wigderson 07 ] π‘œ 𝑆 DISJ 𝑙 β‰₯ Ξ© 𝑙 for all 𝑙 ≀ 2 . 4

  5. A lower bound using CC method Testing if a Boolean function is a k-parity

  6. Linear Functions Over Finite Field 𝔾 2 A Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} is linear (also called parity ) if 𝑔 𝑦 1 , … , 𝑦 π‘œ = 𝑏 1 𝑦 1 + β‹― + 𝑏 π‘œ 𝑦 π‘œ for some 𝑏 1 , … , 𝑏 π‘œ ∈ {0,1} no free term β€’ Work in finite field 𝔾 2 – Other accepted notation for 𝔾 2 : 𝐻𝐺 example 2 and β„€ 2 – Addition and multiplication is mod 2 001001 – π’š = 𝑦 1 , … , 𝑦 π‘œ , 𝒛 = 𝑧 1 , … , 𝑧 π‘œ , that is, π’š, 𝒛 ∈ 0,1 π‘œ + 011001 π’š + 𝒛 = 𝑦 1 + 𝑧 1 , … , 𝑦 π‘œ + 𝑧 π‘œ 010000 6

  7. Linear Functions Over Finite Field 𝔾 2 A Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} is linear (also called parity ) if 𝑔 𝑦 1 , … , 𝑦 π‘œ = 𝑏 1 𝑦 1 + β‹― + 𝑏 π‘œ 𝑦 π‘œ for some 𝑏 1 , … , 𝑏 π‘œ ∈ {0,1} ⇕ [π‘œ] is a shorthand for {1, … π‘œ} 𝑔 𝑦 1 , … , 𝑦 π‘œ = 𝑦 𝑗 for some 𝑇 βŠ† π‘œ . π‘—βˆˆS Notation: πœ“ 𝑇 𝑦 = 𝑦 𝑗 . π‘—βˆˆπ‘‡ 7

  8. Testing if a Boolean function is Linear Input: Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} Question: Is the function linear or 𝜁 -far from linear ( β‰₯ 𝜁2 π‘œ values need to be changed to make it linear)? Later in the course: 1 Famous BLR (Blum Lubi Rubinfeld 90) test runs in 𝑃 𝜁 time 8

  9. k-Parity Functions 𝑙 -Parity Functions A function 𝑔 ∢ 0,1 π‘œ β†’ {0,1} is a 𝑙 -parity if 𝑔 𝑦 = πœ“ 𝑇 𝑦 = 𝑦 𝑗 π‘—βˆˆπ‘‡ for some set 𝑇 βŠ† π‘œ of size 𝑇 = 𝑙 . 9

  10. Testing if a Boolean Function is a k-Parity Input: Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} and an integer 𝑙 Question: Is the function a 𝑙 -parity or 𝜁 -far from a 𝑙 -parity ( β‰₯ 𝜁2 π‘œ values need to be changed to make it a 𝑙 -parity)? Time: O 𝑙 log 𝑙 [Chakraborty Garciaβˆ’Soriano Matsliah] W min (𝑙, π‘œ βˆ’ 𝑙 ) [Blais Brody Matulef 11] β€’ Today: Ξ©(𝑙) for 𝑙 ≀ π‘œ/2 β€’ Today’s bound implies W min (𝑙, π‘œ βˆ’ 𝑙 ) 10

  11. Important Fact About Linear Functions Two different linear functions disagree on half of the values. Fact. β€’ Consider functions πœ“ 𝑇 and πœ“ π‘ˆ where 𝑇 β‰  π‘ˆ . 0 0 – Let 𝑗 be an element on which 𝑇 and π‘ˆ differ 1 1 (w.l.o.g. 𝑗 ∈ 𝑇 βˆ– π‘ˆ ) 1 0 – Pair up all π‘œ -bit strings: (π’š, π’š 𝑗 ) π’š 𝑏 𝑐 0 1 where π’š 𝑗 is π’š with the 𝑗 th bit flipped. β‹… β‹… – For each such pair, πœ“ 𝑇 (π’š) β‰  πœ“ 𝑇 (π’š 𝑗 ) β‹… β‹… but πœ“ π‘ˆ (π’š) = πœ“ π‘ˆ (π’š 𝑗 ) β‹… β‹… π’š 𝑗 1 βˆ’ 𝑏 So, πœ“ 𝑇 and πœ“ π‘ˆ differ on exactly one of π’š, π’š 𝑗 . 𝑐 0 0 – Since all π’š ’s are paired up, 1 0 πœ“ 𝑇 and πœ“ π‘ˆ differ on half of the values. 0 1 πœ“ 𝑇 (x) πœ“ π‘ˆ (x) A 𝑙 β€² - parity function, where 𝑙 β€² β‰  𝑙, is Β½-far from any k-parity. Corollary. 11

  12. Reduction from 𝐸𝐽𝑇𝐾 𝒍/πŸ‘ to Testing k-Parity β€’ Let π‘ˆ be the best tester for the 𝑙 -parity property for 𝜁 = 1/2 – query complexity of T is π‘Ÿ testing π‘™βˆ’parity . β€’ We will construct a communication protocol for 𝐸𝐽𝑇𝐾 𝒍/πŸ‘ that runs π‘ˆ and has communication complexity 2 β‹… π‘Ÿ (testing 𝑙 βˆ’parity) . holds for CC of every protocol for 𝐸𝐽𝑇𝐾 𝒍 [Hastad Wigderson 07] β€’ T hen 2 β‹… π‘Ÿ (testing 𝑙 βˆ’parity) β‰₯ 𝑆 DISJ 𝑙/2 β‰₯ Ξ© 𝑙/2 for 𝑙 ≀ π‘œ/2 ⇓ π‘Ÿ (testing 𝑙 -parity) β‰₯ Ξ© 𝑙 for 𝑙 ≀ π‘œ/2 12

  13. Reduction from 𝐸𝐽𝑇𝐾 𝒍/πŸ‘ to Testing k-Parity 1101000101110101110101010110 … β„Ž = 𝑔 + 𝑕 (𝑛𝑝𝑒 2) β„Ž 𝑦 ? 𝑔 𝑦 + 𝑕 𝑦 𝑛𝑝𝑒 2 Alice Bob T 𝒃𝒅𝒅𝒇𝒒𝒖/π’”π’‡π’Œπ’‡π’…π’– 𝑔(𝑦) 𝑕(𝑦) π½π‘œπ‘žπ‘£π‘’: 𝑇 βŠ† [π‘œ] , 𝑇 = 𝑙/2 . Input: π‘ˆ βŠ† [π‘œ] , π‘ˆ = 𝑙/2 Compute: 𝑔 = πœ“ 𝑇 Compute: 𝑕 = πœ“ π‘ˆ Output T’s answer β€’ π‘ˆ receives its random bits from the shared random string. 13

  14. Analysis of the Reduction Queries: Alice and Bob exchange 2 bits for every bit queried by π‘ˆ Correctness: β€’ β„Ž = 𝑔 + 𝑕 𝑛𝑝𝑒 2 = πœ“ 𝑇 + πœ“ π‘ˆ 𝑛𝑝𝑒 2 = πœ“ π‘‡Ξ”π‘ˆ β€’ π‘‡Ξ”π‘ˆ = 𝑇 + π‘ˆ βˆ’ 2 𝑇 ∩ π‘ˆ 𝑙 if S∩T = βˆ… β€’ SΞ”π‘ˆ = 𝑙 βˆ’ 2 if S∩T β‰  βˆ… ≀ β„Ž is π‘™βˆ’parity if S∩T = βˆ… 𝑙 β€² βˆ’parity where 𝑙 β€² β‰  𝑙 if S∩T β‰  βˆ… 1/2-far from every 𝑙 -parity Summary: π‘Ÿ (testing 𝑙 -parity) β‰₯ Ξ© 𝑙 for 𝑙 ≀ π‘œ/2 14

  15. Testing Lipschitz Property on Hypercube Lower Bound

  16. Lipschitz Property of Functions f : 0,1 π‘œ β†’ R 1 [Jha Raskhodnikova] 1 2 2 β€’ A function 𝑔 ∢ 0,1 π‘œ β†’ R is Lipschitz 1 1 0 if changing a bit of 𝑦 changes 𝑔(𝑦) by at most 1. 0 Lipschitz β€’ Is 𝑔 Lipschitz or 𝜁 -far from Lipschitz 0 ( 𝑔 has to change on many points to become Lipschitz)? – Edge 𝑦 βˆ’ 𝑧 is violated by 𝑔 if 𝑔 𝑦 βˆ’ 𝑔(𝑧) > 1 . 0 0 2 2 0 2 Time: – 𝑃(π‘œ 2 /𝜁) , logarithmic in the size of the input, 2 π‘œ 1 2 2 -far from Lipschitz – Ξ©(π‘œ) 16

  17. Testing Lipschitz Property Theorem Testing Lipschitz property of functions f: 0,1 π‘œ β†’ {0,1,2} requires Ξ©(π‘œ) queries. Prove it. 17

  18. Summary of Lower Bound Methods β€’ Yao’s Principle – testing membership in 1*, sortedness of a list and monotonicity of Boolean functions β€’ Reductions from communication complexity problems – testing if a Boolean function is a 𝑙 -parity 20

  19. Other Models of Sublinear Computation

  20. Tolerant Property Tester [Rubinfeld Parnas Ron] Randomized Algorithm Tolerant Property Tester YES YES Accept with Accept with probability β‰₯ πŸ‘/πŸ’ probability β‰₯ πŸ‘/πŸ’ 𝜺 -close to YES Don’t care NO Reject with 𝜻 -far from Reject with ο‚³ probability 2/3 ο‚³ probability 2/3 YES 22

  21. Sublinear- Time β€œRestoration” Models Local Decoding Input: A slightly corrupted codeword Requirement: Recover individual bits of the closest codeword with a constant number of queries per recovered bit. Program Checking Input: A program 𝑄 computing 𝑔 correctly on most 𝑄 inputs. 𝑔 Requirement: Self-correct program 𝑄 : for a given input 𝑦 , compute 𝑔(𝑦) by making a few calls to P . Local Reconstruction Input: Function 𝑔 nearly satisfying some property 𝑄 Requirement: Reconstruct function 𝑔 to ensure that the reconstructed function 𝑕 satisfies 𝑄 , changing 𝑔 𝑔 only when necessary. For each input 𝑦 , compute 𝑕(𝑦) with a few queries to 𝑔. 23

  22. Generalization: Local Computation [Rubinfeld Tamir Vardi Xie 2011] β€’ Compute the 𝑗 -th character 𝑧 𝑗 of a legal output 𝑧 . β€’ If there are several legal outputs for a given input, be consistent with one. β€’ Example: maximal independent set in a graph. 24

  23. Sublinear-Space Algorithms What if we cannot get a sublinear-time algorithm? Can we at least get sublinear space? Note: sublinear space is broader (for any algorithm, space complexity ≀ time complexity) 25

  24. Data Stream Model B L A - B L A - B L A - B L A - B L A - B L A - B L A - Streaming (1) Quickly process each element Algorithm (2) Limited working memory (3) Quickly produce output Motivation: internet traffic analysis Model the stream as 𝑛 elements from [π‘œ] , e.g., 𝑦 1 , 𝑦 2 , … , 𝑦 𝑛 = 3, 5, 3, 7, 5, 4, … Goal: Compute a function of the stream, e.g., median, number of distinct elements, longest increasing sequence. Based on Andrew McGregor’s slides: http://www.cs.umass.edu/~mcgregor/slides/10-jhu1.pdf 26

  25. Streaming Puzzle A stream contains π‘œ βˆ’ 1 distinct elements from π‘œ in arbitrary order. Problem: Find the missing element, using 𝑃(log π‘œ) space. 27

Recommend


More recommend