Polynomial-Time Problems Lecture 3: The polynomial method Part I: - PowerPoint PPT Presentation

Complexity Theory of Polynomial-Time Problems Lecture 3: The polynomial method Part I: Orthogonal Vectors Sebastian Krinninger

Organization of lecture • No lecture on 26.05. (State holiday) • 2 nd exercise sheet: Next week • Tutorials: • New slot: Friday, 12:15 - 14:00, U12 E1.1, biweekly • Fr, 13.05. (tomorrow) • Fr, 03.06. • Etc.

The polynomial method • Recently developed technique in Algorithm Design • Current fastest algorithms for • All-Pairs Shortest Paths [Williams 14] • Orthogonal Vectors [Abboud/Williams/Yu 15] • Hamming Nearest Neighbors [Alman/Williams 15] • Two main tools 1. Razborov-Smolensky from Circuit Complexity 2. Fast rectangular matrix multiplication

Reminder: Orthogonal Vectors Problem Input: Two sets 𝐵, 𝐶 ⊆ 0,1 𝑒 of 𝑒 -dimensional 0/1-vectors of size 𝑜 Output: Is there a pair 𝑏 ∈ 𝐵, 𝑐 ∈ 𝐶 s.t. 𝑏 and 𝑐 are orthogonal? ∃𝑏 ∈ 𝐵, 𝑐 ∈ 𝐶: 〈𝑏, 𝑐〉 = 0 𝑒 𝑙=1 𝑏 𝑙 ⋅ 𝑐 𝑙 = 0 ∀1 ≤ 𝑙 ≤ 𝑒: (𝑏 𝑙 = 0) ∨ (𝑐 𝑙 = 0) Trivial algorithms: • 𝑃 𝑜 2 𝑒 Interesting Regime: 𝑒 = 𝑑 log 𝑜 • 𝑃(2 𝑒 𝑜)

Today’s result Reminder: Conjecture : There is no algorithm for the orthogonal vectors problem with running time 𝑃(𝑜 2−𝜗 poly(𝑒)) for any 𝜗 > 0 . State of the art: Theorem : There is an algorithm for the orthogonal vectors problem with running time 𝑜 2−1/𝑃(log(𝑒/ log 𝑜)) . In this lecture: 𝑜 2−1/𝑃(log 𝑒) Algorithm is randomized and correct with high probability, i.e., probability ≥ 1 − 1/𝑜

Overview 1. Reduce problem to many subproblems of very small size 2. Precompute small circuits for solving subproblems 3. Evaluate circuits with probabilistic polynomials of low degree 4. Evaluate polynomials using fast rectangular matrix multiplication

Overview 1. Reduce problem to many subproblems of very small size 2. Precompute small circuits for solving subproblems 3. Evaluate circuits with probabilistic polynomials of low degree 4. Evaluate polynomials using fast rectangular matrix multiplication

Dividing into smaller subproblems 𝑜 Divide 𝐵 and 𝐶 into 𝑟 = ⌈ 𝑡 ⌉ subsets of size ≤ 𝑡 : 1. 𝐵 1 , … , 𝐵 𝑟 and 𝐶 1 , … , 𝐶 𝑟 Construct a polynomial 𝑄(𝑏 1 1 , … 𝑏 1 𝑒 , … , 𝑏 𝑡 1 … , 𝑏 𝑡 𝑒 , 2. 𝑐 1 1 , … 𝑐 1 𝑒 , … , 𝑐 𝑡 1 … , 𝑐 𝑡 1 ) 𝑄 𝐵 𝑗 , 𝐶 𝑘 = 1 if and only if 𝐵 𝑗 , 𝐶 𝑘 contains orthogonal pair …only with high probability For every pair of subsets 𝐵 𝑗 , 𝐶 𝑘 : evaluate 𝑄 on 𝐵 𝑗 , 𝐶 3. 𝑘 𝑜 2 …simultaneously! → 𝑃( 𝑡 2 polylog(𝑜)) Return “yes” if some 𝐵 𝑗 , 𝐶 4. 𝑘 contains orthogonal pair, “no” otherwise We set 𝑡 = 2 𝜗 log 𝑜/ log 𝑒 = 𝑜 𝜗/ log 𝑒 for some sufficiently small 𝜗

Questions 1. How to construct suitable polynomial 𝑄 ? 2. How to evaluate 𝑄 fast on all pairs of inputs?

Overview 1. Reduce problem to many subproblems of very small size 2. Precompute small circuits for solving subproblems 3. Evaluate circuits with probabilistic polynomials of low degree 4. Evaluate polynomials using fast rectangular matrix multiplication

Boolean circuits Boolean circuit • Directed acyclic graph • Sources: input bits • Sink: output bit • Inner nodes: Boolean operations • AND: ∧ • OR: ∨ • Arbitrary “fan - in”

Circuit for checking orthogonality of vectors Are 𝑦 and 𝑧 orthogonal? 𝑦 and 𝑧 orthogonal iff ¬∃𝑗: 𝑦 𝑗 = 1 ∧ 𝑧 𝑗 = 1 𝑨 Output bit 𝑨 = 1 iff ∧ 𝑦 and 𝑧 orthogonal AND of ORs with … … ∨ ∨ ∨ • 2𝑒 negated inputs • 1 output ¬𝑧[1] ¬𝑦[2] ¬𝑧[2] ¬𝑦[1] ¬𝑦[𝑒] ¬𝑧[𝑒]

Circuit for finding orthogonal pair Is there an orthogonal pair? Check orthogonality for every pair 𝑦 1 , 𝑧 1 orthogonal OR 𝑨 𝑦 1 , 𝑧 2 orthogonal OR ∨ … OR 𝑦 𝑡 , 𝑧 𝑡 orthogonal? … … … … ∧ ∧ ∧ OR of ANDs of ORs … … ∨ … … ∨ ∨ … … ∨ ∨ ∨ 2𝑒𝑡 2 negated inputs … … … ¬𝑦 𝑡 [𝑒] ¬𝑧 𝑡 [𝑒] ¬𝑦 1 [1] ¬𝑧 2 [1] ¬𝑦 1 [1] ¬𝑧 1 [1]

Overview 1. Reduce problem to many subproblems of very small size 2. Precompute small circuits for solving subproblems 3. Evaluate circuits with probabilistic polynomials of low degree 4. Evaluate polynomials using fast rectangular matrix multiplication

From circuits to polynomials • Obtain polynomial over 𝐺 2 outputting 1 if and only if circuit outputs 1 • 𝐺 2 : Field of {0, 1} with operations ⊕ and ⋅ • ⊕ is XOR-operation: • 0 ⊕ 0 = 0 1 ⊕ 0 = 1 0 ⊕ 1 = 1 1 ⊕ 1 = 0 • XOR of multiple variables: 𝑦 1 ⊕ 𝑦 2 ⊕ ⋯ ⊕ 𝑦 𝑙 = 1 if and only if odd number of 𝑦 𝑗 ’s is 1 • Expanded polynomials: • 𝑏 ⋅ 𝑐 ⊕ 𝑑 ⋅ 𝑏 ⊕ 𝑐 ⊕ 𝑒 = 𝑏𝑑 ⊕ 𝑏𝑐𝑑 ⊕ 𝑏𝑐𝑒 ⊕ 𝑏𝑑𝑒 • XOR of monomials • Goal: Few monomials allows fast evaluation later

Representing circuit by polynomial Straightforward approach: 𝑏 ∧ 𝑐 ⇒ 𝑏 ⋅ 𝑐 • AND: • Negation: ¬𝑏 ⇒ 1 ⊕ 𝑏 (addition=subtraction in 𝐺 2 ) 𝑏 ∨ 𝑐 ⇒ ¬(¬𝑏 ∧ ¬𝑐) • OR: ( DeMorgan’s Law) Example: Bottom-level of circuit ¬𝑏 ∨ ¬𝑐 ⇒ 1 ⊕ 𝑏 ⋅ 𝑐

Expanding a multiplication (distributive law) ∧ ⊕ XOR of 𝑙 𝑢 ANDs AND of 𝑢 XORs … … ⊕ ⊕ ⊕ ∧ ∧ ∧ Each XOR has Each AND has 𝑙 inputs 𝑢 inputs 𝑦 1,𝑙 𝑦 𝑢,1 𝑦 1,1 … 𝑦 1,1 … 𝑦 1,𝑙 … 𝑦 𝑢,𝑙 … … … 𝑦 𝑢,1 𝑦 𝑢,𝑙 … … 𝑦 1,1 ⊕ ⋯ ⊕ 𝑦 1,𝑙 ⋅ 𝑦 2,1 ⊕ ⋯ ⊕ 𝑦 2,𝑙 ⋅ … ⋅ 𝑦 𝑢,1 ⊕ ⋯ ⊕ 𝑦 𝑢,𝑙 = 𝑦 1,1 ⋅ 𝑦 2,1 ⋅ … ⋅ 𝑦 𝑢,1 ⊕ ⋯ ⊕ 𝑦 1,𝑙 ⋅ 𝑦 2,𝑙 ⋅ … ⋅ 𝑦 𝑢,𝑙 𝑙 𝑢 monomials 𝑙 choices 𝑙 choices 𝑙 choices Running time: 𝑃(𝑙 𝑢 𝑢 ⋅ deg) where deg is degree after expansion (maximum size of any monomial); here deg ≤ 𝑢

Razborov/Smolensky trick [Raz87] [Smo87] Naïve representation of OR Probabilistic representation of OR ∨ ∨ OR of 𝑢 XORs … ⊕ ⊕ ⊕ … … 𝑦 1 𝑦 2 𝑦 𝑙 Add each edge With probability 1 2 DeMorgan: 𝑙 𝑦 1 𝑦 2 … … 𝑦 𝑙 𝑃𝑆 𝑦 1 , … , 𝑦 𝑙 = 1 ⊕ 1 ⊕ 𝑦 𝑗 Parameter 𝑢 𝑗=1 Fewer monomials, correct whp After expansion: 2 𝑙 monomials

Correct representation with high probability Case 1: 𝑦 1 ∨ ⋯ ∨ 𝑦 𝑙 = 0 Probabilistic representation of OR Easy case: each XOR outputs 0, top OR outputs 0 ∨ Case 2: 𝑦 1 ∨ ⋯ ∨ 𝑦 𝑙 = 1 OR of 𝑢 XORs Let 𝑌 be set of inputs with 𝑦 𝑗 = 1 … ⊕ ⊕ ⊕ For each XOR: • If XOR has odd number of links to 𝑌 : XOR outputs 1 Add each edge With probability 1 (good event: top OR outputs 1) 2 • If XOR has even number of links to 𝑌 : XOR outputs 𝑦 1 𝑦 2 … … 𝑦 𝑙 0 (bad event!) Probability that XOR has even number of links to 𝑌 : 𝑢 1 1 ⇒ Probability that all XORs output 0 : = = = 1/2 because last element of X “decides” whether 2 𝑢 2 number of links is even or odd (each with prob. 1/2 ) 1 ⇒ Probability that OR outputs 1 : = 1 − 2 𝑢

Bounding number of monomials Formal definition of polynomial Probabilistic representation of OR For 𝑗 = 1 … 𝑢, 𝑘 = 1 … 𝑙 : ∨ 1 2 : Set 𝑠 𝑗,𝑘 = 1 • With probability OR of 𝑢 XORs Set 𝑠 𝑗,𝑘 = 0 • Otherwise: … ⊕ ⊕ ⊕ Polynomial: 𝑃𝑆 𝑢 𝑦 1 , … , 𝑦 𝑙 = Add each edge 𝑢 With probability 1 𝑙 1 ⊕ 1 ⊕⊕ 𝑘=1 𝑠 𝑗,𝑘 ⋅ 𝑦 𝑗 2 𝑦 1 𝑦 2 … … 𝑦 𝑙 𝑗=1 After expansion: 𝑙 + 1 𝑢 monomials

Formal definition Definition: Let 𝐷 be a Boolean circuit with 𝑙 input gates and let 𝐸 be a finite distribution of polynomials on 𝑙 variables over a ring 𝑆 containing 0 and 1 ( ∗ ) . The distribution 𝐸 is a probabilistic polynomial over 𝑆 representing 𝐷 with error 𝜀 if for all 𝑦 1 , … , 𝑦 𝑙 ∈ 0,1 𝑙 : 𝑞∼𝐸 𝑞 𝑦 1 , … , 𝑦 𝑙 = 𝐷 𝑦 1 , … , 𝑦 𝑙 Pr > 1 − 𝜀 . Example: OR-gate represented by 1 𝑢 𝑙 𝑃𝑆 𝑢 𝑦 1 , … , 𝑦 𝑙 = 1 ⊕ 𝑗=1 1 ⊕⊕ 𝑘=1 𝑠 𝑗,𝑘 ⋅ 𝑦 𝑗 with error 𝜀 = 1 − 2 𝑢 (*) In our case, 𝑆 is the field 𝐺 2

Representing OV circuit I Are 𝑦 and 𝑧 orthogonal? Bottom OR: ¬𝑦 𝑙 ∨ ¬𝑧 𝑙 𝑨 ⇒ 1 ⊕ 𝑦[𝑙] ⋅ 𝑧[𝑙] ∧ Middle AND: 1. DeMorgan 2. Razborov/Smolensky … … ∨ ∨ ∨ with 𝑢 1 = 3 log 𝑡 Number of monomials: 𝑒 + 1 𝑢 1 ¬𝑧[1] ¬𝑦[2] ¬𝑧[2] ¬𝑦[1] ¬𝑦[𝑒] ¬𝑧[𝑒]

Representing OV circuit II Is there an orthogonal pair? Middle ANDs: XOR of 𝑡 2 polynomials, each 𝑨 with 𝑒 + 1 𝑢 1 monomials ⇒ 𝑡 2 𝑒 + 1 𝑢 1 monomials ∨ … … … … ∧ ∧ ∧ Top OR: Raz/Smol with 𝑢 2 = 2 … … ∨ … … ∨ ∨ … … ∨ ∨ ∨ ⇒ 𝑡 2 𝑒 + 1 𝑢 1 𝑢 2 monomials = 𝑡 4 𝑒 + 1 6 log 𝑡 … … … ¬𝑦 𝑡 [𝑒] ¬𝑧 𝑡 [𝑒] ¬𝑦 1 [1] ¬𝑧 2 [1] ¬𝑦 1 [1] ¬𝑧 1 [1]