Exact (Bichromatic) Maximum Inner Product , - PowerPoint PPT Presentation

On The Hardness of Approximate and Exact (Bichromatic) Maximum Inner Product 𝑁𝑏𝑦 𝑏,𝑐 ∈𝐵×𝐶 𝑏 ⋅ 𝑐 Lijie Chen (Massachusetts Institute of Technology)

Max-IP and Z-Max-IP • (Boolean) Max-IP : • Given sets 𝐵 and 𝐶 of Boolean vectors (each of size n) find 𝑏 in 𝐵 and 𝑐 in 𝐶 with maximum inner product: • For sets 𝐵 and 𝐶 , set 𝑁𝑏𝑦𝐽𝑄 𝐵, 𝐶 ≔ 𝑏,𝑐 ∈𝐵×𝐶 ⟨𝑏, 𝑐⟩ . max • Approx. version: find a r-multiplicative approximation to the answer: • Want an 𝐵𝑀𝐻 s.t. 𝑁𝑏𝑦𝐽𝑄(𝐵, 𝐶) ≤ 𝐵𝑀𝐻(𝐵, 𝐶) ≤ 𝑁𝑏𝑦𝐽𝑄(𝐵, 𝐶) ⋅ 𝑠 . • Z-Max-IP : • Two sets of 𝑜 Integer vectors.

Max-IP and Z-Max-IP • Basic problems, relevant in practice. • Important theoretical implications as well. • Approx. Boolean Max-IP: • [ARW’17]: basis of the recent breakthrough result • Bichromatic LCS Closest Pair over permutations, in Hardness for Approximation in P, implies • Approximate Regular Expression Matching, hardness for many other problems. • Diameter in Product Metric, • Approximate Closest Pair in Euclidian Space [Rub’18] • Z-Max-IP: • [Wil’18]: Hardness for Z-Max-IP implies hardness for finding furthest pair in low dimension Euclidean space.

New Hardness for Z-Max-IP (under SETH) • Z-Max-IP for n vectors of 𝟑 𝑷 𝐦𝐩𝐡 ∗ 𝒐 dimensions requires 𝑜 2−𝑝(1) time under SETH. Z-Max-IP : • Z-Max-IP for 𝑜 vectors of 𝟑 𝑷 𝐦𝐩𝐡 ∗ 𝒐 in 𝑜 1.99 time. Given sets 𝐵 and 𝐶 of Integer • ⇒ Max-IP for 𝑜 vectors of 𝑃(log 𝑜) dim. in 𝑜 1.99 time. vectors (each of size n) find 𝑏 in • ⇒ CNF-SAT for 𝑜 variables and 𝑃(𝑜) clauses in 2 0.995𝑜 time. 𝐵 and 𝑐 in 𝐶 with maximum inner product: • ⇔ SETH is false. [CIP’06] • Closer to the upper bound • [Mat’92] : Z -Max-IP in n 2−1/O(d) time. Upper Bound Lower Bound Lower Bound 𝑜 2−𝜗 when 𝑒 = 𝑃(1) 𝑜 2−o(1) when 𝑒 = 𝜕(log 2 log 𝑜) 𝑜 2−o(1) when 𝑒 = 2 𝑃 log ∗ 𝑜 ? [Mat’92] [This work] Implicit in [Wil’18]

New Hardness for Z-Max-IP (under SETH) Z-Max-IP : 1. New Hardness for Z-Max-IP (under SETH): Given sets 𝐵 and 𝐶 of Integer vectors (each of size n) find 𝑏 in • Z-Max-IP for n vectors of 𝟑 𝑷 𝐦𝐩𝐡 ∗ 𝒐 dimensions 𝐵 and 𝑐 in 𝐶 with maximum requires 𝑜 2−𝑝(1) time. inner product: • Separation for Boolean Max-IP / Z-Max-IP: • Z-Max-IP is much harder than Boolean Max-IP. • Progress on Open Problem 23 in Dagstuhl workshop on Structure and Hardness in P Z-Max-IP Boolean Max-IP 𝑜 2−𝜗 when 𝑒 = 𝑑 log 𝑜 . 𝑜 2−o(1) when 𝑒 = 2 𝑃 log ∗ 𝑜 [AW15, ACW16] [This work] HARD EASY

New Hardness for Z-Max-IP (under SETH) Z-Max-IP : 1. New Hardness for Z-Max-IP (under SETH): Given sets 𝐵 and 𝐶 of Integer vectors (each of size n) find 𝑏 in • Z-Max-IP for n vectors of 𝟑 𝑷 𝐦𝐩𝐡 ∗ 𝒐 dimensions 𝐵 and 𝑐 in 𝐶 with maximum require 𝑜 2−𝑝(1) time. inner product. New Hardness for ℓ 2 -Furthest Pair in 𝑆 𝑒 . (reductions from [Wil18]) • Finding ℓ 2 -Furthest Pair in 𝑆 𝑒 among 𝑜 points for 𝑒 = • 2 𝑃 log ∗ 𝑜 requires 𝑜 2−𝑝(1) time. • Stronger separation between furthest and closest pair. ℓ 2 -Furthest Pair ℓ 2 -Furthest Pair ℓ 2 -Closest Pair 𝑜 2−o(1) when 𝑒 = 𝜕(log 2 log 𝑜) 𝑜 2−o(1) when 𝑒 = 2 𝑃 log ∗ 𝑜 2 𝑃 𝑒 ⋅ 𝑜 𝑞𝑝𝑚𝑧𝑚𝑝𝑕(𝑜) [This work] [Wil’18] [BS76, KM95, DHKP97] HARD EASY

Characterization of Boolean Approx. Max-IP 2. Characterization of Approx. Max-IP: • Boolean Max-IP : • For sets 𝐵 and 𝐶 with 𝑜 Boolean • [ARW’17]: Finding 2 (log 1−o 1 𝑜) approximation to vectors, find 𝑁𝑏𝑦𝐽𝑄 𝐵, 𝐶 ≔ Max-IP with 𝑜 𝑝 1 dimensions, requires 𝑏,𝑐 ∈𝐵×𝐶 ⟨𝑏, 𝑐⟩ . max 𝑜 2−𝑝(1) 𝑢𝑗𝑛𝑓. • Approx. version: find a r- multiplicative approximation to the • A more refined question: answer: • For each vector dimension 𝑒 = 𝑒 𝑜 , what is the 𝑁𝑏𝑦𝐽𝑄(𝐵, 𝐶) ≤ 𝐵𝑀𝐻(𝐵, 𝐶) ≤ 𝑁𝑏𝑦𝐽𝑄(𝐵, 𝐶) ⋅ 𝑠 . smallest 𝑠 such that Max-IP can be 𝑠 -approximated in truly sub-quadratic time? • d = d(n) : vector dimensions • r = r(n) : approx. ratio

Characterization of Boolean Approx. Max-IP 2. Characterization of Approx. Max-IP: • Boolean Max-IP : • For sets 𝐵 and 𝐶 with 𝑜 Boolean • A more refined question: vectors, find 𝑁𝑏𝑦𝐽𝑄 𝐵, 𝐶 ≔ • For each vector dimension 𝑒 = 𝑒 𝑜 , what is the 𝑏,𝑐 ∈𝐵×𝐶 ⟨𝑏, 𝑐⟩ . max smallest 𝑠 such that Max-IP can be 𝑠 -approximated in truly sub-quadratic time? • Approx. version: find a r- multiplicative approximation to the • We obtain a characterization (under SETH)! answer: 𝑁𝑏𝑦𝐽𝑄(𝐵, 𝐶) ≤ 𝐵𝑀𝐻(𝐵, 𝐶) ≤ • For all 𝑒 satisfying 𝜕 log 𝑜 < 𝑒 < 𝑜 𝑝 1 𝑁𝑏𝑦𝐽𝑄(𝐵, 𝐶) ⋅ 𝑠 . Ω(1) 𝑒 • Truly sub-quadratic time for 𝑠 = EASY! log 𝑜 • d = d(n) : vector dimensions o(1) • 𝑒 • Requires 𝑜 2−𝑝(1) time for 𝑠 = r = r(n) : approx. ratio HARD! log 𝑜

Characterization of Boolean Approx. Max-IP 2. Characterization of Approx. Max-IP: • Boolean Max-IP : • We obtain a characterization! • For sets 𝐵 and 𝐶 with 𝑜 Boolean Ω(1) 𝑒 𝑜 2−𝜗 time. EASY! • 𝑠 = vectors, find 𝑁𝑏𝑦𝐽𝑄 𝐵, 𝐶 ≔ log 𝑜 𝑏,𝑐 ∈𝐵×𝐶 ⟨𝑏, 𝑐⟩ . max o(1) 𝑒 𝑜 2−𝑝 1 time. HARD! • 𝑠 = log 𝑜 • Approx. version: find a r- • Example: multiplicative approximation to the • 𝑒 = 𝑑 log 𝑜 , 𝑃(1) -approximation is EASY. answer: • 𝑒 = log 2 𝑜 , 𝑃(log 0.1 𝑜) −approximation is EASY. 𝑁𝑏𝑦𝐽𝑄(𝐵, 𝐶) ≤ 𝐵𝑀𝐻(𝐵, 𝐶) ≤ • 𝑒 = log 2 𝑜 , (log 𝑝(1) 𝑜) -approximation is HARD. 𝑁𝑏𝑦𝐽𝑄(𝐵, 𝐶) ⋅ 𝑠 . • Upper Bound via polynomial method . • d = d(n) : vector dimensions • r = r(n) : approx. ratio • Lower Bound follows from [Rub’18] .

New Merlin-Arthur Protocol for Set- Disjointness • MA Communication Protocol : • Alice holds x, Bob holds y, want 3. A new MA Protocol for Set-Disjointness to compute F(x,y). • [AW’09] : An O( 𝑜 log 𝑜) MA protocol. • [Kla’03] : Ω( 𝑜) Lower Bound. • This work: an O( 𝑜 log 𝑜 log log 𝑜) protocol. Ω( 𝑜) O( 𝑜 log 𝑜) O( 𝑜 log 𝑜 log log 𝑜) 2 • F(x,y) = 1 ⇒ exists a proof, Pr 𝑏𝑑𝑑 ≥ ? 3 . [Lower Bound] [Upper Bound] [Upper Bound] F(x,y) = 0 ⇒ for all proofs, Pr 𝑏𝑑𝑑 ≤ 1 [Kla’03] [AW’09] [This work] • 3 . • Complexity = Proof Length + Communication

New Connection with Communication Complexity 4. New Connection with Communication Complexity Quantum! • [ARW’17]: • 𝑜 log 𝑜 MA protocol for Set-Disjointness • ⇒ SETH-Hardness for Approx. Boolean Max-IP. • Open Question from [ARW’17]: • There is a 𝑜 BQP protocol for Set-Disjointness. Does it also imply some hardness results? {−𝟐, 𝟐} -Max-IP: • [This work]: YES! Given sets 𝐵 and 𝐶 of vectors • 𝑜 BQP protocol for Set-Disjointness with {−1,1} entries (each of • ⇒ SETH-Hardness for Approx. {−1,1} -Max-IP size 𝑜 ) find 𝑏 in 𝐵 and 𝑐 in 𝐶 with maximum inner product.

Proof Overview: SETH-Hardness of Z-Max-IP • Starting Point ： SETH implies OV Conjecture. • Orthogonal Vectors (OV) Problem: • Given two sets A,B of Boolean vectors, find an orthogonal pair between them. OV Conjecture: OV with sets of 𝑜 vectors, 𝜕(log 𝑜) dimensions requires 𝑜 2−𝑝(1) time. Our Goal : A “dimensionality” reduction from 𝜕(log 𝑜) dimensional OV to 2 𝑃 log ∗ 𝑜 dimensional Z-Max-IP

Reduction RoadMap Orthogonal Vectors (OV) Problem : • First cover a baby version which shows 𝜕 log 2 log 𝑜 Two sets A,B of dimensional Z-Max- IP is hard. (same as [Wil’18]) Boolean vectors, • Then outline the key ideas to get the 2 𝑃 log ∗ 𝑜 dimensional find an orthogonal pair hardness. between them. • An intermediate problem: Z-Max-IP : • Z-OV : Given two sets A,B of Integer vectors, find an orthogonal Two sets of 𝑜 pair between them. Integer vectors. find a pair between them which maximize 𝜕 log 2 log 𝑜 -dim. Hard Easy 𝜕 log log 𝑜 -dim. 𝜕 log 𝑜 -dim. their inner Z-OV OV Z-Max-IP product.

𝜕 log 2 log 𝑜 -dim. Hard Easy 𝜕 log log 𝑜 -dim. 𝜕 log 𝑜 -dim. Z-OV OV Z-Max-IP Easy Part: Z-OV ⇒ Z-Max-IP • Implicit in [Wil’18]. Z-OV : Two sets A,B of • 𝑏, 𝑐 ∈ 𝑎 𝑒 . (Squaring trick) Integer vectors, find an orthogonal • 𝑏 ⋅ 𝑐 = 0 ⇒ − 𝑏 ⋅ 𝑐 2 = 0 pair between • 𝑏 ⋅ 𝑐 ≠ 0 ⇒ − 𝑏 ⋅ 𝑐 2 < 0 them. • To solve Z-OV, it suffices to calculate the maximum value of − 𝑏 ⋅ 𝑐 2 for 𝑏, 𝑐 ∈ 𝐵 × 𝐶 . Z-Max-IP : Given sets 𝐵 and 𝐶 • − 𝑏 ⋅ 𝑐 2 = − σ 𝑗 𝑏 𝑗 ⋅ 𝑐 𝑗 2 = − σ 𝑗,𝑘 𝑏 𝑗 𝑏 𝑘 𝑐 𝑗 𝑐 of Integer vectors 𝑘 (each of size n) find 𝑏 𝑗,𝑘 = 𝑏 𝑗 ⋅ 𝑏 𝑘 , ത • ത 𝑐 𝑗,𝑘 = −𝑐 𝑗 ⋅ 𝑐 𝑘 . 𝑏 in 𝐵 and 𝑐 in 𝐶 with maximum 𝑐 , Z-Max-IP with 𝑒 2 dim. 𝑏 ⋅ ത • Maximize ത inner product.