Stronger Connections Between Circuit Analysis and Circuit Lower Bounds, via PCPs of Proximity Lijie Chen Ryan Williams
Context: The Algorithmic Method for Proving Circuit Lower Bounds Proving limitations on non-uniform circuits is extremely hard. Prior approaches ( restrictions , polynomial approximations , etc.) face barriers ( Relativization , Algebrization , Natural Proofs ). Algorithmic Method • Non-trivial circuit-analysis algorithm ⇒ Circuit Lower Bounds . • Breakthroughs where previous approaches failed ( NEXP ⊄ ACC 0 ). • Believed to be possible for strong circuits (even 𝑄/𝑞𝑝𝑚𝑧 ).
Context: A Frontier of Circuit Complexity, Depth-2 Threshold Circuits THR gates : 𝑔 𝑦 = 𝑥 ⋅ 𝑦 ≥ 𝑢 𝑥 ∈ 𝑎 𝑜 , 𝑢 ∈ 𝑎 . MAJ gates : when 𝑥 𝑗 ’s and 𝑢 are bounded by poly(n). THR ∘ THR We can also define 𝑼𝑰𝑺 ∘ 𝑵𝑩𝑲 THR 𝑵𝑩𝑲 ∘ 𝑼𝑰𝑺 𝑵𝑩𝑲 ∘ 𝑵𝑩𝑲 THR THR THR
Context: A Frontier of Circuit Complexity, Depth-2 Threshold Circuits Exponential Lower Bounds are known for 𝑁𝐵𝐾 ∘ 𝑁𝐵𝐾 [Hajnal-Maass-Pudlák-Szegedy- Turán’93] NEXP 𝑁𝐵𝐾 ∘ 𝑈𝐼𝑆 [Nisan’94] Non-deterministic 𝑈𝐼𝑆 ∘ 𝑁𝐵𝐾 [Forster-Krause-Lokam-Mubarakzjanov- Exponential Time. Schmitt- Simon’01] Frontier Open Question : Is NEXP ⊆ 𝑼𝑰𝑺 ∘ 𝑼𝑰𝑺 ? Potential Approaches in this talk.
Motivation: Apply the Algorithmic Method to THR of THR? What Circuit-Analysis Tasks? Non-trivial Circuit- Analysis Algorithms ℭ -SAT ℭ -CAPP ⇒ Circuit Lower Bounds Derandomization!! Estimate quantity 𝑦∼𝑉 𝑜 [𝐷 𝑦 = 1] , Pr ∃ x s.t. 𝐷 𝑦 = 1? with additive error 𝜁 𝐷 𝐷 𝜁 : constant or 2 𝑜 /𝑜 𝜕(1) time? inverse polynomial ∃𝑦 ? 𝑦 ∼ 𝑉 𝑜
Motivation: Apply the Algorithmic Method to THR of THR? Most previous work on the algorithmic method exploits SAT algorithms. THR ∘ THR Problem THR SAT of THR of THR is probably very hard. A special case is MAX- 𝑙 -SAT, for which no non- trivial ( 2 𝑜 /𝑜 𝜕(1) time) algorithm is known for THR THR THR 𝒍 = 𝝏(log 𝒐) and 𝑞𝑝𝑚𝑧(𝑜) clauses. MAX- 𝑙 -SAT MAJ Considered to be a barrier for the Algorithmic Approach. 𝑃𝑆 𝑙 𝑃𝑆 𝑙 𝑃𝑆 𝑙
Motivation: Apply the Algorithmic Method to THR of THR? From Derandomization (CAPP) SAT of THR of THR : probably very hard ⇒ Circuit Lower Bounds For a circuit class ℭ , But derandomization • 2 𝑜 /𝑜 𝜕(1) -time CAPP for ( 𝐁𝐎𝐄 𝐪𝐩𝐦𝐳(𝒐) ∘ 𝐏𝐒 𝟒 ∘ 𝕯 ) is widely believed to ⇒ 𝑂𝐹𝑌𝑄 ⊄ ℭ [Williams’13/14, Santhanam Williams’14, Ben -Sasson Viola’14] be possible . 2 𝑜 /𝑜 𝜕(1) -time CAPP for ( 𝑩𝑫 𝟏 ∘ 𝕯 ) • 1 ⇒ 𝑂𝐹𝑌𝑄 can’t be 2 + 𝑝(1) -approximated by ℭ [R. Chen Oliveira Santhanam’18] NQP 2 𝑜−𝑜 𝜁 -time CAPP for ( 𝐁𝐎𝐄 𝐪𝐩𝐦𝐳(𝒐) ∘ 𝐏𝐒 𝟒 ∘ 𝕯 ) • Non-deterministic ⇒ 𝑂𝑅𝑄 ⊄ ℭ [Murray Williams’18] Quasi-Polynomial 2 𝑜−𝑜 𝜁 -time CAPP for ( 𝐁𝐃 𝟏 ∘ 𝕯 ) • Time. (𝒐 𝒒𝒑𝒎𝒛𝒎𝒑𝒉(𝒐) ) 1 ⇒ 𝑂𝑅𝑄 can’t be 2 + 𝑝(1) -approximated by ℭ [L. Chen’19]
Back to THR of THR SAT of THR of THR : probably very hard To show 𝑂𝐹𝑌𝑄 ⊄ Our result 1 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆 , we need It suffices to derandomize 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆 . to derandomize AND poly(𝑜) ∘ OR 3 ∘ 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆 , which Our result 2 could be harder. Surprisingly, it indeed only suffices to derandomize 𝑈𝐼𝑆 ∘ 𝑁𝐵𝐾 or 𝑁𝐵𝐾 ∘ 𝑁𝐵𝐾 !
General Result: A Stronger Connection Between Circuit-Analysis Algorithms and Circuit Lower Bounds For a circuit class ℭ : • 𝟑 𝒐 /𝒐 𝝏(𝟐) -time CAPP for ⊕ 2 ∘ ℭ , 𝐵𝑂𝐸 2 ∘ ℭ , or 𝑃𝑆 2 ∘ ℭ ⇒ 𝑂𝐹𝑌𝑄 ⊄ ℭ . • 𝟑 𝒐−𝒐 𝜻 -time CAPP for ⊕ 2 ∘ ℭ , 𝐵𝑂𝐸 2 ∘ ℭ , or 𝑃𝑆 2 ∘ ℭ Why the constant “2”? ⇒ 𝑂𝑅𝑄 ⊄ ℭ . • Short answer : A PCP system needs to make at least 2 queries. • Long answer : See the paper ☺
Tighter Connections for Algorithms/Lower Bounds for THR of THR 2 𝑜 /𝑜 𝜕(1) -time CAPP algorithm Luckily, the “2” doesn’t for 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆 matter for 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆 ☺ ⇒ 𝑂𝐹𝑌𝑄 ⊄ 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆 . ⊕ 𝟑 ∘ 𝑼𝑰𝑺 ∘ 𝑼𝑰𝑺 ⊆ 𝑼𝑰𝑺 ∘ 𝑼𝑰𝑺 2 𝑜 /𝑜 𝜕(1) -time CAPP algorithm for 𝑈𝐷 𝑒 ⇒ 𝑂𝐹𝑌𝑄 ⊄ 𝑈𝐷 𝑒 . 𝑼𝑫 𝒆 : depth-d, poly-size, linear threshold circuits
Let Us Make Our Life Even Easier Poly-size 𝑼𝑰𝑺 ∘ 𝑼𝑰𝑺 and 𝑵𝑩𝑲 ∘ 𝑵𝑩𝑲 are equivalent for Non-Trivial ( 2 𝑜 /𝑜 𝜕(1) time) CAPP Algorithms when 𝜁 = 1/𝑞𝑝𝑚𝑧 𝑜 ! THR MAJ THR THR THR MAJ MAJ MAJ Proved by new structure lemmas for 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆
Let Us Make Our Life Even Easier Poly-size 𝑼𝑰𝑺 ∘ 𝑼𝑰𝑺 and 𝑼𝑰𝑺 ∘ 𝑵𝑩𝑲 are equivalent for Non-Trivial ( 2 𝑜 /𝑜 𝜕(1) time) CAPP Algorithms for any constant 𝜁 > 0 ! THR THR THR THR THR MAJ MAJ MAJ Proved by new structure lemmas for 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆
Corollary If there are 2 𝑜 /𝑜 𝜕(1) -time CAPP for 𝑁𝐵𝐾 ∘ 𝑁𝐵𝐾 with 𝜁 = 1/𝑞𝑝𝑚𝑧(𝑜) , or a 2 𝑜 /𝑜 𝜕(1) -time CAPP for 𝑈𝐼𝑆 ∘ 𝑁𝐵𝐾 with constant 𝜁 , then 𝑶𝑭𝒀𝑸 ⊄ 𝑼𝑰𝑺 ∘ 𝑼𝑰𝑺 .
Another Application: Inapproximability by Depth-2 Neural Networks Thm Depth-2 Neural Network For every 𝑙 and constant 𝜀 < 1/2 , there is a function 𝑔 ∈ 𝑂𝑄 such that ∑ 𝑔 cannot be 𝜀 approximated by Depth-2 𝑥 1 𝑂 𝑦 ≔ 𝑥 𝑗 ⋅ 𝑈𝐼𝑆 𝑗 𝑦 ∈ ℝ Neural Networks of size 𝑜 𝑙 𝑥 3 𝑥 2 𝑗 THR THR THR Improved [Wil’18] , which proved that there is such an 𝑔 ∈ 𝑂𝑄 which cannot ∑ 𝑂 𝑦 ≔ 𝑥 𝑗 ⋅ 𝑆𝑓𝑀𝑉 𝑗 𝑦 ∈ ℝ be exactly computed by Depth-2 Neural 𝑥 1 𝑥 3 𝑥 2 𝑗 Networks of size 𝑜 𝑙 . ReLU ReLU ReLU
Philosophy Using PCP Algorithmically to Prove Circuit Lower Bounds (Remember: PCPs are algorithms!) If you want to prove 𝑸 = 𝑶𝑸 , then PCPs should make your 𝟖 life much easier (now you only need an algorithm for ( 𝟗 + 𝜻) - approximation to 3-SAT!) [ Håstad ’97] (Well, I don’t really believe in 𝑄 = 𝑂𝑄 .) We only want to derandomize circuits. But PCPs still make our life easier (though in a more indirect way)
Starting Point: Non-deterministic Derandomization Suffices for Circuit Lower Bounds ℭ -GAP-TAUT ( tautology ) [Wil’13] 2 𝑜 /𝑜 𝜕(1) time non-deterministic Distinguish between algorithm for GAP-TAUT 1. Pr 𝑦 [𝐷 𝑦 = 1] = 1 . Non-deterministic Algorithm for GAP-TAUT on poly-size general Given a general circuit 𝐷 , we want a 2 𝑜 /𝑜 𝜕(1) time ( Yes Case ) circuits with 𝜁 = 1/2 𝐷 2. Pr 𝑦 [𝐷 𝑦 = 1] ≤ 𝜁 . non-deterministic algo 𝔹 , such that: ⇒ 𝑂𝐹𝑌𝑄 ⊄ 𝑄/𝑞𝑝𝑚𝑧 . ( No Case ) 1. If 𝐷 is a tautology, then 𝔹 accepts on some guesses. 2. If Pr 𝑦 𝐷 𝑦 = 1 ≤ 1/2 , 𝔹 rejects on all guesses. 𝑦 ∼ 𝑉 𝑜
Proof Overview: Outline Starting Point [Wil’13] Key point: make use of 2 𝑜 /𝑜 𝜕(1) time non-deterministic algorithm for GAP-TAUT this assumption as on poly-size general circuits with 𝜁 = 1/2 much as possible! ⇒ 𝑂𝐹𝑌𝑄 ⊄ 𝑄/𝑞𝑝𝑚𝑧 . Assume 𝑂𝐹𝑌𝑄 ⊂ ℭ 2 𝑜 /𝑜 𝜕(1) non- 𝑂𝐹𝑌𝑄 ⊄ 𝑄/𝑞𝑝𝑚𝑧 Think of ℭ as ⇒ 𝑂𝐹𝑌𝑄 ⊄ ℭ deterministic GAP- 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆 Contradiction! TAUT for 𝑄/𝑞𝑝𝑚𝑧 Non-trivial CAPP on OR 3 ∘ ℭ with constant 𝜁
Goal: Designing the Algorithm under Assumption Assume 𝑂𝐹𝑌𝑄 ⊂ ℭ 2 𝑜 /𝑜 𝜕(1) non-deterministic GAP-TAUT Think of ℭ as on 𝑄/𝑞𝑝𝑚𝑧 𝑈𝐼𝑆 ∘ 𝑈𝐼𝑆 𝑂𝐵𝑂𝐸 𝑦, 𝑧 ≔ 𝑂𝑃𝑈(𝐵𝑂𝐸(𝑦, 𝑧)) Non-trivial CAPP on OR 3 ∘ ℭ with constant 𝜁 It is universal Goal Given an 𝑂𝐵𝑂𝐸 circuit 𝐷 , under the two assumptions, design a 2 𝑜 /𝑜 𝜕(1) time non-deterministic algo 𝔹 , such that: 1. If 𝐷 is a tautology, then 𝔹 accepts on some guesses. 2. If Pr 𝑦 𝐷 𝑦 = 1 ≤ 1/2 , 𝔹 rejects on all guesses.
Review: Approach of [Wil’14] Guess-and-Verify-Equivalence 𝑂𝐹𝑌𝑄 ⊂ ℭ implies 𝑄/𝑞𝑝𝑚𝑧 collapses to ℭ . That is, under assumption , the given general circuit 𝐷 has an equivalent 𝕯 circuit 𝑬 . If we can find 𝑬 , then we can derandomize 𝐸 instead, where we have algorithms! Problem: How to find 𝑬 ? Solution Allowed to use non-determinism so one can guess 𝐸 . Well, just guess more But still have to verify 𝐸 is equivalent to 𝐷 , which circuits! seems HARD .
Recommend
More recommend
