Lecture 8: SOS Lower Bound for 3-XOR Lecture Outline Part I: SOS - PowerPoint PPT Presentation

Lecture 8: SOS Lower Bound for 3-XOR

Lecture Outline • Part I: SOS Lower Bounds from Pseudo- expectation Values • Part II: Random 3-XOR Equations and Pseudo- expectation Values • Part III: Proving PSDness • Part IV: Analyzing Parameter Regimes • Part V: Gaussian Elimination and SOS • Part VI: Further Work

Part I: SOS Lower Bounds from Pseudo-expectation Values

Positivstellensatz Proofs Review • Recall: a degree d Positivstellensatz proof that constraints 𝑡 1 𝑦 1 , … , 𝑦 𝑜 = 0, 𝑡 1 𝑦 1 , … , 𝑦 𝑜 = 0 , etc. are infeasible is an expression of the form 2 where: − 1 = σ 𝑗 𝑔 𝑗 𝑡 𝑗 + σ 𝑘 𝑕 𝑘 ∀𝑗, deg 𝑔 𝑗 + deg 𝑡 𝑗 ≤ 𝑒 1. 𝑒 ∀𝑘, deg 𝑕 𝑘 ≤ 2. 2 • How do we show that there is no degree d Positivstellensatz proof of infeasibility?

Positivstellensatz Proofs Review • Recall: a degree d Positivstellensatz proof that ℎ(𝑦 1 , … , 𝑦 𝑜 ) ≥ 𝑑 given constraints 𝑡 1 𝑦 1 , … , 𝑦 𝑜 = 0, 𝑡 1 𝑦 1 , … , 𝑦 𝑜 = 0 , etc. is an 2 expression of the form ℎ = 𝑑 + σ 𝑗 𝑔 𝑗 𝑡 𝑗 + σ 𝑘 𝑕 𝑘 where: ∀𝑗, deg 𝑔 𝑗 + deg 𝑡 𝑗 ≤ 𝑒 1. 𝑒 ∀𝑘, deg 𝑕 𝑘 ≤ 2. 2 • How do we show that there is no degree d Positivstellensatz proof that ℎ(𝑦 1 , … , 𝑦 𝑜 ) ≥ 𝑑 ?

Pseudo-expectation Values Review • Recall: Given constraints 𝑡 1 𝑦 1 , … , 𝑦 𝑜 = 0, 𝑡 1 𝑦 1 , … , 𝑦 𝑜 = 0 , etc., degree d Pseudo- expectation values consist of a linear map ෨ 𝐹 from polynomials of degree ≤ 𝑒 to ℝ such that: ෨ 1. 𝐹 1 = 1 ∀𝑔, 𝑗, ෨ 𝐹 𝑔𝑡 𝑗 = 0 whenever deg 𝑔 𝑗 + deg 𝑡 𝑗 ≤ 𝑒 2. 𝐹 𝑕 2 ≥ 0 whenever deg 𝑕 ≤ 𝑒 ∀𝑕, ෨ 3. 2 • The third condition is equivalent to 𝑁 ≽ 0 where 𝑁 is the moment matrix with entries 𝑁 𝑞𝑟 = ෨ 𝐹[𝑞𝑟]

SOS Lower Bound Strategy • Recall: degree d pseudo-expectation values imply there is no degree d Positivstellensatz proof of infeasibility • Analogously, degree d pseudo-expectation values with ෨ 𝐹 ℎ < 𝑑 imply there is no degree d Positivstellensatz proof that ℎ ≥ 𝑑 . • Proof: can assume both exist and get the following contradiction: 2 ≥ 𝑑 c > ෩ E[ℎ] = ෨ 𝐹[𝑑] + σ 𝑗 ෨ 𝑗 𝑡 𝑗 ] + σ 𝑘 ෨ 𝐹[𝑔 𝐹 𝑕 𝑘

SOS Lower Bound Strategy • To prove an SOS lower bound, we generally do the following: 1. Come up with pseudo-expectation values ෨ 𝐹 which obey the required linear equations 2. Show that the moment matrix 𝑁 is PSD • In the examples we’ll see, part 1 is relatively easy and the technical part is part 2. • That said, for several very important problems, we’re stuck on part 1!

Part II: Random 3-XOR Equations and Pseudo-expectation Values

Equations for Random 3-XOR • Want each 𝑦 𝑗 ∈ {−1,1} • 3-XOR constraint: 𝑦 𝑗 𝑦 𝑘 𝑦 𝑙 = 1 or 𝑦 𝑗 𝑦 𝑘 𝑦 𝑙 = −1 • We will take 𝑛 3-XOR constraints at random • Problem equations: 2 = 1 ∀𝑗, 𝑦 𝑗 1. ∀𝑏 ∈ 1, 𝑛 , 𝑦 𝑗 𝑏 𝑦 𝑘 𝑏 𝑦 𝑙 𝑏 = 𝑑 𝑏 where ∀𝑏 ∈ [1, 𝑛] , 2. 𝑗 𝑏 , 𝑘 𝑏 , 𝑙 𝑏 ∈ [1, 𝑜] and 𝑑 𝑏 ∈ {−1,1}

SOS Lower Bound for Random 3-XOR • Problem equations: 2 = 1 ∀𝑗, 𝑦 𝑗 1. ∀𝑏 ∈ 1, 𝑛 , 𝑦 𝑗 𝑏 𝑦 𝑘 𝑏 𝑦 𝑙 𝑏 = 𝑑 𝑏 where ∀𝑏 ∈ [1, 𝑛] , 2. 𝑗 𝑏 , 𝑘 𝑏 , 𝑙 𝑏 ∈ [1, 𝑜] and 𝑑 𝑏 ∈ {−1,1} • Theorem [Gri02], rediscovered by [Sch08]: If 3 2−𝜗 𝑜 𝑛 ≤ 𝑒 then w.h.p., degree d SOS does not refute these equations.

Choosing Pseudo-expectation Values • How do we choose the pseudo -expectation values? • Many choices are fixed. • Example: If 𝑦 1 𝑦 2 𝑦 3 = 1 and 𝑦 1 𝑦 4 𝑦 5 = −1 then 2 𝑦 2 𝑦 3 𝑦 4 𝑦 5 = 𝑦 2 𝑦 3 𝑦 4 𝑦 5 = −1 𝑦 1 • However, we only want to make these deductions at low degrees…

Choosing Pseudo-expectation Values • Def: Define 𝑦 𝐽 = ς 𝑗∈𝐽 𝑦 𝑗 • Proposition: ∀𝐽, 𝐾, 𝑦 𝐽 𝑦 𝐾 = 𝑦 𝐽Δ𝐾 where 𝐽 Δ 𝐾 = 𝐽 ∪ 𝐾 ∖ (𝐽 ∩ 𝐾) is the disjoint union of 𝐽 and 𝐾. • To decide which 𝑦 𝐽 have fixed values: 1. Keep track of a collection of equations {𝑦 𝐽 = 𝑑 𝐽 } starting with the problem constraints. 2. If we have equations 𝑦 𝐽 = 𝑑 𝐽 and 𝑦 𝐾 = 𝑑 𝐾 where 𝐽, J , and 𝐽 Δ 𝐾 all have size at most 𝑒 , then we add the equation 𝑦 𝐽Δ𝐾 = 𝑑 𝐽 𝑑 𝐾 (if we don’t have it already)

Choosing Pseudo-expectation Values • Set ෨ 𝐹 𝑦 𝐽 = 𝑑 𝐽 if our collection has 𝑦 𝐽 = 𝑑 𝐽 • What if we don’t have an equation for 𝑦 𝐽 ? • If we have no equation for 𝑦 𝐽 , set ෨ 𝐹 𝑦 𝐽 = 0 • Set ෨ 2 𝑔 = ෨ 𝐹 𝑦 𝑗 𝐹[𝑔] for all 𝑔 of degree ≤ 𝑒 − 2 • These pseudo-expectation values are well- defined as long as we never have both the equations 𝑦 𝐽 = 1 and 𝑦 𝐽 = −1 .

Part III: Proving PSDness

To-Do List • Here we assume that ෨ 𝐹 is well defined. We will analyze when this holds w.h.p. in the next section. • Need to check linear equations. This follows from the definitions: – Whenever we have a constraint 𝑦 𝐽 = 𝑑 𝐽 , for all 𝐾 of size ≤ 𝑒 − 3 , either ෨ 𝐹 𝑦 𝐽 𝑦 𝐾 = 𝑑 𝐽 𝑑 𝐾 = 𝑑 𝐽 ෨ 𝐹[𝑦 𝐾 ] or 𝐹 𝑦 𝐽 𝑦 𝐾 = 𝑑 𝐽 ෨ ෨ 𝐹 𝑦 𝐾 = 0 – ∀𝑗, 𝑔: deg 𝑔 ≤ 𝑒 − 2, ෨ 2 𝑔 = ෨ 𝐹 𝑦 𝑗 𝐹[𝑔] • Need to check moment matrix is PSD.

Restriction to Multilinear Indices • Observation: Whenever we have constraints 2 = 𝑦 𝑗 or 𝑦 𝑗 2 = 1 , it is sufficient to consider the 𝑦 𝑗 entries of 𝑁 indexed by multilinear monomials. • Reason: Given any 𝑕 of degree ≤ 𝑒 2 , ∃ multilinear 𝐹 𝑕 ′2 = ෨ g’ such that ෨ 𝐹[𝑕 2 ] . 2 𝑔 in 𝑕 • Proof idea: Any non-multilinear term 𝑦 𝑗 can be replaced by 𝑔 . 𝐹 𝑕 2 ≥ 0 for all 𝑕 of degree ≤ 𝑒/2 • Corollary: ෨ ⬄ ෨ 𝐹[𝑕 2 ] for all multilinear 𝑕 of degree ≤ 𝑒/2 .

Key Idea: Equivalence Classes 𝑒 • Definition: For sets 𝐽, 𝐾 of size ≤ 2 , we say 𝑦 𝐽 ∼ 𝑦 𝐾 if 𝑦 𝐽 𝑦 𝐾 = 𝑦 𝐽Δ𝐾 is determined • Proposition: If 𝑦 𝐽 ∼ 𝑦 𝐾 and 𝑦 𝐾 ∼ 𝑦 𝐿 then 𝑦 𝐽 ∼ 𝑦 𝐿 . • Proof: If 𝑦 𝐽 ∼ 𝑦 𝐾 and 𝑦 𝐾 ∼ 𝑦 𝐿 then 𝑦 𝐽Δ𝐾 and 𝑦 𝐾Δ𝐿 are determined. Now 𝑦 𝐽Δ𝐾 𝑦 𝐾Δ𝐿 = 2 𝑦 𝐿 = 𝑦 𝐽Δ𝐿 is determined. Thus, 𝑦 𝐽 ∼ 𝑦 𝐿 𝑦 𝐽 𝑦 𝐾 • Remark: We carefully chose which deductions to make so that this would work.

PSD Decomposition • Proposition: ෨ 𝐹 𝑦 𝐽 𝑦 𝐾 ≠ 0 if and only 𝐽 ∼ 𝐾 . • Choose a representative 𝐽 𝐹 from every equivalence class 𝐹 . • Take 𝑤 𝐹 𝑦 𝐽 = ෨ 𝐹 𝑦 𝐽 𝑦 𝐽 𝐹 • 𝑤 𝐹 𝑦 𝐽 = 𝑑 𝐽Δ𝐽 𝐹 if 𝑦 𝐽 ∈ 𝐹 . Otherwise, 𝑤 𝐹 𝑦 𝐽 = 0 • 𝑤 𝐹 𝑦 𝐽 𝑤 𝐹 𝑦 𝐾 = 𝑑 𝐽Δ𝐽 𝐹 𝑑 JΔ𝐽 𝐹 = 𝑑 𝐽Δ𝐾 if 𝐽, 𝐾 ∈ 𝐹 . Otherwise, 𝑤 𝐹 𝑦 𝐽 𝑤 𝐹 𝑦 𝐾 = 0

PSD Decomposition • 𝑤 𝐹 𝑦 𝐽 𝑤 𝐹 𝑦 𝐾 = 𝑑 𝐽Δ𝐽 𝐹 𝑑 JΔ𝐽 𝐹 = 𝑑 𝐽Δ𝐾 if 𝐽, 𝐾 ∈ 𝐹 . Otherwise, 𝑤 𝐹 𝑦 𝐽 𝑤 𝐹 𝑦 𝐾 = 0 • Corollary: ∀𝐽, 𝐾 , σ 𝐹 𝑤 𝐹 𝑦 𝐽 𝑤 𝐹 𝑦 𝐾 = ෨ 𝐹 𝑦 𝐽 𝑦 𝐾 𝑈 ≽ 0 • Corollary: 𝑁 = σ 𝐹 𝑤 𝐹 𝑤 𝐹

Part IV: Analyzing Parameter Regimes

Parameter Regimes • How large does 𝑛 have to be before the random 3-XOR constraints are unsatisifable w.h.p.? • For which 𝑛 will the pseudo-expectation values be well-defined w.h.p., giving us the SOS lower bound?

Unsatisfiability of 3-XOR Constraints • For any given possible solution (𝑦 1 , … , 𝑦 𝑜 ) , the probability it is valid if there are 𝑛 random 3-XOR constraints is 2 −𝑛 . • Using a union bound, 𝑄 ∃𝑡𝑝𝑚𝑣𝑢𝑗𝑝𝑜 ≤ 2 𝑜−𝑛 • Equations are unsatisfiable w.h.p. if 𝑛 ≫ 𝑜 • In fact, not hard to show that ∀𝜗 > 0, ∃𝐷, 𝑜 0 > 0: if 𝑛 ≥ 𝐷𝑜, 𝑜 ≥ 𝑜 0 then 1 2 + 𝜗 of w.h.p. there is no solution satisfying the constraints

Local Consistency • If ෨ 𝐹 is not well-defined then we must be able to derive the contradiction −1 = 1 without going to degree higher than 2𝑒 . • Multiplying all of the constraints involved in such a contradiction, every variable appears an even number of times.

Local Contradiction Picture • Draw a triangle (𝑦 𝑗 𝑏 , 𝑦 𝑘 𝑏 , 𝑦 𝑙 𝑏 ) for each constraint 𝑦 𝑗 𝑏 𝑦 𝑘 𝑏 𝑦 𝑙 𝑏 = 𝑑 𝑏 involved in the contradiction. • Every vertex is covered an even number of times • Example: If we have the constraints 𝑦 1 𝑦 2 𝑦 3 = 1, 𝑦 4 𝑦 5 𝑦 6 = 1 , 𝑦 1 𝑦 2 𝑦 4 = 1 , 𝑦 3 𝑦 5 𝑦 6 = 1 , we get the following picture: 𝑦 1 𝑦 3 𝑦 5 𝑦 2 𝑦 4 𝑦 6