Randomness vs structure DNF compression and sunflower lemma Kewen - PowerPoint PPT Presentation

Overview DNF compression Sunflower lemma Open problems Thanks Randomness vs structure DNF compression and sunflower lemma Kewen Wu Peking University Joint works with Ryan Alweiss Shachar Lovett Jiapeng Zhang Princeton UCSD Harvard 1 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Overview This talk is about two recent works accepted by STOC 2020. Decision list compression by mild random restrictions Shachar Lovett, Kewen Wu , Jiapeng Zhang Improved bounds for the sunflower lemma Ryan Alweiss, Shachar Lovett, Kewen Wu , Jiapeng Zhang The main approach is to study mild random restrictions . Small-width DNFs simplify under (mild) random restrictions. 2 / 25

Overview DNF compression Sunflower lemma Open problems Thanks DNF (disjunctive normal form) literal: x i or ¬ x i term: a conjuction of literals DNF: a disjunction of terms width of a DNF: maximum number of literals in a term size of a DNF: number of conjunctions Example ( x 1 ∧ x 2 ) ∨ ( ¬ x 1 ∧ x 3 ∧ x 5 ) is a DNF of width 3 and size 2. 3 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Section 2 DNF compression 4 / 25

Overview DNF compression Sunflower lemma Open problems Thanks What we proved Definition ( ε -approximation) f is ε -approximated by g if Pr x ∼{ 0 , 1 } n [ f ( x ) � = g ( x )] ≤ ε. Theorem (DNF compression) Width- w DNF can be ε -approximated by a width- w size- s DNF. Gopalan, Meka and Reingold 2013: s = ( w log(1 /ε )) O ( w ) . Lovett and Zhang 2019: s = (1 /ε ) O ( w ) . � O ( w ) and this is tight. 2 + 1 w log 1 � Now: s = ε 5 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Applications Decision list compression Small-width decision lists can be approximated by decision lists of same width and small size. If C 1 ( x ) = True then output v 1 , . . . , else if C m − 1 ( x ) = True then output v m − 1 , else output default value v m . Junta theorem Small-width DNFs can be approximated by DNFs of same width and few input bits. Learning small-width DNFs Small-width DNFs are efficiently PAC learnable. 6 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview How we proved – more definitions Let f = C 1 ∨ · · · ∨ C m be a DNF. Definition (Index function Ind f ) Ind f ( x ) is the index of first satisfied term (or ⊥ if f ( x ) = 0 ). Definition (Useful index) Index i is useful if there exists x such that Ind f ( x ) = i . # useful ( f ) is the number of useful indices. Example Assume f = ( x 1 ∧ x 2 ) ∨ ( x 2 ) ∨ ( x 1 ∧ x 2 ∧ ¬ x 3 ) . Then Ind f (1 , 1 , 0) = 1 and # useful ( f ) = 2 . 7 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview How we proved – Step 1: randomness kills structure We should be able to compress f (in some sense) under restrictions. Lemma (H˚ astad’s switching lemma 1987) Let f be a width- w DNF, α ∈ (0 , 1) , and d be an integer. If ρ randomly restrict each input bit to 0 , 1 , ∗ w.p. (1 − α ) / 2 , (1 − α ) / 2 , α , then ρ [DecisionTree( f ↾ ρ ) ≥ d ] ≤ (5 αw ) d . Pr Prove by encoding bad ρ . Meaningful only when α ≤ O (1 /w ) = ⇒ most bits are fixed. 8 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview Mild random restrictions Let’s directly analyze f ’s size under restrictions. Lemma Let f be a width- w DNF, α ∈ (0 , 1) , and s be an integer. If ρ randomly restrict each input bit to 0 , 1 , ∗ w.p. (1 − α ) / 2 , (1 − α ) / 2 , α , then � w ρ [# useful ( f ↾ ρ ) ≥ s ] ≤ 1 � 4 Pr . s 1 − α Prove by encoding bad ρ , ( ρ, i ) → ( ρ ′ , aux ) . ρ ′ activates ∗ ’s in C i ↾ ρ for useful i = ⇒ Ind f ( ρ ′ ) = i . Meaningful for all kinds of α . 9 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview How we proved – Step 2: intuition for compression Let f = C 1 ∨ · · · ∨ C m be a width- w DNF. If index i is not useful, we can safely remove C i . Assume p i = Pr x [ Ind f ( x ) = i ] is decreasing in i . If p i decrease quickly, we only need to keep the top ones. Let g = C 1 ∨ · · · ∨ C t . Then � Pr [ f ( x ) � = g ( x )] = Pr [ Ind f ( x ) > t ] = p i . i>t 10 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview Now what? Let f = C 1 ∨ · · · ∨ C m be a width- w DNF. What we can do so far? We can analyze q i ( α ) = Pr ρ [ index i is useful in f ↾ ρ ] , since � ρ [# useful ( f ↾ ρ )] . q i ( α ) = E i What we want to do next? We want to bound p i = Pr x [ Ind f ( x ) = i ] . 11 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview How we proved – Step 3: noise stability Let’s introduce noise stability. Definition (Noise distribution N β ) y ∼ N β ( x ) is sampled by taking Pr[ y i = x i ] = (1 + β ) / 2 . Then for x ∼ { 0 , 1 } n , y ∼ N β ( x ) , we can also do it as 1 sample common restriction ρ with Pr [ ρ i = ∗ ] = 1 − β ; 2 sample x ′ by uniformly filling out ∗ ’s in ρ , and set x = ρ ◦ x ′ ; 3 sample y ′ by uniformly filling out ∗ ’s in ρ , and set y = ρ ◦ y ′ . 12 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview How we proved – Step 4: bridging lemma Let f = C 1 ∨ · · · ∨ C m be a width- w DNF and fix i . Sample x ∼ { 0 , 1 } n , y ∼ N β ( x ) , which can be seen as ρ, x ′ , y ′ . Define Stab ( β ) = Pr [ Ind f ( x ) = Ind f ( y ) = i ] and recall p = Pr[ Ind f ( x ) = i ] , q = Pr[ index i is useful in f ↾ ρ ] . Fact (Hypercontractivity) 2 2 1+ β = p 1+ β . Stab ( β ) ≤ (Pr [ Ind f ( x ) = i ]) We also have Stab ( β ) = Pr [ Ind f ( x ) = Ind f ( y ) = i, index i is useful in f ↾ ρ ] = q Pr [ Ind f ( x ) = Ind f ( y ) = i | index i is useful in f ↾ ρ ] ≥ q (Pr [ Ind f ( x ) = i | index i is useful in f ↾ ρ ]) 2 = p 2 /q. 13 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Section 3 Sunflower lemma 14 / 25

Overview DNF compression Sunflower lemma Open problems Thanks What we proved Definition ( w -set system and r -sunflower) A w -set system is a family of sets of size at most w . An r -sunflower is r sets with same pairwise intersection. Theorem (Erd˝ os-Rado sunflower lemma) Any w -set system of size s has r -sunflower. Let’s focus on r = 3 . os and Rado 1960: s = w ! · 2 w ≈ w w . Erd˝ Kostochka 2000: s ≈ ( w log log log w/ log log w ) w . Fukuyama 2018: s ≈ w 0 . 75 w . Now: s ≈ (log w ) w and this is tight for our approach. 15 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Applications Combinatorics Intersecting set systems Erd˝ os-Szemer´ edi sunflower lemma Alon-Jaeger-Tarsi conjecture Random graph Theoretical computer science Circuit lower bounds and data structure lower bounds Matrix multiplication Pseudorandomness: DNF compression Cryptography Property testing Fixed parameter complexity 16 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview How we proved – Step 1: make it robust Assume F = { S 1 , . . . , S m } is a w -set system. Define a width- w DNF f F as f F = � m � j ∈ S i x j . i =1 Definition (Satisfying system) F is satisfying if Pr [ f F ( x ) = 0] < 1 / 3 with Pr [ x i = 1] = 1 / 3 , i.e., Pr [ ∀ i ∈ [ m ] , S i �⊂ S ] < 1 / 3 with Pr [ x i ∈ S ] = 1 / 3 . Lemma If F is satisfying, then it has 3 pairwise disjoint sets (3-sunflower). Prove by randomly 3-coloring x and union bound. 17 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview How we proved – Step 2: induction Assume F = { S 1 , . . . , S m } , m > κ w is a w -set system. Define link F Y = { S i \ Y | Y ⊂ S i } , which is a ( w − | Y | ) -set system. If there exists Y such that |F Y | ≥ m/κ | Y | > κ w −| Y | , then we can apply induction and find 3 -sunflower in F Y . So induction starts at such F , that |F Y | <m/κ | Y | holds for any Y . Thus it suffices to prove Lemma Let κ ≥ (log w ) O (1) . If |F Y | < m/κ | Y | holds for any Y , then F is satisfying, which means there are 3 pairwise disjoint sets in F . 18 / 25

Overview DNF compression Sunflower lemma Open problems Thanks Proof overview How we proved – Step 3: randomness preserves structure Assume F = { S 1 , . . . , S m } , m > κ w is a w -(multi-)set system. Assume |F Y | < m/κ | Y | holds for any Y . ⇐ F is structured Take ≈ 1 / √ κ -fraction of the ground set as W , and construct a w/ 2 -(multi-)set system F ′ from each S i : Good: If there exists | S j \ W | ≤ w/ 2 and S j \ W ⊂ S i \ W , then put S j \ W into F ′ ; ( j may equal i ) To satisfy {{ x 1 , x 2 } , { x 1 , x 2 , x 3 , x 5 } , { x 4 }} , it suffices to satisfy {{ x 1 , x 2 } , { x 1 , x 2 } , { x 4 }} . Bad: otherwise, we do nothing for S i . Y | ≤ |F Y | , ∀ Y . ⇐ F ′ is also structured Then |F ′ | ≈ m and |F ′ Prove by encoding bad ( W, i ) → ( W ′ = W ∪ S i , k, aux ) , where S j \ W ⊂ S i \ W and S i ranks k < m/κ w/ 2 in F S j ∩ S i . 19 / 25