Performance of Local Algorithms in Random Structures. Power and - PowerPoint PPT Presentation

Performance of Local Algorithms in Random Structures. Power and limitations David Gamarnik MIT Sub-Linear Algorithms Bootcamp June, 2018 D.Gamarnik Local Algorithms in Random Structures June, 2018 1 / 29

Outline Part I. Challenges Part II. Solution Space Geometry Part III. Local Algorithms D.Gamarnik Local Algorithms in Random Structures June, 2018 2 / 29

Part I. Challenges D.Gamarnik Local Algorithms in Random Structures June, 2018 3 / 29

(Arguably) Most Embarrassing Algorithmic Problem in Random Graphs D.Gamarnik Local Algorithms in Random Structures June, 2018 4 / 29

(Arguably) Most Embarrassing Algorithmic Problem in Random Graphs Consider G ( n , p ) . D.Gamarnik Local Algorithms in Random Structures June, 2018 4 / 29

(Arguably) Most Embarrassing Algorithmic Problem in Random Graphs Consider G ( n , p ) . The largest clique (fully connected subgraph) is 2 ( 1 + o ( 1 )) log 1 p n . D.Gamarnik Local Algorithms in Random Structures June, 2018 4 / 29

(Arguably) Most Embarrassing Algorithmic Problem in Random Graphs Consider G ( n , p ) . The largest clique (fully connected subgraph) is 2 ( 1 + o ( 1 )) log 1 p n . A trivial greedy algorithm finds a clique of size ( 1 + o ( 1 )) log 1 p n . D.Gamarnik Local Algorithms in Random Structures June, 2018 4 / 29

(Arguably) Most Embarrassing Algorithmic Problem in Random Graphs Consider G ( n , p ) . The largest clique (fully connected subgraph) is 2 ( 1 + o ( 1 )) log 1 p n . A trivial greedy algorithm finds a clique of size ( 1 + o ( 1 )) log 1 p n . Karp [1976] Find a better algorithm. D.Gamarnik Local Algorithms in Random Structures June, 2018 4 / 29

(Arguably) Most Embarrassing Algorithmic Problem in Random Graphs Consider G ( n , p ) . The largest clique (fully connected subgraph) is 2 ( 1 + o ( 1 )) log 1 p n . A trivial greedy algorithm finds a clique of size ( 1 + o ( 1 )) log 1 p n . Karp [1976] Find a better algorithm. Still open. This is embarrassing... D.Gamarnik Local Algorithms in Random Structures June, 2018 4 / 29

More Examples: Ind Sets in Sparse Random Graphs D.Gamarnik Local Algorithms in Random Structures June, 2018 5 / 29

More Examples: Ind Sets in Sparse Random Graphs enyi G ( n , d Erd¨ os-R´ n ) or random d -regular graph G d ( n ) . D.Gamarnik Local Algorithms in Random Structures June, 2018 5 / 29

More Examples: Ind Sets in Sparse Random Graphs enyi G ( n , d Erd¨ os-R´ n ) or random d -regular graph G d ( n ) . The largest independent set (a largest subset of nodes with no edges in between) is 2 ( 1 + o d ( 1 )) log d n , d Frieze, Luczak [1992] D.Gamarnik Local Algorithms in Random Structures June, 2018 5 / 29

More Examples: Ind Sets in Sparse Random Graphs enyi G ( n , d Erd¨ os-R´ n ) or random d -regular graph G d ( n ) . The largest independent set (a largest subset of nodes with no edges in between) is 2 ( 1 + o d ( 1 )) log d n , d Frieze, Luczak [1992] Greedy algorithm finds an independent set of size ( 1 + o d ( 1 )) log d n . d D.Gamarnik Local Algorithms in Random Structures June, 2018 5 / 29

More Examples: Ind Sets in Sparse Random Graphs enyi G ( n , d Erd¨ os-R´ n ) or random d -regular graph G d ( n ) . The largest independent set (a largest subset of nodes with no edges in between) is 2 ( 1 + o d ( 1 )) log d n , d Frieze, Luczak [1992] Greedy algorithm finds an independent set of size ( 1 + o d ( 1 )) log d n . d Nothing better is known. D.Gamarnik Local Algorithms in Random Structures June, 2018 5 / 29

More Examples: Coloring of Sparse Random Graphs D.Gamarnik Local Algorithms in Random Structures June, 2018 6 / 29

More Examples: Coloring of Sparse Random Graphs enyi G ( n , d Erd¨ os-R´ n ) or random d -regular graph G d ( n ) . D.Gamarnik Local Algorithms in Random Structures June, 2018 6 / 29

More Examples: Coloring of Sparse Random Graphs enyi G ( n , d Erd¨ os-R´ n ) or random d -regular graph G d ( n ) . Chromatic number is χ ( G ) = ( 1 + o d ( 1 )) d . 2 log d D.Gamarnik Local Algorithms in Random Structures June, 2018 6 / 29

More Examples: Coloring of Sparse Random Graphs enyi G ( n , d Erd¨ os-R´ n ) or random d -regular graph G d ( n ) . Chromatic number is χ ( G ) = ( 1 + o d ( 1 )) d . 2 log d Greedy can color with twice as many colors ( 1 + o d ( 1 )) d . log d D.Gamarnik Local Algorithms in Random Structures June, 2018 6 / 29

More Examples: Coloring of Sparse Random Graphs enyi G ( n , d Erd¨ os-R´ n ) or random d -regular graph G d ( n ) . Chromatic number is χ ( G ) = ( 1 + o d ( 1 )) d . 2 log d Greedy can color with twice as many colors ( 1 + o d ( 1 )) d . log d Nothing better is known. D.Gamarnik Local Algorithms in Random Structures June, 2018 6 / 29

More Examples: Random K-SAT D.Gamarnik Local Algorithms in Random Structures June, 2018 7 / 29

More Examples: Random K-SAT Generate an instance Φ( n , dn ) of the K-SAT problem u.a.r. on n boolean 0 , 1 variables and dn clauses. x 2 , 1 ∨ · · · ¯ ( x 1 , 1 ∨ · · · ∨ ¯ x 1 , K ) ∧ (¯ ∨ x 2 , K ) ∧ · · · ∧ (¯ x dn , 1 ∨ · · · ∨ x dn , K ) D.Gamarnik Local Algorithms in Random Structures June, 2018 7 / 29

More Examples: Random K-SAT Generate an instance Φ( n , dn ) of the K-SAT problem u.a.r. on n boolean 0 , 1 variables and dn clauses. x 2 , 1 ∨ · · · ¯ ( x 1 , 1 ∨ · · · ∨ ¯ x 1 , K ) ∧ (¯ ∨ x 2 , K ) ∧ · · · ∧ (¯ x dn , 1 ∨ · · · ∨ x dn , K ) Satisfiable iff d ≤ ( 1 + o K ( 1 ) 2 K log 2 ≡ d ∗ ( K ) Achlioptas & Moore [2002], Achlioptas & Peres [2003], Coja-Oghlan [2014], Ding, Sly & Sun [2015] D.Gamarnik Local Algorithms in Random Structures June, 2018 7 / 29

More Examples: Random K-SAT Generate an instance Φ( n , dn ) of the K-SAT problem u.a.r. on n boolean 0 , 1 variables and dn clauses. x 2 , 1 ∨ · · · ¯ ( x 1 , 1 ∨ · · · ∨ ¯ x 1 , K ) ∧ (¯ ∨ x 2 , K ) ∧ · · · ∧ (¯ x dn , 1 ∨ · · · ∨ x dn , K ) Satisfiable iff d ≤ ( 1 + o K ( 1 ) 2 K log 2 ≡ d ∗ ( K ) Achlioptas & Moore [2002], Achlioptas & Peres [2003], Coja-Oghlan [2014], Ding, Sly & Sun [2015] Algorithmically can find a satisfying assignment only when d ≤ log K d ∗ ( K ) . K Coja-Oghlan [2011] D.Gamarnik Local Algorithms in Random Structures June, 2018 7 / 29

More Examples: Random K-SAT Generate an instance Φ( n , dn ) of the K-SAT problem u.a.r. on n boolean 0 , 1 variables and dn clauses. x 2 , 1 ∨ · · · ¯ ( x 1 , 1 ∨ · · · ∨ ¯ x 1 , K ) ∧ (¯ ∨ x 2 , K ) ∧ · · · ∧ (¯ x dn , 1 ∨ · · · ∨ x dn , K ) Satisfiable iff d ≤ ( 1 + o K ( 1 ) 2 K log 2 ≡ d ∗ ( K ) Achlioptas & Moore [2002], Achlioptas & Peres [2003], Coja-Oghlan [2014], Ding, Sly & Sun [2015] Algorithmically can find a satisfying assignment only when d ≤ log K d ∗ ( K ) . K Coja-Oghlan [2011] Nothing better is known. D.Gamarnik Local Algorithms in Random Structures June, 2018 7 / 29

Examples in Statistics. Largest submatrix problem D.Gamarnik Local Algorithms in Random Structures June, 2018 8 / 29

Examples in Statistics. Largest submatrix problem   C 11 . . . C 1 n . . ... d . . C n =  , C i , j = i.i.d. N ( 0 , 1 )   . .  C n 1 . . . C nn D.Gamarnik Local Algorithms in Random Structures June, 2018 8 / 29

Examples in Statistics. Largest submatrix problem   C 11 . . . C 1 n . . ... d . . C n =  , C i , j = i.i.d. N ( 0 , 1 )   . .  C n 1 . . . C nn Given k , find I , J ⊂ [ n ] , | I | = | J | = k Ave ( C I , J ) . max D.Gamarnik Local Algorithms in Random Structures June, 2018 8 / 29

Examples in Statistics. Largest submatrix problem   C 11 . . . C 1 n . . ... d . . C n =  , C i , j = i.i.d. N ( 0 , 1 )   . .  C n 1 . . . C nn Given k , find I , J ⊂ [ n ] , | I | = | J | = k Ave ( C I , J ) . max Bhamidi, Dey & Nobel [2017]. Largest value is � log n max Ave ( C I , J ) = 2 ( 1 + o ( 1 )) . k D.Gamarnik Local Algorithms in Random Structures June, 2018 8 / 29

Examples in Statistics. Largest submatrix problem   C 11 . . . C 1 n . . ... d . . C n =  , C i , j = i.i.d. N ( 0 , 1 )   . .  C n 1 . . . C nn Given k , find I , J ⊂ [ n ] , | I | = | J | = k Ave ( C I , J ) . max Bhamidi, Dey & Nobel [2017]. Largest value is � log n max Ave ( C I , J ) = 2 ( 1 + o ( 1 )) . k Algorithmically (G & Li [2017]) � Ave A LG ( C I , J ) = 4 ( 1 + o ( 1 )) log n √ . k 3 2 D.Gamarnik Local Algorithms in Random Structures June, 2018 8 / 29

Examples in Statistics. Sparse Regression D.Gamarnik Local Algorithms in Random Structures June, 2018 9 / 29

Examples in Statistics. Sparse Regression X ∈ R n × p , W ∈ R n i.i.d. N ( 0 , σ 2 ) . D.Gamarnik Local Algorithms in Random Structures June, 2018 9 / 29

Examples in Statistics. Sparse Regression X ∈ R n × p , W ∈ R n i.i.d. N ( 0 , σ 2 ) . β ∗         Y 1 X 11 X 12 . . . X 1 p W 1 1 . . . . . . ... . . . . . .  =  +  .         . . . . . .      β ∗ Y n X n 1 X n 2 . . . X np W n p D.Gamarnik Local Algorithms in Random Structures June, 2018 9 / 29