Recent advances on mean-field spin glasses Wei-Kuo Chen University - PowerPoint PPT Presentation

Recent advances on mean-field spin glasses Wei-Kuo Chen University of Minnesota Joint work with A. Auffinger, M. Handschy, D. Gamarnik, G. Lerman, D. Panchenko, M. Rahman, A. Sen, Q. Zeng July, 2018 1 / 17

What are spin glasses? 2 / 17

What are spin glasses? Spin Glasses are alloys with strange magnetic properties. Ex: CuMn - In physics: spin + glass - In mathematics: quenched disorder + frustration 2 / 17

What are spin glasses? Spin Glasses are alloys with strange magnetic properties. Ex: CuMn - In physics: spin + glass - In mathematics: quenched disorder + frustration Spin glass features appear in many real world problems: - Traveling salesman problem. - Hopfield neural network. - Spike detection and recovery problems. 2 / 17

Edwards-Anderson model Consider a finite graph ( V , E ) on Z d . Hamiltonian: For σ ∈ {− 1 , 1 } V , � H ( σ ) = g ij σ i σ j , ( i , j ) ∈ E where g ij are i.i.d. N ( 0 , 1 ) . 3 / 17

Edwards-Anderson model Consider a finite graph ( V , E ) on Z d . Hamiltonian: For σ ∈ {− 1 , 1 } V , � H ( σ ) = g ij σ i σ j , ( i , j ) ∈ E where g ij are i.i.d. N ( 0 , 1 ) . Frustration appears when computing max H N ( σ ) . Figure: Frustration 3 / 17

Mean field approach: The Sherrington-Kirkpatrick model Hamiltonian: N N 1 � � H N ( σ ) = √ g ij σ i σ j + h σ i N i , j = 1 i = 1 i . i . d . for σ ∈ {− 1 , + 1 } N , where g ij ∼ N ( 0 , 1 ) . 4 / 17

Mean field approach: The Sherrington-Kirkpatrick model Hamiltonian: N N 1 � � H N ( σ ) = √ g ij σ i σ j + h σ i N i , j = 1 i = 1 i . i . d . for σ ∈ {− 1 , + 1 } N , where g ij ∼ N ( 0 , 1 ) . Covariance Structure: � 1 �� 1 N N � � 2 , � g ij σ 1 i σ 1 � g ij σ 2 i σ 2 R ( σ 1 , σ 2 ) � E √ √ = N j j N N i , j = 1 i , j = 1 where N R ( σ 1 , σ 2 ) = 1 � σ 1 i σ 2 i . N i = 1 4 / 17

Dean’s problem Assign N students into two dorms and avoid conflicts. 5 / 17

Dean’s problem Assign N students into two dorms and avoid conflicts. 5 / 17

Dean’s problem Assign N students into two dorms and avoid conflicts. 5 / 17

Dean’s problem Assign N students into two dorms and avoid conflicts. Dean’s problem: Find the optimizer of N � max g ij σ i σ j . σ ∈{− 1 , + 1 } N i , j = 1 5 / 17

A soft approximation: Free energy For any β = 1 T > 0 (inverse temperature), define the free energy 1 � e β H N ( σ ) F N ( β ) = β N log σ ∈{− 1 , + 1 } N 6 / 17

A soft approximation: Free energy For any β = 1 T > 0 (inverse temperature), define the free energy 1 � e β H N ( σ ) F N ( β ) = β N log σ ∈{− 1 , + 1 } N Simple observation: H N ( σ ) H N ( σ ) + log 2 max ≤ F N ( β ) ≤ max N N β σ ∈{− 1 , + 1 } N σ ∈{− 1 , + 1 } N 6 / 17

A soft approximation: Free energy For any β = 1 T > 0 (inverse temperature), define the free energy 1 � e β H N ( σ ) F N ( β ) = β N log σ ∈{− 1 , + 1 } N Simple observation: H N ( σ ) H N ( σ ) + log 2 max ≤ F N ( β ) ≤ max N N β σ ∈{− 1 , + 1 } N σ ∈{− 1 , + 1 } N Physicists’ replica method: E log Z n log E Z n 1 ? N N lim N E log Z N = lim N →∞ lim = lim n ↓ 0 lim nN nN N →∞ n ↓ 0 N →∞ 6 / 17

Theorem (Parisi formula) (Talagrand ’06) � 1 Φ α,β ( 0 , h ) − 1 � � N →∞ F N ( β ) = inf lim βα ( s ) sds , a . s ., 2 α 0 where for any CDF α on [ 0 , 1 ] , ∂ s Φ α,β = − 1 � ∂ xx Φ α,β + βα ( s )( ∂ x Φ α,β ) 2 � , ∀ ( s , x ) ∈ [ 0 , 1 ) × R 2 with Φ α,β ( 1 , x ) = 1 β log cosh( β x ) . (Guerra’ 03) Minimizer exists. (Auffinger-C. ’14) Minimizer is unique. Denote this minimizer by α β and call it the Parisi measure. 7 / 17

Significance of the Parisi measure Three major predictions: (1) α β is the limiting distribution of the overlap: d R ( σ 1 , σ 2 ) ⇒ α β , where σ 1 , σ 2 are i.i.d. samplings from the Gibbs measure e β H N ( σ ) G N ( σ ) = σ ′ e β H N ( σ ′ ) . � 8 / 17

(2) Phase Transition: 1 1 Aizemann-Lebowitz-Ruelle ’87 Toninelli ’02 q 0 1 1 Replica Symmetric M Full Replica Symmetry Breaking Figure: SK model with h = 0 9 / 17

(3) Ultrametricity: with probab. ≈ 1, for i.i.d. σ 1 , σ 2 , σ 3 ∼ G N , � σ 1 − σ 2 �≤ max � σ 1 − σ 3 � , � σ 2 − σ 3 � � � + o ( 1 ) . The whole space Pure States (no further structure) 10 / 17

(3) Ultrametricity: with probab. ≈ 1, for i.i.d. σ 1 , σ 2 , σ 3 ∼ G N , � σ 1 − σ 2 �≤ max � σ 1 − σ 3 � , � σ 2 − σ 3 � � � + o ( 1 ) . The whole space Pure States (no further structure) Panchenko ’11: Ultrametricity holds for the SK model with a vanishing perturbation, but we do not know if it is still true without perturbation. 10 / 17

Theorem (Auffinger-C.-Zeng ’17) The cardinality of supp α β diverges as β → ∞ . As a consequence: If we add perturbation so that ultrametricity holds, then the total levels of the trees diverge as β ↑ ∞ . 11 / 17

Parisi formula for the maximal energy 12 / 17

Parisi formula for the maximal energy � 1 For any γ with γ ( s ) = µ ([ 0 , s ]) and 0 γ ( s ) ds < ∞ , consider the PDE solution Ψ γ , Ψ γ ( 1 , x ) = | x | , ∂ s Ψ γ = − 1 � ∂ xx Ψ γ + γ ( s )( ∂ x Ψ γ ) 2 � , ∀ ( s , x ) ∈ [ 0 , 1 ) × R . 2 12 / 17

Parisi formula for the maximal energy � 1 For any γ with γ ( s ) = µ ([ 0 , s ]) and 0 γ ( s ) ds < ∞ , consider the PDE solution Ψ γ , Ψ γ ( 1 , x ) = | x | , ∂ s Ψ γ = − 1 � ∂ xx Ψ γ + γ ( s )( ∂ x Ψ γ ) 2 � , ∀ ( s , x ) ∈ [ 0 , 1 ) × R . 2 Theorem (Auffinger-C. ’16) Parisi formula at zero temperature: � 1 H N ( σ ) Ψ γ ( 0 , h ) − 1 � � lim max = inf s γ ( s ) ds N →∞ E N 2 σ ∈{− 1 , + 1 } N γ 0 (C.-Handschy-Lerman ’16) Minimizer γ P exists and is unique. 12 / 17

Energy landscape: multiple peaks � N Overlap R ( σ, σ ′ ) = 1 i = 1 σ i σ ′ i . N 13 / 17

Energy landscape: multiple peaks � N Overlap R ( σ, σ ′ ) = 1 i = 1 σ i σ ′ i . N Theorem (Multiple peaks, C.-Handschy-Lerman ’16) Assume h = 0 . For any ε > 0 , there exists a constant K > 0 s.t. for any N ≥ 1 , with probability at least 1 − Ke − N / K , ∃ S N ⊂ {− 1 , + 1 } N such that ( i ) | S N | ≥ e N / K . � H N ( σ ′ ) � � H N ( σ ) ( ii ) ∀ σ ∈ S N , − max σ ′ ∈ Σ N � < ε. � � N N ( iii ) ∀ σ, σ ′ ∈ S N with σ � = σ ′ , | R ( σ, σ ′ ) | < ε . 13 / 17

Energy landscape: multiple peaks � N Overlap R ( σ, σ ′ ) = 1 i = 1 σ i σ ′ i . N Theorem (Multiple peaks, C.-Handschy-Lerman ’16) Assume h = 0 . For any ε > 0 , there exists a constant K > 0 s.t. for any N ≥ 1 , with probability at least 1 − Ke − N / K , ∃ S N ⊂ {− 1 , + 1 } N such that ( i ) | S N | ≥ e N / K . � H N ( σ ′ ) � � H N ( σ ) ( ii ) ∀ σ ∈ S N , − max σ ′ ∈ Σ N � < ε. � � N N ( iii ) ∀ σ, σ ′ ∈ S N with σ � = σ ′ , | R ( σ, σ ′ ) | < ε . Chatterjee ’09: | S N | ≥ (log N ) c . Ding-Eldan-Zhai ’14: | S N | ≥ N c . 13 / 17

Pure p -spin model for p ≥ 3 : Overlap gap property Hamiltonian: 1 � H N ( σ ) = g i 1 ,..., i p σ i 1 · · · σ i p . N ( p − 1 ) / 2 1 ≤ i 1 ,..., i p ≤ N (Overlap gap property) There exist c , C > 0 such that with overwhelming probability, any two near ground states σ 1 and σ 2 satisfy | R ( σ 1 , σ 2 ) | / ∈ [ c , C ] . 14 / 17

Computational hardness: 15 / 17

Computational hardness: - Suppose σ 1 is a near ground state and σ 2 is the ground state so that | R ( σ 1 , σ 2 ) | ≤ c . 15 / 17

Computational hardness: - Suppose σ 1 is a near ground state and σ 2 is the ground state so that | R ( σ 1 , σ 2 ) | ≤ c . - Locally update algorithms take exponential time to find the ground state since | R ( σ 1 , σ ( n )) | / ∈ [ c , C ] . 15 / 17

Computational hardness: - Suppose σ 1 is a near ground state and σ 2 is the ground state so that | R ( σ 1 , σ 2 ) | ≤ c . - Locally update algorithms take exponential time to find the ground state since | R ( σ 1 , σ ( n )) | / ∈ [ c , C ] . Results: - C.-Gamarnik-Rahman-Panchenko ’17 - Jagannath-Ben Arous ’17 15 / 17

New challenges Bipartite SK model: Let N 1 = cN and N 2 = ( 1 − c ) N . N 1 N 2 1 � � √ H N ( σ ) = g ij τ i ρ j N i = 1 j = 1 for σ = ( τ, ρ ) ∈ {− 1 , + 1 } N 1 × {− 1 , + 1 } N 2 . Note E H N ( σ ) H N ( σ ′ ) = c ( 1 − c ) NR ( τ, τ ′ ) R ( ρ, ρ ′ ) . Questions: - Free energy? - Ground state energy? - Energy landscape? 16 / 17

Thank you for your attention. 17 / 17