Replica Cluster Variational Method Federico Ricci-Tersenghi - PowerPoint PPT Presentation

Replica Cluster Variational Method Federico Ricci-Tersenghi Physics Department, Sapienza University of Rome joint work with Tommaso Rizzo, Alejandro Lage-Castellanos and Roberto Mulet arXiv:0906.2695

Bethe-Peierls approx. Belief Propagation (BP) random graph topologies (i.e. locally tree-like) replica symmetric (RS) single state, single BP f.p.

Bethe-Peierls approx. Belief Propagation (BP) BEYOND BP Kikuchi/GBP Survey Propagation (SP) topological structures counts the number of e.g. regular lattices states (a.k.a. complexity)

Bethe-Peierls approx. Belief Propagation (BP) BEYOND BP Kikuchi/GBP Survey Propagation (SP) topological structures counts the number of e.g. regular lattices states (a.k.a. complexity) GSP?! from Wikipedia: “The cluster variational method and the survey propagation algorithms are two different improvements to belief propagation. The name generalized survey propagation (GSP) is waiting to be assigned to the algorithm that merges both generalizations. ”

Models we are interested in Spin glasses on regular lattices Edwards-Anderson (EA) model β = 1 � σ ] ∝ e − β H , H = − P [ � J ij σ i σ j , T <ij> Topologies with many short loops. Quenched disorder, frustration... are Gaussian or ±1 i.i.d. r.v. J ij Ising spins =±1 σ i

±J EA model 3D L=32 T=0.7 Complex systems at low temprature with many state and metastable states

Two type of questions Average over the ensemble mean free-energy, energy and entropy dominated by typical samples Properties of a given sample free-energy, energy, entropy and marginal probabilities

Two kind of results Analytic results for quantities averaged over the ensemble An algorithm for computing marginals on a given sample Many links between the two...

Kikuchi’ s CVM � � F = H [ � σ ] P [ � σ ] − T P [ � σ ] log P [ � σ ] � σ � σ Energy: easy Entropy: difficult Mean field � P [ � σ ] = P i ( σ i ) i P ij ( σ i , σ j ) Bethe � � P [ � σ ] = P i ( σ i ) P i ( σ i ) P j ( σ j ) <ij> i CVM (plaquette)

Bethe free-energy � � = F J ij σ i σ j P ij − σ i , σ j <ij> � � � � + T P ij ln P ij − T ( d i − 1) P i ln P i σ i , σ j <ij> i σ i + constraints imposing normalizations and consistency � P ij ( σ i , σ j ) = P i ( σ i ) σ j Lagrange multipliers are the messages in the MPA

Kikuchi free-energy �� � � � F = c r P r E r + T P r ln P r r x r x r to be minimized under normalization and compatibility constraints � P r = P s x r \ s possibly with a fast MPA (GBP) sending messages = Lagrange multipliers

Alternative expression for the Kikuchi free-energy   � � � F = − c r ln ψ r ( x r ) m rs  r x r m rs ∈ M ( r ) Partial derivatives w.r.t. -> BP/GBP eqs. m rs No P ln P term

Plaquette CVM 2 kind of messages u m ∝ e − β u u 1 u 2 U 2 kind of equations = =

Plaquette CVM 2 kind of messages u m ∝ e − β u u 1 u 2 U 2 kind of equations = = Single and triple messages appear together!

Introducing RSB The cavity interpretation of messages turns out to be wrong beyond Bethe We came back to the replica trick and the hierarchical ansatz We obtained general expressions for the free-energy at any level of RSB and any set of regions in the CVM

1RSB CVM for a given sample (GSP) messages become functions of messages and satisfying region q ( u ) Q ( U, u 1 , u 2 ) ∀ � dQ 1 () dQ 2 () dq 1 () dq 2 () dq 3 () δ [ u − f ()] N m 1 q ( u ) = � N m 1 � Q 1 q 1 q = N 1 q 3 q 2 Q 2

1RSB CVM for a given sample (GSP) � Q ( U, x 1 , x 2 ) q 1 ( u 1 − x 1 ) q 2 ( u 2 − x 2 ) = 1 � dQ 1 () dQ 2 () dQ 3 () dq 3 () dq 4 () dq 5 () dq 6 () � N m 3 � δ [ U − F ()] δ [ u 1 − f 1 ()] δ [ u 2 − f 2 ()] N m 3 q 1 = Q N 3 q 2

1RSB CVM for a given sample (GSP) From fixed point functions and Q ( U, u 1 , u 2 ) q ( u ) compute the replicated free-energy � � F ( m ) = − c r ln dQ () . . . dq () [ N r ()] m r and by Legendre trasform the complexity Σ ( f ) Marginals will depend on the free-energy value

RS CVM average case No marginals, but just free-energy � F = − c r ln � N r � J r Average over the disorder Traslation invariance on the lattice Just one equation per kind of message

RS CVM average case � q ( u ) = dQ () dQ () dq () dq () dq () δ [ u − f ()] � Q ( U, x 1 , x 2 ) q ( u 1 − x 1 ) q ( u 2 − x 2 ) dx 1 dx 2 = � dQ () dQ () dQ () dq () dq () dq () dq () δ [ U − F ()] δ [ u 1 − f 1 ()] δ [ u 2 − f 2 ()]

RS CVM average case (difficulties) and are not distributions Q ( U, u 1 , u 2 ) q ( u ) nice cavity interpretation fails some numerical problems signed populations or histograms Fourier transform to solve the convolution

Analytical results for the Edwards-Anderson model Bethe: Q ( U, u 1 , u 2 ) = δ ( U ) δ ( u 1 ) δ ( u 2 ) Solve for q ( u ) for T > T Bethe � δ ( u ) ∃ T Bethe : q ( u ) = c broad and for T < T Bethe c symmetric c Paramagnetic ( ): m i = 0 ∀ i Q ( U, u 1 , u 2 ) = Q ( U ) δ ( u 1 ) δ ( u 2 ) q ( u ) = δ ( u ) Solve for Q ( U )

RS CVM for EA 2D d tanh( β U ) = tanh( β ( J 1 + U 1 )) tanh( β ( J 2 + U 2 )) tanh( β ( J 3 + U 3 )) 2.5 Q ( U ) 2 at 1.5 T = 0 . 1 1 0.5 -2 -1 1 2 U fields concentrate over the integers for T->0

RS CVM for ±J EA 2D F true -1.4 RS CVM -1.5 Bethe -1.6 Text -1.7 -1.8 T Bethe c -1.9 2 T 0.5 1 1.5 Entropy is always positive Improves the GS energy

RS CVM for Gaussian EA 2D F -1.4 -1.5 -1.6 -1.7 T 0.5 1 1.5 2

RS CVM for EA 2D Local stability of the solution w.r.t. u, u 1 , u 2 � = 0 � � q ( u ) u 2 du a = a ij ( U ) = Q ( U, u 1 , u 2 ) u i u j du 1 du 2 j = 1 , 2

RS CVM for EA 2D Local stability of the solution w.r.t. u, u 1 , u 2 � = 0 � � q ( u ) u 2 du a = a ij ( U ) = Q ( U, u 1 , u 2 ) u i u j du 1 du 2 j = 1 , 2 small

RS CVM for EA 2D Local stability of the solution w.r.t. u, u 1 , u 2 � = 0 � � q ( u ) u 2 du a = a ij ( U ) = Q ( U, u 1 , u 2 ) u i u j du 1 du 2 j = 1 , 2 T small 0.1 0.2 0.3 0.4 0.5 -0.05 -0.1 1 -0.15 ln Det -0.2 -0.25

Summary of analytical results for the ±J EA model 2D T plaq T Bethe T c = 0 = 1 . 5186 ... = 0 c c 3D first order T Bethe Λ small c transition or 0.15 T c 0.125 need to 0.1 0.075 consider a 0.05 0.025 larger region 2 T 1.2 1.4 1.6 1.8 (the cube)

Summary of analytical results for the EA model 4D Λ small 0.03 0.02 0.01 2.4 T 1.8 1.9 2.1 2.2 2.3 T plaq T Bethe T c = 2 . 03 = 2 . 2 = 2 . 515 ... c c

Functions, not distributions! � Q ( U, u 1 , u 2 ) u 2 a 11 ( U ) = 1 du 1 du 2 a 11 0.05 U -1 -0.5 0.5 1 -0.05 -0.1 -0.15

Functions, not distributions! � Q ( U, u 1 , u 2 ) u 2 a 11 ( U ) = 1 du 1 du 2 a 11 0.05 U -1 -0.5 0.5 1 -0.05 q 11 -0.1 0.175 0.15 -0.15 0.125 0.1 0.075 0.05 0.025 U -3 -2 -1 1 2 3

MPA for solving a given sample of 2D EA model Set u=0 and solve iteratively for U’ s according to eq. =

Converges for any T Gaussian 2D EA model τ typ ǫ = 10 − 10 ǫ = 10 − 1 β BP converges only for !! β < 0 . 84

Comparison with MC Energy vs. β MC MPA

Two spins marginals β = 0 . 1 � σ i σ j � MC � σ i σ j � MPA

Two spins marginals β = 1 . 1 � σ i σ j � MC � σ i σ j � MPA

Two spins marginals β = 2 . 1 � σ i σ j � MC � σ i σ j � MPA

Stronger test: find GS MPA + decimation or reinforcement

Stronger test: find GS MPA + decimation or reinforcement never finds GS !!

Stronger test: find GS MPA + decimation or reinforcement never finds GS !! exact GS energy

Stronger test: find GS MPA + decimation or reinforcement never finds GS !! mean relative error: 0.0013 for Gauss 0.00078 for ±J exact GS energy

It works on a 3D lattice! entropy 3D Gauss EA L=50 β c energy β

Conclusions By the Replica CVM we derived GSP eqs. The solution is a computational challenge! Very good approximation scheme: average case, no transition in 2D EA model single sample, MPA for the paramagnetic phase Future work find the AT line (paramagnetic phase in field) going in the SG phase with GSP: 1RSB factorized solution few first moments of Q(U,u1,u2)