Empirical comparison of the mini- bucket and MPLP algorithms for - PowerPoint PPT Presentation

Empirical comparison of the mini- bucket and MPLP algorithms for MPE problem

Solving MPE  A finite COP is a triple R X , D , F where : Global Cost Funct ion  X { X ,..., X } - variables 1 n       D { D ,..., D } - domains m 1 n F ( X ) f X  F { f ,..., f } - probabilit ies i i 1 1 m    S {S ,..., S } scopes F * max F ( x ) 1 m x • Exact by Bucket elimination • Bounding scheme : MBE, MPLP, Soft arc-consistency. • MBE : capitalize on large cluster processed exactly. Similar to i- consistency for large i (parameter z), directional, non-iterative. • SAC, MPLP : equivalent to arc-consistency. Can be extended to cluster graphs, but may not be efficient. Based on cost shifting subject t equivalence-preserving transformations (EPT)

Mini-Bucket Approximation [Dechter, Rish 2003] Split a bucket into mini-buckets => bound complexity bucket (X) = { h 1 , …, h r , h r+1 , …, h n } X X’ X’’   n  X h max h i i 1 X { h 1 , …, h r } { h r+1 , …, h n } Bucket(X’)= Bucket(X’’)=   r n   X g (max h ) (max h )  i   i i 1 i r 1 X X  X X g h    n r n r Exponentia l complexity decrease : O(e ) O ( e ) O ( e ) 4

Mini-Bucket Approximation • Properties: – Guaranteed to output upper bound on the exact solution (for maximization) – Control parameter iBound (number of variables in the mini-bucket) allows to flexibly trade space for accuracy • If iBound>=treewidth the algorithm turns into the exact bucket elimination scheme – The solution can be recovered by the upward pass – If the values assigned to ‘ duplicate ’ variables agree, the solution found is exact

Mini-Bucket Approximation • Possible ways to improve accuracy: – Increase i-Bound – Try different heuristics to split the functions into mini- buckets: • Scope-based: rely only on functions arguments [Dechter, Rish 1997] • Content-based (the product of mini-bucket functions should be as close as possible to the original bucket function, according to some distance metric (l1,l2…) * Rollon Dechter 2010] – Force the values of duplicates to be the same -> force max- marginals of the mini- buckets to be the same (‘moment matching’)

Max Product Linear Programming algorithm (MPLP) [Globerson, Jakkola] • Variation of max-product • Always guaranteed to converge (but not necessarily at the global optimum) • monotonically improves with more iterations x • If the costs are not tied, the solution found is exact x • If all variables are binary, MPLP converges to x exact solution x F( x

Empirical evaluation • Genetic linkage instances from UAI’2008 competition – ~300-1000 variables – 15-20 treewidth – Domain size 4-5 • Mini-buckets: – iBound=5,10 – Heuristics: scope-based, content-based(l1,l2,linf, KL,HPM,MAS) – Without and with moment-matching before eliminating bucket variable • MPLP: – On original factors, cluster trees constructed by MBE with iBounds=1, 5, 10 – Iterations: 5, 10, 15, 30, 60, 120, 240, 500, 1000, 1500

-102 pedigree1: log(MPE) as a function of number of iterations of MPLP, [ -104.956 ] -102.5 -103 -103.5 log(MPE) -104 -104.5 -105 5 10 15 30 60 120 240 500 1000 1500 -105.5 number of iterations MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10 exact solution

pedigree1: time, sec, as a function of number of iterations of MPLP 900 800 700 600 500 time, sec 400 300 200 100 0 5 10 15 30 60 120 240 500 1000 1500 number of iterations MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

Pedigree23, log(MPE) as a function of number of iterations of MPLP,[ -143.662 ] -135 -136 -137 -138 runtime, sec -139 -140 -141 -142 -143 -144 5 10 15 30 60 120 240 500 1000 1500 number of iterations MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10 exact solution

pedigree23: time, sec, as a function of number of iterations of MPLP 1000 900 800 700 600 time, sec 500 400 300 200 100 0 5 10 15 30 60 120 240 500 1000 1500 number of iterations MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

Pedigree37, log(MPE) as a function of number of iterations of MPLP,[ -333.603 ] -287 -292 -297 -302 runtime, sec -307 -312 -317 -322 -327 5 10 15 30 60 120 240 500 1000 1500 number of iterations MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

pedigree37: time, sec, as a function of number of iterations of MPLP 2300 1800 1300 time, sec 800 300 5 10 15 30 60 120 240 500 1000 1500 -200 number of iterations MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

Pedigree13, log(MPE) as a function of number of iterations of MPLP,[ -168.952 ] -156 -157 -158 -159 -160 runtime, sec -161 -162 -163 -164 -165 -166 5 10 15 30 60 120 240 number of iterations MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

pedigree13: time, sec, as a function of number of iterations of MPLP 250 200 150 time, sec 100 50 0 5 10 15 30 60 120 240 number of iterations MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

Average relative bound improvement due to moment matching 0.049 0.044 0.039 0.034 0.029 0.024 0.019 0.014 0.009 scope l1 l2 linf KL HPM MAS distance metrics iBound=5 iBound=10