A Multilevel Vertex Separator Algorithm Based on the Solution of - PowerPoint PPT Presentation

A Multilevel Vertex Separator Algorithm Based on the Solution of Bilinear Programs William Hager 1 James Hungerford 2 Ilya Safro 3 1 University of Florida 2 M.A.I.O.R. Srl, Lucca, Italy 3 Clemson University October 12th, 2016 CSC16 Albuquerque, New Mexico 1 / 55

Outline Introduction 1 Bilinear programming formulation 2 Solution approach 3 Multilevel algorithm 4 Computational Results 5 2 / 55

Outline Introduction 1 Bilinear programming formulation 2 Solution approach 3 Multilevel algorithm 4 Computational Results 5 3 / 55

Notation G = ( V , E ) simple undirected graph V = { 1 , 2 , . . . , n } E ⊆ ( V × V ); ( i, j ) ∈ E ⇔ ( j, i ) ∈ E ; ( i, i ) / ∈ E ∀ i ∈ V Vertex costs: c i ∈ R ∀ i ∈ V Vertex weights: w i > 0 ∀ i ∈ V For any Z ⊂ V , � � C ( Z ) = c i and W ( Z ) = w i i ∈Z i ∈Z 4 / 55

Definition Let A , B , S ⊆ V be a partition of V such that A , B � = ∅ . Then ( A , S , B ) is a vertex separator if ( A × B ) ∩ E = ∅ . S A B 5 / 55

Definition Let A , B , S ⊆ V be a partition of V such that A , B � = ∅ . Then ( A , S , B ) is a vertex separator if ( A × B ) ∩ E = ∅ . Definition (Vertex Separator Problem (VSP)) Given ℓ a , ℓ b , u a , u b ∈ R + , find a vertex separator ( A , S , B ) such that ℓ a ≤ W ( A ) ≤ u a , ℓ b ≤ W ( B ) ≤ u b , and C ( S ) is minimized: min C ( S ) A , S , B⊆V subject to S = V \ ( A ∪ B ) , A ∩ B = ∅ , ( A × B ) ∩ E = ∅ , ℓ a ≤ W ( A ) ≤ u a , and ℓ b ≤ W ( B ) ≤ u b . NP Hard [1] 1 T. Bui and C. Jones , Finding good approximate vertex and edge partitions is NP-hard , IPL 42 (1992), pp. 153–159. 6 / 55

Applications Sparse matrix factorizations Cyber-security, social network analysis 7 / 55

Applications 8 / 55

Applications A B S A B S 9 / 55

Applications 10 / 55

Applications 11 / 55

Applications Sparse matrix factorizations speed vs. solution quality Cyber-security, social network analysis speed vs. solution quality 12 / 55

Software/related work Edge-separator based methods : A. Pothen, H. D. Simon, and K. Liou , Partitioning sparse matrices with eigenvectors of graphs , SIAM J. Matrix Anal. Appl., 11 (1990), pp. 430–452. J. R. Gilbert and E. Zmijewski , A parallel graph partitioning algorithm for a message-passing multiprocessor , Intl. J. Parallel Programming, 16 (1987), pp. 498–513. S. Acer, E. Kayaaslan, and C. Aykanat , A recursive bipartitioning algorithm for permuting sparse square matrices into block diagonal form with overlap , SIAM Journal on Scientific Computing, 35 (2013), pp. C99–C121. 13 / 55

Software/related work Direct methods : C. C. Ashcraft and J. W. H. Liu , A partition improvement algorithm for generalized nested dissection , Tech. Rep. BCSTECH-94-020, Boeing Computer Services, Seattle, WA, 1994. E. Balas and C. C. de Souza , The vertex separator problem: a polyhedral investigation , Math. Program., 103 (2005), pp. 583–608. U. Benlic and J. Hao , Breakout local search for the vertex separator problem , in Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, 2013. U. Feige, M. Hajiaghayi, and J. Lee , Improved approximation algorithms for vertex separators , SIAM J. Comput., 38 (2008), pp. 629–657. C. Leiserson and J. Lewis , Orderings for parallel sparse symmetric factorization , in Third SIAM Conference on Parallel Processing for Scientific Computing, SIAM Publications, 1987, pp. 27–31. 14 / 55

Software/related work MeTiS BEND SCOTCH KaHIP Hypergraphs: hMeTiS, PaToH, Mondriaan, Parkway, Zoltan, ... 15 / 55

Outline Introduction 1 Bilinear programming formulation 2 Solution approach 3 Multilevel algorithm 4 Computational Results 5 16 / 55

Combinatorial formulation min C ( S ) A , S , B⊆V subject to S = V \ ( A ∪ B ) , A ∩ B = ∅ , ( A × B ) ∩ E = ∅ , ℓ a ≤ W ( A ) ≤ u a , and ℓ b ≤ W ( B ) ≤ u b 17 / 55

max C ( A ∪ B ) A , B⊆V subject to A ∩ B = ∅ , ( A × B ) ∩ E = ∅ , ℓ a ≤ W ( A ) ≤ u a , and ℓ b ≤ W ( B ) ≤ u b 18 / 55

max C ( A ∪ B ) A , B⊆V subject to A ∩ B = ∅ , ( A × B ) ∩ E = ∅ , ℓ a ≤ W ( A ) ≤ u a , and ℓ b ≤ W ( B ) ≤ u b � 1 , � 1 , if i ∈ A if i ∈ B x i = and y i = ∀ i ∈ V 0 , if i / ∈ A 0 , if i / ∈ B ℓ a ≤ w T x ≤ u a , ℓ b ≤ w T y ≤ u b c T ( x + y ) ← C ( A ∪ B ) 18 / 55

A ij ∈ { 0 , 1 } , A ij = 1 ⇔ ( i, j ) ∈ E n n x T Ay � � � � = x i a ij y j = a ij x i =1 y j =1 i =1 j =1 � � = a ij = | ( A × B ) ∩ E| . i ∈A j ∈B n x T y � � = x i y i = 1 i =1 x i = y i =1 = |A ∩ B|  A ∩ B = ∅  x T ( A + I ) y = 0 ⇔ and ( A × B ) ∩ E = ∅  19 / 55

Integer formulation c T ( x + y ) max x T ( A + I ) y = 0 , subject to ℓ a ≤ w T x ≤ u a , ℓ b ≤ w T y ≤ u b , x , y ∈ { 0 , 1 } n 20 / 55

Integer formulation γ ∈ R (penalty parameter) f ( x , y ) := c T ( x + y ) − γ x T ( A + I ) y max (1) ℓ a ≤ w T x ≤ u a , ℓ b ≤ w T y ≤ u b , subject to x , y ∈ { 0 , 1 } n 21 / 55

Integer formulation Proposition If w ≥ 1 and γ > 0 with γ ≥ max { c i : i ∈ V} , then for any feasible point ( x , y ) in (1) satisfying f ( x , y ) ≥ γ ( ℓ a + ℓ b ) , there is a feasible point (¯ x , ¯ y ) in (1) such that x T ( A + I )¯ f (¯ x , ¯ y ) ≥ f ( x , y ) and ¯ y = 0 . Hence, if the optimal objective value in (1) is at least γ ( ℓ a + ℓ b ) , then there exists an optimal solution ( x ∗ , y ∗ ) to (1) such that an optimal vertex separator is given by A = { i : x ∗ i = 1 } , B = { i : y ∗ i = 1 } , and S = { i : x ∗ i = y ∗ i = 0 } . 22 / 55

Continuous formulation � � = c T ( x + y ) − γ x T ( A + I ) y max f ( x , y ) (2) ℓ a ≤ w T x ≤ u a , ℓ b ≤ w T y ≤ u b , subject to x , y ∈ R n 0 ≤ x ≤ 1 , 0 ≤ y ≤ 1 , Theorem (Theorem 2.1, Part 1 [1]) Suppose w = 1 , c ≥ 0 , γ ≥ max { c i : i ∈ V} > 0 , and C ( S ∗ ) ≤ C ( V ) − γ ( ℓ a + ℓ b ) . Then (2) has a binary optimal solution ( x ∗ , y ∗ ) ∈ { 0 , 1 } 2 n satisfying x ∗ T ( A + I ) y ∗ = 0 . Hence, an optimal vertex separator is given by A = { i : x ∗ i = 1 } , B = { i : y ∗ i = 1 } , and S = { i : x ∗ i = y ∗ i = 0 } . 1W. W. Hager and J.T. Hungerford, ”Continuous quadratic programming formulations of optimization problems on graphs”, European Journal of Operations Research, 2014. 23 / 55

Continuous formulation � � = c T ( x + y ) − γ x T ( A + I ) y max f ( x , y ) (2) ℓ a ≤ w T x ≤ u a , ℓ b ≤ w T y ≤ u b , subject to x , y ∈ R n 0 ≤ x ≤ 1 , 0 ≤ y ≤ 1 , Definition A point ( x , y ) ∈ R 2 n is called mostly binary if x and y each have at most one non-binary component. Proposition For any w > 0 , c ∈ R , and γ ∈ R , (2) has a mostly binary optimal solution. 24 / 55

Local optimality Definition Let ( A , S , B ) be a feasible vertex separator. 1. For every i ∈ V , let  0 , if i ∈ A  � gain( i, A ) = c i − if i ∈ S ∪ B , c j ,  j ∈N ( i ) ∩B 0 , if i ∈ B   � gain( i, B ) = if i ∈ S ∪ A . c i − c j .  j ∈N ( i ) ∩A 2. ( A , S , B ) is strongly FM-optimal if gain( i, A ) ≤ 0 and gain( i, B ) ≤ 0 for every i ∈ V . 25 / 55

Local optimality Proposition Suppose that c ≥ 0 and γ ≥ max { c i : i ∈ V} , and let ( A , S , B ) be a feasible vertex separator. If ( A , S , B ) is strongly FM-optimal, then the pair of incidence vectors ( x , y ) for A and B is locally optimal in (2) . 26 / 55

Outline Introduction 1 Bilinear programming formulation 2 Solution approach 3 Multilevel algorithm 4 Computational Results 5 27 / 55

Mountain Climbing Algorithm Input: A feasible point ( x , y ) and η > 0 . while ( ( x , y ) not a first-order maximizer ) x ← argmax { f ( x , y ) : x ∈ P a } ˆ y ← argmax { f ( x , y ) : y ∈ P b } ˆ y ) > max { f (ˆ y ) } + η ) if ( f (ˆ x , ˆ x , y ) , f ( x , ˆ ( x , y ) ← (ˆ x , ˆ y ) else if ( f (ˆ x , y ) > f ( x , ˆ y ) ) x ← ˆ x else y ← ˆ y end if end while return ( x , y ) Algorithm : MCA: A modified version of Konno’s Mountain Climbing Algorithm. 28 / 55

Mountain Climbing Algorithm Proposition (Konno [1]) Let ( x , y ) be a feasible initial guess. Then MCA terminates in a finite number of iterations, returning a point which satisfies the first-order optimality conditions. 1 H. Konno , A cutting plane algorithm for solving bilinear programs , Mathematical Programming, 11 (1976), pp. 14–27. 29 / 55

Escaping first-order optima 30 / 55

Escaping first-order optima A feasible point ( x , y ) is a first-order maximizer if and only if there exist multipliers µ a ∈ M ( x ) , µ b ∈ M ( y ) , λ a ∈ L ( x , ℓ a , u a ) , λ b ∈ L ( y , ℓ b , u b ) such that � ∇ x f ( x , y ) � λ a w � µ a � � � + + = 0 , µ b λ b w ∇ y f ( x , y ) where { µ ∈ R n : µ i z i ≤ min { µ i , 0 } for all 1 ≤ i ≤ n } M ( z ) = and { λ ∈ R : λ w T z ≤ min { λu, λℓ }} . L ( z , ℓ, u ) = 31 / 55