Greedy Algorithms, Frank-Wolfe and Friends a modern perspective NIPS 2013 Workshop sites.google.com/site/nips13greedyfrankwolfe Zaid Harchaoui, Martin Jaggi, Federico Pierucci, INRIA Simons Institute, LJK lab, Grenoble (France) Berkeley (USA) Grenoble (France)
Research Areas • Continuous Optimization • Combinatorial Optimization • Signal Processing • Machine Learning • Statistics • Harmonic Analysis
Specific Applications • Recommender Systems • Image Processing • Image Categorization • Sparse Recovery • Sparse Regression • Matrix Factorizations and Low Rank Methods • more to come...
Optimization over Atomic Sets x ∈ D f ( x ) min convex (or affine) hull of simple things A D := conv A [ Chandrasekaran et al. 2012 ]
Examples of Atomic Domains Suitable for Frank-Wolfe Optimization Domain Atoms A D = conv( A ) R n Sparse vectors k . k 1 -ball R n Sign-vectors k . k 1 -ball R n ` p -Sphere k . k p -ball R n Sparse non-neg. vectors Simplex ∆ n R n Latent group sparse vect. k . k G -ball R m ⇥ n Matrix trace norm k . k tr -ball R m ⇥ n Matrix operator norm k . k op -ball R m ⇥ n Schatten matrix norms k ( � i ( . )) k p -ball R m ⇥ n Matrix max-norm k . k max -ball R n ⇥ n Permutation matrices Birkho ff polytope R n ⇥ n Rotation matrices S n ⇥ n Rank-1 PSD matrices { x ⌫ 0 , Tr( x ) = 1 } of unit trace PSD matrices S n ⇥ n { x ⌫ 0 , x ii 1 } of bounded diagonal and many more...
Greedy algorithms x ( k +1) := α x ( k ) + β s s ∈ A • Frank-Wolfe / Conditional Gradient • composite Frank-Wolfe / Conditional Gradient • (orthogonal) Matching Pursuit • Forward/Backward Sparse Selection (away steps and variants) • Minimum Norm Point Algorithm (for submodular minimization)
Time Speaker Title 7:30 - 7:40am Organizers Introduction 7:40 - 8:20am Robert M. Freund Remarks on Frank-Wolfe and Structural Friends 8:20 - 9:00am Ben Recht The Algorithmic Frontiers of Atomic Norm Minimization: Relaxation, Discretization, and Greedy Pursuit 9:00 - 9:30pm Coffee Break 9:30 - 9:45am Nikhil Rao, Parikshit Shah and Conditional Gradient with Enhancement and Truncation for Atomic Stephen Wright Norm Regularization (canceled) Hector Allende, Pairwise Away Steps for the Frank-Wolfe Algorithm Emanuele Frandi, Ricardo Nanculef, Claudio Sartori 9:55 - 10:05am Simon Lacoste-Julien and An Affine Invariant Linear Convergence Analysis for Frank-Wolfe Martin Jaggi Algorithms 10:05 - 10:15am Vamsi Potluru, Jonathan Le Pairwise Coordinate Descent for Sparse NMF Roux, Barak Pearlmutter, John Hershey and Matthew Brand 10:15 - 10:30am Robert M. Freund and Paul New Analysis and Results for the Conditional Gradient Method Grigas 10:30 - 3:30pm Lunch Break 3:30 - 3:45pm Marguerite Frank Honorary Guest 3:45 - 4:25pm Shai Shalev-Schwartz Efficiently Training Sum-Product Neural Networks using Forward Greedy Selection 4:25 - 4:40pm Xiaocheng Tang and Katya Complexity of Inexact Proximal Newton methods Scheinberg 4:40 - 4:50pm Jacob Steinhardt and Jonathan A Greedy Framework for First-Order Optimization Huggins 4:50 - 5:00pm Ahmed Farahat, Ali Ghodsi and A Fast Greedy Algorithm for Generalized Column Subset Selection Mohamed Kamel 5:00 - 5:30pm ! Coffee Break 5:30 - 6:10pm Francis Bach Duality between Subgradient and Conditional Gradient Methods 6:10 - 6:25pm David Belanger, Dan Sheldon Marginal Inference in MRFs using Frank-Wolfe and Andrew McCallum
Marguerite Frank today 3:30pm ALGORITHM FOR QUADRATIC PROGRAMMING 6 5 9 1 l e f l o W P h i l i p A finite iteration method for calculating the solution of quadratic - n d y o i t n n a v e r s l k a n r a t o n Un i e r n F e e g t i e N r e r A u o ce g m Pr in r a M o t s n o i s n The problem of maximizing a concave quadratic function whose variables are subject e t x E . d e b from both the i r c s e d is Bibliography). Our aim here has been to develop a suggested. s r m e l solving this non-linear programming problem which should be particularly well b , s o e r i p d e u g r t a n s linear inequality constraints has been the subject of several recent i s m m m e a l b r o g r o D r [ Frank & p r a e such, called PI, is set forth in Section aid of generalized Lagrange multipliers the'solutions n i Wolfe 1956 ] l the solutions of a new quadratic programming INTRODUCTION e e from the fact that the boundedness s ( maximum sought in PI1 side and the theoretical high-speed machine computation. . 1 s a quadratic programming problem computation (linear) constraints of
Recommend
More recommend
