Faster Stochastic Alternating Direction Method of Multipliers for - PowerPoint PPT Presentation

Feb 12, 2024 •39 likes •179 views

Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization Feihu Huang 1 , Songcan Chen 2,3 and Heng Huang 1,4* 1. Department of Electrical and Computer Engineering, University of Pittsburgh, USA 2. College of

Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization Feihu Huang 1 , Songcan Chen 2,3 and Heng Huang 1,4* 1. Department of Electrical and Computer Engineering, University of Pittsburgh, USA 2. College of Computer Science & Technology, Nanjing University of Aeronautics & Astronautics 3. MIIT Key Laboratory of Pattern Analysis and Machine Intelligence 4. JD Finance America Corporation * heng.huang@pitt.edu ICML-2019 Long Beach, USA
Outline • Background • Faster Stochastic ADMM Methods • Convergence Analysis: Lower IFO Complexity • Experiments
Current Data Current data not only has large sample size, but also contains some complex structures.
Problem Statement • A finite/infinite-sum constraint problem encoding some complex encoding empirical or structures expected loss for big data
Contributions In the paper, our main contributions are summarized as follows:
IFO Complexity
Faster Stochastic ADMM (SPIDER-ADMM)
Convergence Analysis
Online SPIDER-ADMM
Experiments 1. Graph-Guided Binary Classification datasets # samples # features # classes a9a 32,561 123 2 w8a 64,700 300 2 ijcnn1 126,702 22 2 covtype.binary 581,012 54 2
Experiments
Experiments 2. Multi-Task Learning
Experiments datasets # samples # features # classes letter 15,000 16 26 sensorless 58,509 48 11 mnist 60,000 780 10 covtype 581,012 54 7
Thanks! Poster: Pacific ballroom #93

Recommend

Scalable Tensor Computations with Cyclops and Faster Algorithms for Alternating Least Squares

Scalable Tensor Computations with Cyclops and Faster Algorithms for Alternating Least Squares Edgar Solomonik Department of Computer Science University of Illinois at Urbana-Champaign Invited Workshop on Compiler Techniques for Sparse Tensor

222 views • 9 slides

Alternating Permutations Richard P. Stanley M.I.T. Alternating Permutations p. 1

Alternating Permutations Richard P. Stanley M.I.T. Alternating Permutations p. 1 Definitions A sequence a 1 , a 2 , . . . , a k of distinct integers is alternating if a 1 > a 2 < a 3 > a 4 < , and reverse alternating if

1.14k views • 98 slides

Alternating Permutations Richard P. Stanley M.I.T. Alternating Permutations p. Basic

Alternating Permutations Richard P. Stanley M.I.T. Alternating Permutations p. Basic definitions A sequence a 1 , a 2 , . . . , a k of distinct integers is alternating if a 1 > a 2 < a 3 > a 4 < , and reverse alternating

1.4k views • 119 slides

Motivation Alternating-time temporal logic plays a key role in Stochastic Multi-Agent Systems

Probabilistic Alternating-Time -Calculus Fu Song 1 , Yedi Zhang 1 , Taolue Chen 2 , Yu Tang 3 , Zhiwu Xu 4 1 ShanghaiTech University, China 2 Birkbeck, University of London, UK 3 East China Normal University, Shanghai, China 4 Shenzhen

551 views • 53 slides

Unit 10: Alternating-current circuits Introduction. Alternating current features. Phasor

Unit 10: Alternating-current circuits Introduction. Alternating current features. Phasor diagram. Behaviour of basic dipoles (resistor, inductor, capacitor) to an alternating current. RLC series circuit. Impedance and phase lag.

491 views • 18 slides

Alternating Permutations Richard P. Stanley M.I.T. Alternating Permutations p. 1 Basic

Alternating Permutations Richard P. Stanley M.I.T. Alternating Permutations p. 1 Basic definitions A sequence a 1 , a 2 , . . . , a k of distinct integers is alternating if a 1 > a 2 < a 3 > a 4 < , and reverse

1.59k views • 119 slides

Alternating Current Slide 2 / 69 Topics to be covered Sources of alternating EMF Transformers

Slide 1 / 69 Alternating Current Slide 2 / 69 Topics to be covered Sources of alternating EMF Transformers AC Circuits and Impedance LRC Series AC Circuits Resonance in AC Circuit Oscillations Slide 3 / 69 Sources of Alternating EMF

273 views • 23 slides

A.C. generates an alternating field Alternating field generates eddy currents in

An alternating current is passed through a coil A.C. generates an alternating field Alternating field generates eddy currents in conductors An alternating current is passed through a coil A.C. generates an alternating field

328 views • 19 slides

What If We Only Have Stochastic . . . What if the Stochastic . . . Approximate Stochastic

In Finance, We Need . . . How to Make . . . From a Theoretical . . . How to Make . . . What If We Only Have Stochastic . . . What if the Stochastic . . . Approximate Stochastic Additional Reasonable . . . Dominance? The Assumption 0 < .

550 views • 15 slides

Alternating-time temporal logic Mehdi Dastani BBL-521 M.M.Dastani@uu.nl ATL: Alternating-time

Alternating-time temporal logic Mehdi Dastani BBL-521 M.M.Dastani@uu.nl ATL: Alternating-time temporal logic . ATL is a multiagent logic ATL is a branching time that extends CTL with multiple players. ATL combines logic with game theory

337 views • 18 slides

Alternating Minimizations Converge to Second-order Optimal Solutions Qiuwei Li 1 Joint work with

Alternating Minimizations Converge to Second-order Optimal Solutions Qiuwei Li 1 Joint work with Zhihui Zhu 2 and Gongguo Tang 1 1 Colorado School of Mines 2 Johns Hopkins University Why is Alternating Minimization so popular? v 1 ( j ) u 2 ( i )

465 views • 10 slides

Two-Way Alternating Automata and Finite Models Tedious proofs of irrelevant results Mikolaj

Two-Way Alternating Automata and Finite Models Tedious proofs of irrelevant results Mikolaj Bojanczyk Warsaw University Two-Way Alternating Automata and Finite Models p.1/18 Intuition on the automaton A two-way alternating automaton

799 views • 57 slides

Stochastic Processes Will Perkins March 7, 2013 Stochastic Processes Q: What is a Stochastic

Stochastic Processes Will Perkins March 7, 2013 Stochastic Processes Q: What is a Stochastic Process? A: A collection of random variables defined on the same probability space and indexed by a time parameter. { Z t } t T where each Z

1.52k views • 17 slides

Minimax strategy for prediction with expert advice under stochastic assumptions Wojciech Kot

Minimax strategy for prediction with expert advice under stochastic assumptions Wojciech Kot lowski Pozna n University of Technology, Poland Learning Faster from Easy Data II NIPS 2015 Workshop 1 / 3 Prediction with expert advice In

472 views • 9 slides

Alternating Sign Matrices and Descending Plane Partitions R. Behrend, P. Di Francesco and P.

Alternating Sign Matrices Descending Plane Partitions The ASM-DPP conjecture Proof: determinant formulae Generalizations Alternating Sign Matrices and Descending Plane Partitions R. Behrend, P. Di Francesco and P. Zinn-Justin Laboratoire de

1.12k views • 98 slides

Alternating Direction Method of Multipliers Prof S. Boyd HYCON 2, Trento, 23/6/11 source:

Alternating Direction Method of Multipliers Prof S. Boyd HYCON 2, Trento, 23/6/11 source: Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers (Boyd, Parikh, Chu, Peleato, Eckstein) 1 Goals

878 views • 70 slides

Some References P. Carpentier Master MMMEF Cours MNOS 2014-2015 263 / 263 Stochastic

Stochastic Approximation and Stochastic Gradient Optimization Theory and Large Scale Systems Dual Effect and Discretization Stochastic Programming and Stochastic Optimal Control Some References P. Carpentier Master MMMEF Cours MNOS

385 views • 9 slides

Alternating Current Electricity 15-1 Frequency and Period Frequency (linear and angular) Period

Alternating Current Electricity 15-1 Frequency and Period Frequency (linear and angular) Period = T Professional Publications, Inc. FERC Alternating Current Electricity 15-2a Average Value Square wave, positive pulse = negative pulse: X ave

756 views • 41 slides

Lecture 2.3: Symmetric and alternating groups Matthew Macauley Department of Mathematical

Lecture 2.3: Symmetric and alternating groups Matthew Macauley Department of Mathematical Sciences Clemson University http://www.math.clemson.edu/~macaule/ Math 4120, Modern Algebra M. Macauley (Clemson) Lecture 2.3: Symmetric and alternating

447 views • 15 slides

Faster Haskell Neil Mitchell www.cs.york.ac.uk/~ndm The Goal Make Haskell faster

Faster Haskell Neil Mitchell www.cs.york.ac.uk/~ndm The Goal Make Haskell faster Reduce the runtime But keep high-level declarative style Full automatic - no special functions Different from foldr/build,

478 views • 24 slides

On the O (1 / k ) Convergence of Asynchronous Distributed Alternating Direction Method of

On the O (1 / k ) Convergence of Asynchronous Distributed Alternating Direction Method of Multipliers (ADMM) Ermin Wei Asu Ozdaglar Laboratory for Information and Decision Systems Department of Electrical Engineering and Computer Science

606 views • 36 slides

FASTER TRANSFORMER Bo Yang Hsueh, 2019/12/18 AGENDA What is Faster Transformer Introduce the

FASTER TRANSFORMER Bo Yang Hsueh, 2019/12/18 AGENDA What is Faster Transformer Introduce the Transformer and Faster Transformer 1.0 New Features in Faster Transformer 2.0 Introduce the Faster Transformer 2.0 Faster Transformer 2.0 performance

932 views • 55 slides

Symplectic alternating algebras Gunnar Traustason (with Layla Sorkatti) Department of

Symplectic alternating algebras Gunnar Traustason (with Layla Sorkatti) Department of Mathematical Sciences University of Bath Groups St Andrews 2013 Gunnar Traustason (with Layla Sorkatti) Symplectic alternating algebras Symplectic

709 views • 27 slides

Alternating Current AC Circuits and Impedance LRC Series AC Circuits Resonance in AC Circuit

Slide 1 / 69 Slide 2 / 69 Topics to be covered Sources of alternating EMF Transformers Alternating Current AC Circuits and Impedance LRC Series AC Circuits Resonance in AC Circuit Oscillations Slide 3 / 69 Slide 4 / 69 Sources of

286 views • 12 slides