A A Modi dified d Frank nk-Wo Wolfe Algorithm for Te Tensor Fa Factorization with Unimodal Signals Junting Chen The Chinese University of Hong Kong, Shenzhen Guangdong, China Urbashi Mitra University of Southern California CA, USA Acknowledgement This research has been funded in part by one or more of the following grants: ONR N00014-15-1-2550, NSF CNS-1213128, NSF CCF-1718560, NSF CCF-1410009 , NSF CPS- 1446901, and AFOSR FA9550-12-1-0215.
Many signals to be es>mated have unimodal Ma pr proper> per>es es False estimate acous^c sensors (known as in the ocean outlier) when not exploiting unimodality Known spectrum curves Signal propaga^on (unimodal) (spa^ally unimodal) (Bro&Sidiropoulos’98) Source localiza^on exploi^ng only Estimate spectra of different chemicals unimodality in an unknown environment from compound samples (e.g., underwater) CUHK-Shenzhen Unimodal Frank-Wolfe 2
Fo Formula>on: Add a uni unimoda dality co constraint to im improve the es>m >ma>o >on cost func^on for the es^ma^on problem P : minimize f ( x ) x ∈ R n subject to x ∈ U ∩ M an n-dimensional vector some other constraints. the set of all unimodal Here, we assume: vectors M = { x : a k x k 1 b } a 3D unimodal cone U x s x s +1 intersected a sphere … … x s − 1 non-convex! s s - 1 s + 1 0 ≤ x 1 ≤ x 2 · · · ≤ x s Goal: Design low complexity algorithms x x +1 ≥ x s +2 ≥ · · · ≥ x n ≥ 0 CUHK-Shenzhen Unimodal Frank-Wolfe 3
Projec^on will be expensive when we need it for very update! Prior work mainly focused on projec^ons: Ø For simple objec^ves (e.g., least-squares, L 1 , L -infinity norm): § Fast isotonic projec^on: Németh&Németh’10 Prefix isotonic regression: Stout’10 § Complexity: roughly O ( n ) – O ( n 2 ) Ø For general objec^ve: use projec^on § Alterna^ng least-squares with unimodal prefix isotonic projection projec^on: Bro&Sidiropoulos’98 § projected gradient: Chen&Mitra’17 x s x ( t +1) = P U h x ( t ) + λ t r f ( x ( t ) ) i x s +1 … … x s − 1 s - 1 s s + 1 CUHK-Shenzhen Unimodal Frank-Wolfe 4
Can Can we design low complexity y projection-fr free me meth thod ods? The Frank-Wolfe update procedure (no projection required) If the constraint set is convex, then the Frank- Wolfe update can guarantee to stay inside the constraint set. Not the case here! (Garber&Hazan) x ( t +1) = x ( t ) + λ t (ˆ y − x ( t ) ) y ∈ M r f ( x ( t ) ) T y y = arg min ˆ CUHK-Shenzhen Unimodal Frank-Wolfe 5
Pr Proposed design: successive ve linear approximation co could be a way to handle the non-co convex co constraint x ( t +1) = x ( t ) + λ t (ˆ y − x ( t ) ) Frank-Wolfe update f ( x ( t ) ) + r f ( x ( t ) ) T y minimize y ∈ R n dynamically construct y 2 U ( x ( t ) ) subject to a convex constraint set f ( x ) U ∩ M Original constraint Convex local set (non-convex) constraint set CUHK-Shenzhen Unimodal Frank-Wolfe 6
Ne New chal allenges: need to dyn ynam amical ally y design the co convex co constraint set U (x (x ( t ) ) x ( t +1) = x ( t ) + λ t (ˆ y − x ( t ) ) Challenge 1 : The sub- f ( x ( t ) ) + r f ( x ( t ) ) T y problems need to be minimize y ∈ R n solved efficiently y 2 U ( x ( t ) ) subject to O ( n ) complexity or better f ( x ) U ∩ M Original constraint Convex local set (non-convex) constraint set Challenge 2 : Needs to justify the convergence CUHK-Shenzhen Unimodal Frank-Wolfe 7
Recommend
More recommend
