Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Nonnegative Tensor Factorization using a proximal algorithm: application to 3D fluorescence spectroscopy Caroline Chaux Joint work with X. Vu, N. Thirion-Moreau and S. Maire (LSIS, Toulon) Aix-Marseille Univ. I2M IHP, Feb. 2 2017 1 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Outline Introduction 3D fluorescence spectroscopy Tensor definition Goal Proximal tools Criterion formulation Proximity operator Proximal algorithm Application to CPD Minimization problem Algorithm Numerical simulations Synthetic case Real case: water monitoring Conclusion and future work 2 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work 3D fluorescence spectroscopy Normalized excitation spectra Normalized emission spectra Normalized concentrations 0.035 0.04 250 0.035 0.03 200 0.03 0.025 0.025 150 0.02 0.02 0.015 0.015 100 0.01 0.01 50 0.005 0.005 0 0 200 220 240 260 280 300 320 340 360 380 400 300 320 340 360 380 400 420 440 460 480 500 0 0 10 20 30 40 50 60 70 80 90 100 λ ex λ em Experiment 3 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Coumpounds characterisation Normalized emission spectra 0.04 0.035 0.03 0.025 0.02 0.015 0.01 0.005 0 300 320 340 360 380 400 420 440 460 480 500 λ em FEEM 500 450 400 λ ex 8 400 380 350 7 360 220 240 260 280 300 320 340 360 380 400 Normalized excitation spectra 6 340 500 450 5 320 λ ex 400 λ ex 4 350 300 220 240 260 280 300 320 340 360 380 400 280 3 500 260 2 450 λ ex 240 400 1 350 220 0 220 240 260 280 300 320 340 360 380 400 λ em 200 0.035 0.03 0.025 0.02 0.015 0.01 0.005 0 4 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Tensor What is a tensor? An N th-order tensor is represented by an N -way array in a chosen basis. Example: ◮ N = 1: a vector. ◮ N = 2: a matrix. 5 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Third-order tensors ◮ A special case: nonnegative third-order tensors ( N = 3) T = ( t i 1 i 2 i 3 ) i 1 , i 2 , i 3 ∈ R + I 1 × I 2 × I 3 . ◮ The Canonical Polyadic (CP) decomposition: Tensor rank R � [¯ A ( 1 ) , ¯ A ( 2 ) , ¯ a ( 1 ) a ( 2 ) a ( 3 ) A ( 3 ) ] ¯ ◦ ¯ ◦ ¯ T = = [ ] r r r r = 1 Loading vectors Loading matrices A ( n ) ∈ R I n × R ∈ R + I n and ¯ a ( n ) ∀ n ∈ { 1 , 2 , 3 } , ¯ r ◦ : the outer product. ◮ Entry-wise form: � R a ( 1 ) a ( 2 ) a ( 3 ) ¯ i 1 r ¯ i 2 r ¯ t i 1 i 2 i 3 = i 3 r , ∀ ( i 1 , i 2 , i 3 ) r = 1 6 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Standard operations ◮ Outer product: let u ∈ R I , v ∈ R J , u ◦ v = uv ⊤ ∈ R I × J ◮ Khatri-Rao product: let U = [ u 1 , u 2 , . . . , u J ] ∈ R I × J and V = [ v 1 , v 2 , . . . , v J ] ∈ R K × J U ⊙ V = [ u 1 ⊗ v 1 , u 2 ⊗ v 2 , . . . , u J ⊗ v J ] ∈ R IK × J . where u ⊗ v = [ u 1 v ; . . . ; u I v ] ∈ R IK (Kronecker product). ◮ Hadamard division: let U ∈ R I × J , V ∈ R I × J , U ⊘ V = ( u ij / v ij ) i , j ∈ R I × J 7 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Tensor flattening: example Objective: to handle matrices instead of tensors. Mode 1 Tensor Mode 2 Mode 3 8 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work 3D fluorescence spectroscopy and tensors � R a ( 1 ) a ( 2 ) a ( 3 ) T = ¯ ◦ ¯ ◦ ¯ r r r r = 1 Normalized excitation spectra Normalized emission spectra Normalized concentrations 0.035 0.04 250 0.035 0.03 200 0.03 0.025 0.025 150 0.02 0.02 0.015 100 0.015 0.01 0.01 50 0.005 0.005 0 0 200 220 240 260 280 300 320 340 360 380 400 300 320 340 360 380 400 420 440 460 480 500 0 λ ex λ em 0 10 20 30 40 50 60 70 80 90 100 Experiment 9 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Objective: tensor decomposition ◮ Input: ◮ Observed tensor T : observation of an original (unknown) tensor T possibly degraded (noise). ◮ Output: A ( n ) for all n ∈ { 1 , 2 , 3 } ◮ Estimated loading matrices � ◮ Difficulty: ◮ Rank R unknown ( i.e. � R � = R ): generally i) estimated or ii) overestimated. Proposed approach Formulate the problem under a variational approach. 10 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Minimization problem ◮ Standard problem: minimize F ( x ) + R ( x ) . ���� � �� � x ∈ R L Fidelity Regularization ◮ Taking into account several regularizations ( J terms): J � R ( x ) = R j ( x ) j = 1 ◮ For large size problem or for other reasons, can be interesting to work on data blocks x ( j ) of size L j ( x = ( x ( j ) ) 1 ≤ j ≤ J ) � J R j ( x ( j ) ) R ( x ) = j = 1 Technical assumptions: F , R and R j are proper lower semi-continuous functions. F is differentiable with a β -Lipschitz gradient. R j is assumed to be bounded from below by an affine function, and its restriction to its domain is continuous. 11 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Proximity operator ◮ let ϕ : R → ] −∞ , + ∞ ] be a proper lower semi-continuous function. The proximity operator is defined as 1 2 � u − v � 2 + ϕ ( u ) , prox ϕ : R → R : v �→ arg min u ∈ R ◮ let ϕ : R L → ] −∞ , + ∞ ] be a proper lower semi-continuous function. The proximity operator associated with a Symmetric Positive Definite (SPD) matrix P is defined as 1 prox P ,ϕ : R L → R L : v �→ arg min 2 � u − v � 2 P + ϕ ( u ) , u ∈ R L where ∀ x ∈ R L , � x � 2 P = � x , Px � and �· , ·� is the inner product. Remark : Note that if P reduces to the identity matrix, then the two definitions coincides. 12 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Criterion to be minimized J � R j ( x ( j ) ) F ( x ) + minimize x ∈ R L j = 1 Some solutions (non exhaustive list, CPD oriented): ◮ Proximal Alternating Linearized Minimization (PALM) [Bolte et al., 2014] ◮ A Block Coordinate Descent Method for both CPD and Tucker decomposition [Xu and Yin, 2013] ◮ An accelerated projection gradient based algorithm [Zhang et al., 2016] ◮ Block-Coordinate Variable Metric Forward-Backward (BC-VMFB) algorithm [Chouzenoux et al., 2016] Advantages of the BC-VMFB: flexible, stable, integrates preconditionning, relatively fast. 13 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Block coordinate proximal algorithm 1: Let x 0 ∈ dom R , k ∈ N and γ k ∈ ] 0 , + ∞ [ // Initialization step 2: for k = 0 , 1 , ... do // k-th iteration of the algorithm Let j k ∈ { 1 , ..., J } // Processing of block number j k (chosen, here, 3: according to a quasi cyclic rule) Let P j k ( x k ) be a SPD matrix // Construction of the preconditioner P j k ( x k ) 4: Let ∇ j k F ( x k ) be the Gradient // Calculation of Gradient 5: x ( j k ) = x ( j k ) − γ k P j k ( x k ) − 1 ∇ j k F ( x k ) ˜ 6: // Updating of block j k according to a k k Gradient step � � x ( j k ) x ( j k ) k + 1 ∈ prox γ − 1 ˜ 7: // Updating of block j k according to a P jk ( x k ) , R jk k k Proximal step ¯ ¯ j k j k k where ¯ k + 1 = x j = { 1 , ..., J } \ { j } 8: x // Other blocks are kept unchanged 9: end for 14 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Prox for CP decomposition CP decomposition: decompose a tensor into a (minimal) sum of rank-1 terms. Order 3: R � a ( 1 ) a ( 2 ) a ( 3 ) [¯ A ( 1 ) , ¯ A ( 2 ) , ¯ A ( 3 ) ] T = ¯ ◦ ¯ ◦ ¯ = [ ] , (1) r r r r = 1 Tensor structure: naturally leads to consider 3 blocks corresponding to the loading matrices A ( 1 ) , A ( 2 ) and A ( 3 ) . Proposed optimization problem A ( n ) ∈ R In × R , n ∈{ 1 , 2 , 3 } F ( A ( 1 ) , A ( 2 ) , A ( 3 ) )+ R 1 ( A ( 1 ) )+ R 2 ( A ( 2 ) )+ R 3 ( A ( 3 ) ) . minimize Some of the fastest classical approaches: Fast HALS [Phan et al., 2013] and N -Way [Bro, 1997] . 15 / 32
Introduction Proximal tools Application to CPD Numerical simulations Conclusion and future work Tensor matricization ( n ) I n , I − n ∈ R I n × I − n ◮ T the matrix obtained by unfolding the tensor T in the + n -th mode where the size I − n is equal to I 1 I 2 I 3 / I n ◮ Tensor expressed under matrix form as ( n ) ( − n ) ) ⊤ , I n , I − n = ¯ A ( n ) ( Z n ∈ { 1 , 2 , 3 } T where ( − 1 ) = ¯ A ( 3 ) ⊙ ¯ A ( 2 ) ∈ R I − 1 × R Z , + ( − 2 ) = ¯ A ( 3 ) ⊙ ¯ A ( 1 ) ∈ R I − 2 × R , Z + ( − 3 ) = ¯ A ( 2 ) ⊙ ¯ A ( 1 ) ∈ R I − 3 × R , Z + 16 / 32
Recommend
More recommend
