Dimitri Nion & Lieven De Lathauwer K.U. Leuven, Kortrijk campus, - PowerPoint PPT Presentation

The Joint Block Diagonalization (JBD) problem: a tensor framework. Dimitri Nion & Lieven De Lathauwer K.U. Leuven, Kortrijk campus, Belgium E-mails: Dimitri.Nion@kuleuven-kortrijk.be Lieven.DeLathauwer@kuleuven-kortrijk.be TDA 2010 , Monopoli, Italy, September 13-17, 2010 1

Joint-Block-Diagonalization (JBD): model I D 1 K 0 T X A L 1 L 1 L R 1 K D D RK 11 0 = I A A X T A 1 R 0 L R 1 D R 1 R      T H X AD A ( N ) or X AD A ( N ), k 1 ,..., K k k k k k k JBD is a generalization of JD (Joint Diagonalization)/INDSCAL D K c 1 c R 0 0 X R K + … + T = = A a 1 a R 0 X 0 a R A I 1 a 1 D 1 2

JBD : ambiguities I D 0 1 K T L 1 L R A L 1 X 1 D D RK 0 K 11 Z -1 Z -T Z T Z = T I 0 A A A D L R X R 1 1 R 1 R ~ T ~  K = { D } A A k  k 1 Observation: if you choose Z arbitrary, you lose the JBD structure. Question: what is the structure of Z such that the JBD model is still valid? 3

JBD : essential uniqueness   Z  K ΛΠ The JBD of X is said essentially unique if  k k 1 Λ  an arbitrary block-diagonal matrix, Π  an arbitrary block-wise permutation matrix. 1 0 Λ 0 1 1 1 0 Λ ~ ~ ~ = A A A 0 1 2 A A A 1 2 3 1 1 2 3 0 Λ 0 3 1 Solving a JBD problem Estimation of {Span( A r )} r=1,…,R in an arbitrary order 4

JBD: State of the art JBD is becoming popular signal processing tool in applications such as:  Blind Source Separation (BSS) of convolutive mixtures in the time-domain,  Independent Subspace Analysis. Two approaches in the literature:  Approach 1: Unitary-JBD [Abed Meraim and Belouchrani, 2004] « A is a square unitary matrix » ( A T A = I )  Approach 2: Non-Unitary JBD  Approach 2.1: [H. Ghennioui, N. Thirion-Moreau, E. Moreau, 2008, 2010] « A is tall and full column-rank. »  Indeed, their approach only works if A is a square non-unitary matrix.  Approach 2.2: This talk « A can be a tall, square or fat non-unitary matrix »  JBD is a particular instance of Block-Component-Decompositions  Computation by a gradient-based algorithm  In the square case, better performance than 2.1 5

Joint-Block-Diagonalization : state of the art (1)  Approach 1: Unitary-JBD [Abed Meraim and Belouchrani, 2004]  A is square unitary matrix ( A T A = I ) A D k A T (+ N k ) A T X k A N k A T ) X k = A D A = D k + (A N K K   2 2 T T max bdiag ( A X A ) or min offbdiag ( A X A ) k k F F A A   k 1 k 1 X K = X 1 A T A 6

Joint-Block-Diagonalization : state of the art (2)  Approach 2: Non-Unitary-JBD [H. Ghennioui, N. Thirion-Moreau, E. Moreau, 2008, 2010]  A is tall and full column-rank (Let B = A , then BA = I ) BX k B T = D k + (B N k B T ) X k = A D A D k A T (+ N k ) BX K K   2 2 T T max bdiag ( BX B ) or min offbdiag ( BX B ) k k F F B B   k 1 k 1 X K = X B T 1 B 7

Joint-Block-Diagonalization : state of the art (3)  Approach 2: Non-Unitary-JBD [H. Ghennioui, N. Thirion-Moreau, E. Moreau, 2008, 2010]  2 gradient-descent based algorithms, JBD OG and JBD ORG to solve K  2   T min offbdiag ( BX B ) ( 1 ) off k F B  k 1  Drawbacks of approach 2:  B =0 is a trivial minimizer  Under-determined case ( A fat, I<N) not handled, since it is assumed that BA = I  Indeed, the over-determined case ( A tall, I>N) is not successfully handled either   T T T because if is solution of an exact JBD problem, i.e., B [ B , B ] A 1 2     T T T ( BX B ) ( BAD A B ) ( D ) 0 , offbdiag offbdiag offbdiag k k k k ~   T T T then is also solution of (1) but not solution of the JBD problem B [( A ) , B ] 2   ~ 0 0 ~     T T T T because is not full rank. Idem for B [ B ( A ) , B ] B A   1 2   X X 8

Joint-Block-Diagonalization : our contributions  Starting point: the gradient-descent based algorithms JBD OG and JBD ORG of [H. Ghennioui, N. Thirion-Moreau, E. Moreau, 2008, 2010] for Non-Unitary JBD can only handle the case where A is square (I=N).  Main motivation: Propose a novel technique to solve non-unitary JBD problems that can handle exactly-, over- and under- determined cases (i.e., A may be square, tall or fat).  Main contributions:  Formulate JBD as a tensor decomposition fitting problem;  Build a Conjugate Gradient (CG) algorithm to compute the tensor decomposition;  In the over-determined case, build a good starting point for any JBD algorithm;  Application: blind source separation via Independent Subspace Analysis (ISA). 9

JBD in tensor format : link to BCD D 0 1 K X L 1 L R T A K 1 = D D RK 0 11 I A A X 1 R T 1 A 0 R D R 1 X R L r  T A = r I A D r  r r 1   R   X D A A r 1 r 2 r  r 1 JD particular case of candecomp-parafac   R JBD particular case of BCD-(L r , M r , .), (« Block-  A  X D * A (or Component-Decomposition in rank-(L r , M r , .) terms ») r 1 r 2 r  r 1 in case of hermitian symmetry) 10

JBD in tensor format : conditions for essential uniqueness   R  BCD-(L r , M r , .) :   where A =[ A 1 ,…, A R ] is I by N X D A B r 1 r 2 r B =[ B 1 ,…, B R ] is J by Q  r 1  Theorem [De Lathauwer, 2008] : Suppose that rank( A )=N, rank( B )=Q, K>2 and that the tensors { D r } r=1,…,R are generic, then the BCD-(L r , M r , .) of X is essentially unique (Sufficient condition).   R   X D  JBD : A =[ A 1 ,…, A R ] is I by N A A where r 1 r 2 r  r 1  The same theorem can be invoked (the proof still holds with A instead of B )  In summary, it means that JBD is generically unique if K>2 and rank( A )=N  This is only a sufficient condition: uniqueness still holds but is harder to prove in several cases where the condition is not satisfied.  For instance, uniqueness may still hold when rank( A )=I ( A fat, I<N) 11

JBD in tensor format : cost function R    2     N N X D min A A with LS r 1 r 2 r   F R D , A   r r 1 1 r K R    2   T N N X A D A with k k k r kr r F   1 1 k r  2   . N with N X ( A A ) D F Where: .  A A = [ A 1 A 1 , …, A R X X A R ] is the Khatri-Rao product (block-wise Kronecker product)  X and N are the I 2 xK matrix unfoldings of the IxIxK tensors X and N , resp. 12

JBD in tensor format : derivation of the gradient Real-valued data K       T T ( ) 2 ( N AD N AD ) ,   T X AD A N A LS k k k k  k k k k 1 .     T ( ) 2 ( A A ) N D LS K Complex-valued data,       * T * * H ( ) ( N A D N A D ) , standard symmetry * LS k k k k A  k 1   T .     X AD A N H ( ) ( A A ) N k k k * LS D Complex-valued data, K  hermitian symmetry      H H ( ) ( N AD N AD ) , * LS k k k k A    k 1 H X AD A N .     * T k k k ( ) ( A A ) N * LS D 13

Conjugate Gradient algorithm for JBD: JBD-CG (stop criterion not satisfied) while   1 - Compute and from and D A A D 2 - Update search directions  2.1 - Compute (e.g. with stabilized Polak - Ribière formula) 2.2 - New search directions :     d d A A A     d d D D D 3 - Joint Exact Line Search :         ( , ) arg min ( A d , D d ) A D LS A A D D   , A D 4 - Update unknwons :    A A d A A    D D d D D End 14

Step 3 of JBD-CG : Joint Exact Line Search      min ( A d , D d ) LS A A D D   , A D   .              2 ( A d , D d ) ( A d ) ( A d ) ( D d ) X A A D D A A A A D D LS F         2 Q ( ) 2 Q ( ) Q ( ) ( 1 ) D 2 A D 1 A 0 A   LS     ( , ) Q ( )   Solve A D   0 1 A ( 2 )    D Q ( ) D 2 A      2 Q ( ) Q ( ) Q ( )     Substitute (2) in (1): 1 A 0 A 2 A ( ) ( 3 )  A LS ( ) Q 2 A   Minimize (3) w.r.t. (degree 7 polynomial rooting) A    Given , is given by (2) A D 15