Linear unmixing Linear Mixing Model (LMM) y p = � R Linear Mixing Model (LMM): r = 1 m r a r , p + n p Main assumptions ◮ generally, additive (Gaussian) noise ◮ materials sitting side-by-side (as on a checkerboard) no interactions between materials ◮ no multiple scattering (e.g., due to relief) ◮ only the materials present in the considered pixel contribute no contribution from materials in neighboring pixels ◮ a single spectral signature characterizes each individual material no spectral variability ◮ generally, positivity and additivity (sum-to-one) constraints on a p a r , p ≥ 0 , r = 1 , . . . , R positivity: � R r = 1 a r , p = 1 sum-to-one: Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 17 / 77
Linear unmixing Representation in the hyperspectral space m 1 m 2 m 2 Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 18 / 77
Linear unmixing Representation in the signal subspace Hence, the interest of subspace learning methods as preprocessing steps (PCA, MNF , Hysime...) Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 19 / 77
Linear unmixing Representation in the signal subspace Hence, the interest of subspace learning methods as preprocessing steps (PCA, MNF , Hysime...) Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 19 / 77
Linear unmixing Endmember extraction Outline Introduction Linear unmixing Endmember extraction Inversion Joint approaches Illustrative results Non-linear unmixing Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 20 / 77
Linear unmixing Endmember extraction Endmember extraction (1) Exploiting convex geometry Searching for purest pixels ( ≈ simplex of max. volume inscribed in the data) (a) Pixel Purity Index (PPI) (b) N-FINDR or successive projections onto orthogonal subspaces (VCA, ORASIS). Searching for simplex of minimum volume inscribing the data “Minimum Volume Transform” (MVT) algorithms and variants. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 21 / 77
Linear unmixing Inversion Outline Introduction Linear unmixing Endmember extraction Inversion Joint approaches Illustrative results Non-linear unmixing Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 22 / 77
Linear unmixing Inversion Inversion (2) Constrained inverse problem Constrained optimization � a r , p ≥ 0 , ∀ r = 1 , . . . , R (ANC) J ( a ) = � y − M a � 2 Minimizing s.t. � R r = 1 a r = 1 (ASC ) with M = [ m 1 , . . . , m R ] . ◮ Fully Constrained Least Squares (FCLS) [Heinz et al. , 2001] , Remarks ◮ In a semi-supervised context, the materials m 1 , . . . , m R belong to a known library S = [ s 1 , . . . , s K ] ( K ≫ R ). The problem is written J ( a ) = � y − S a � 2 Minimizing s.t. ANC and ASC → complementary sparsity constraints on a . ◮ Other contextual constraints: spatial regularizations i.e., to promote smooth or spatially coherent abundance maps Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 23 / 77
Linear unmixing Inversion Inversion (2) Constrained inverse problem Constrained optimization � a r , p ≥ 0 , ∀ r = 1 , . . . , R (ANC) J ( a ) = � y − M a � 2 Minimizing s.t. � R r = 1 a r = 1 (ASC ) with M = [ m 1 , . . . , m R ] . ◮ Fully Constrained Least Squares (FCLS) [Heinz et al. , 2001] , Remarks ◮ In a semi-supervised context, the materials m 1 , . . . , m R belong to a known library S = [ s 1 , . . . , s K ] ( K ≫ R ). The problem is written J ( a ) = � y − S a � 2 Minimizing s.t. ANC and ASC → complementary sparsity constraints on a . ◮ Other contextual constraints: spatial regularizations i.e., to promote smooth or spatially coherent abundance maps Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 23 / 77
Linear unmixing Inversion Inversion (2) Statistical inference problem � a , σ 2 � Unknown parameter vector: θ = ◮ a = [ a 1 , . . . , a R ] T : vector of the R abundance coefficients, ◮ σ 2 : noise variance, Bayes paradigm: f ( θ | y ) ∝ f ( y | θ ) f ( θ ) with ◮ Likelihood: f ( y | θ ) (data-fitting term), ◮ Parameter prior distribution: f ( θ ) (penalization/regularization). Choice of the (Bayesian) estimators ◮ maximizing f ( θ | y ) to reach the maximum a posteriori (MAP) estimator optimization problem (see previously) ◮ computing the mean of f ( θ | y ) to derive the minimum mean square error (MMSE) estimator integration problem: use of Markov chain Monte Carlo algorithms (powerful but computationally demanding) Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 24 / 77
Linear unmixing Inversion Inversion (2) Statistical inference problem � a , σ 2 � Unknown parameter vector: θ = ◮ a = [ a 1 , . . . , a R ] T : vector of the R abundance coefficients, ◮ σ 2 : noise variance, Bayes paradigm: f ( θ | y ) ∝ f ( y | θ ) f ( θ ) with ◮ Likelihood: f ( y | θ ) (data-fitting term), ◮ Parameter prior distribution: f ( θ ) (penalization/regularization). Choice of the (Bayesian) estimators ◮ maximizing f ( θ | y ) to reach the maximum a posteriori (MAP) estimator optimization problem (see previously) ◮ computing the mean of f ( θ | y ) to derive the minimum mean square error (MMSE) estimator integration problem: use of Markov chain Monte Carlo algorithms (powerful but computationally demanding) Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 24 / 77
Linear unmixing Inversion Inversion (2) Computing the Bayesian estimators Maximum a posteriori (MAP) estimator ˆ θ MAP = arg max f ( θ | Y ) θ = arg max f ( Y | θ ) f ( θ ) . θ Minimum mean square error (MMSE) estimator ˆ θ MMSE = E [ θ | Y ] � = θ f ( θ | Y ) d θ � θ f ( Y | θ ) f ( θ ) d θ = � f ( Y | θ ) f ( θ ) d θ Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 25 / 77
Linear unmixing Joint approaches Outline Introduction Linear unmixing Endmember extraction Inversion Joint approaches Illustrative results Non-linear unmixing Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 26 / 77
Linear unmixing Joint approaches Joint approaches (1+2) For a given pixel p observed in L spectral bands: = � R y p r = 1 m r a r , p + n p = M a p + n p Now, consider P pixels: Y = MA + N where Y = [ y 1 , . . . , y P ] , M = [ m 1 , . . . , m R ] , A = [ a 1 , . . . , a P ] , N = [ n 1 , . . . , n P ] . Factorize Y ≈ MA under positivity and additivity constraints on A and positivity constraints on M Spectral Mixture Analysis = (constrained) matrix factorization = (constrained) blind source separation Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 27 / 77
Linear unmixing Joint approaches Joint approaches (1+2) For a given pixel p observed in L spectral bands: = � R y p r = 1 m r a r , p + n p = M a p + n p Now, consider P pixels: Y = MA + N where Y = [ y 1 , . . . , y P ] , M = [ m 1 , . . . , m R ] , A = [ a 1 , . . . , a P ] , N = [ n 1 , . . . , n P ] . Factorize Y ≈ MA under positivity and additivity constraints on A and positivity constraints on M Spectral Mixture Analysis = (constrained) matrix factorization = (constrained) blind source separation Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 27 / 77
Linear unmixing Joint approaches Matrix factorization problem Factorization Y ≈ MA formulated as the minimization problem � � � � min M , A D ( Y | MA ) = D ( y p | M a p ) = d y ℓ, p | [ M a p ] ℓ p p ,ℓ where D ( a | b ) is a “distance measure”, e.g., D ( a | b ) = � a − b � 2 . An ill-posed problem! � � MP , P − 1 A is a solution 1 . If { M , A } is a solution, ⇓ Additional constraints required! 1 For all P invertible matrix. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 28 / 77
Linear unmixing Joint approaches Matrix factorization problem Factorization Y ≈ MA formulated as the minimization problem � � � � min M , A D ( Y | MA ) = D ( y p | M a p ) = d y ℓ, p | [ M a p ] ℓ p p ,ℓ where D ( a | b ) is a “distance measure”, e.g., D ( a | b ) = � a − b � 2 . An ill-posed problem! � � MP , P − 1 A is a solution 1 . If { M , A } is a solution, ⇓ Additional constraints required! 1 For all P invertible matrix. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 28 / 77
Linear unmixing Joint approaches Matrix factorization problem Factorization Y ≈ MA formulated as the minimization problem � � � � min M , A D ( Y | MA ) = D ( y p | M a p ) = d y ℓ, p | [ M a p ] ℓ p p ,ℓ where D ( a | b ) is a “distance measure”, e.g., D ( a | b ) = � a − b � 2 . An ill-posed problem! � � MP , P − 1 A is a solution 1 . If { M , A } is a solution, ⇓ Additional constraints required! 1 For all P invertible matrix. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 28 / 77
Linear unmixing Joint approaches Matrix factorization strategies Y ≈ MA ⇔ Y T ≈ A T M T 1. Principal Component Analysis(PCA) ◮ Searching for orthogonal “principal components” (PCs) m r , ◮ PCs = directions with maximal variance in the data, ◮ Generally used as a dimension reduction procedure. 2. Independent Component Analysis (ICA) (of Y T ) ◮ Maximizing the statistical independence between the sources m r , ◮ Several measures of independence ⇒ several algorithms. 3. Nonnegative Matrix Factorization (NMF) ◮ Searching for M et A with positive entries, ◮ Several measures of divergence d ( ·|· ) ⇒ several algorithms. 4. (Fully Constrained) Spectral Mixture Analysis (SMA) ◮ Positivity constraints on m r ⇒ positive “sources” ◮ Positivity and sum-to-one constraints on a p ⇒ mixing coefficients = proportions/concentrations/probabilities. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 29 / 77
Linear unmixing Joint approaches Matrix factorization strategies Y ≈ MA ⇔ Y T ≈ A T M T 1. Principal Component Analysis(PCA) ◮ Searching for orthogonal “principal components” (PCs) m r , ◮ PCs = directions with maximal variance in the data, ◮ Generally used as a dimension reduction procedure. 2. Independent Component Analysis (ICA) (of Y T ) ◮ Maximizing the statistical independence between the sources m r , ◮ Several measures of independence ⇒ several algorithms. 3. Nonnegative Matrix Factorization (NMF) ◮ Searching for M et A with positive entries, ◮ Several measures of divergence d ( ·|· ) ⇒ several algorithms. 4. (Fully Constrained) Spectral Mixture Analysis (SMA) ◮ Positivity constraints on m r ⇒ positive “sources” ◮ Positivity and sum-to-one constraints on a p ⇒ mixing coefficients = proportions/concentrations/probabilities. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 29 / 77
Linear unmixing Joint approaches Matrix factorization strategies Y ≈ MA ⇔ Y T ≈ A T M T 1. Principal Component Analysis(PCA) ◮ Searching for orthogonal “principal components” (PCs) m r , ◮ PCs = directions with maximal variance in the data, ◮ Generally used as a dimension reduction procedure. 2. Independent Component Analysis (ICA) (of Y T ) ◮ Maximizing the statistical independence between the sources m r , ◮ Several measures of independence ⇒ several algorithms. 3. Nonnegative Matrix Factorization (NMF) ◮ Searching for M et A with positive entries, ◮ Several measures of divergence d ( ·|· ) ⇒ several algorithms. 4. (Fully Constrained) Spectral Mixture Analysis (SMA) ◮ Positivity constraints on m r ⇒ positive “sources” ◮ Positivity and sum-to-one constraints on a p ⇒ mixing coefficients = proportions/concentrations/probabilities. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 29 / 77
Linear unmixing Joint approaches Matrix factorization strategies Y ≈ MA ⇔ Y T ≈ A T M T 1. Principal Component Analysis(PCA) ◮ Searching for orthogonal “principal components” (PCs) m r , ◮ PCs = directions with maximal variance in the data, ◮ Generally used as a dimension reduction procedure. 2. Independent Component Analysis (ICA) (of Y T ) ◮ Maximizing the statistical independence between the sources m r , ◮ Several measures of independence ⇒ several algorithms. 3. Nonnegative Matrix Factorization (NMF) ◮ Searching for M et A with positive entries, ◮ Several measures of divergence d ( ·|· ) ⇒ several algorithms. 4. (Fully Constrained) Spectral Mixture Analysis (SMA) ◮ Positivity constraints on m r ⇒ positive “sources” ◮ Positivity and sum-to-one constraints on a p ⇒ mixing coefficients = proportions/concentrations/probabilities. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 29 / 77
Linear unmixing Joint approaches Geometrical formulation of SMA SMA = looking for a simplex enclosing the data Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 30 / 77
Linear unmixing Joint approaches Geometrical formulation of SMA In practice: non-unique solution + trade-off noise vs. constraints... Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 31 / 77
Linear unmixing Joint approaches Matrix factorization strategies NMF-based algorithms ◮ Assumption: positivity of the endmember spectra and the abundances, ◮ Various counterparts, to handle additional constraints sum-to-one, minimum volume, (collaborative) sparsity... Example: MVC-NMF 2 Bayesian estimation ◮ Choice of prior ensuring various constraints, ◮ Allows nuisance parameters to be jointly estimated noise parameters, hyperparameters, classification maps ◮ Difficult statistical estimation → MCMC algorithm. Example: BLU 3 (efficient implementation strategies available 4 ) 2 Miao and Qi, IEEE TGRS ,2007 3 Dobigeon et al. , IEEE TSP , 2009. 4 Schmidt et al. , IEEE TGRS , 2010. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 32 / 77
Linear unmixing Joint approaches Spectral Mixture Analysis... ... and applications y 1 , p a 1 , p . . m 1 . = m R . p = 1 , . . . , P . . . . . y L , p a R , p Multi-band imaging ◮ remote sensing 5 , planetology 6 , EELS 7 ,... ◮ evolution parameter: pixel index in the image. Spectrochemical analysis ◮ Raman 8 , NIR 9 ,... ◮ evolution parameter: time, temperature,... Bioinformatics ◮ gene expression analysis 1 0 ◮ evolution parameter: time, subject, treatment,... 5 Keshava et al. , IEEE SP Mag., 2002. 6 Schmidt et al. , IEEE TGRS, 2010. 7 de la Pe˜ na et al. , Ultramicroscopy, 2011. 8 Dobigeon et al. , SP , 2009. 9 Moussaoui, IEEE TSP , 2006. 10 Huang et al. , PLoS Genetics, 2011. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 33 / 77
Linear unmixing Joint approaches Spectral Mixture Analysis... ... and applications y 1 , p a 1 , p . . m 1 . = m R . p = 1 , . . . , P . . . . . y L , p a R , p Multi-band imaging ◮ remote sensing 5 , planetology 6 , EELS 7 ,... ◮ evolution parameter: pixel index in the image. Spectrochemical analysis ◮ Raman 8 , NIR 9 ,... ◮ evolution parameter: time, temperature,... Bioinformatics ◮ gene expression analysis 1 0 ◮ evolution parameter: time, subject, treatment,... 5 Keshava et al. , IEEE SP Mag., 2002. 6 Schmidt et al. , IEEE TGRS, 2010. 7 de la Pe˜ na et al. , Ultramicroscopy, 2011. 8 Dobigeon et al. , SP , 2009. 9 Moussaoui, IEEE TSP , 2006. 10 Huang et al. , PLoS Genetics, 2011. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 33 / 77
Linear unmixing Joint approaches Spectral Mixture Analysis... ... and applications y 1 , p a 1 , p . . m 1 . = m R . p = 1 , . . . , P . . . . . y L , p a R , p Multi-band imaging ◮ remote sensing 5 , planetology 6 , EELS 7 ,... ◮ evolution parameter: pixel index in the image. Spectrochemical analysis ◮ Raman 8 , NIR 9 ,... ◮ evolution parameter: time, temperature,... Bioinformatics ◮ gene expression analysis 1 0 ◮ evolution parameter: time, subject, treatment,... 5 Keshava et al. , IEEE SP Mag., 2002. 6 Schmidt et al. , IEEE TGRS, 2010. 7 de la Pe˜ na et al. , Ultramicroscopy, 2011. 8 Dobigeon et al. , SP , 2009. 9 Moussaoui, IEEE TSP , 2006. 10 Huang et al. , PLoS Genetics, 2011. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 33 / 77
Linear unmixing Joint approaches Spectral Mixture Analysis... ... and applications y 1 , p a 1 , p . . m 1 . = m R . p = 1 , . . . , P . . . . . y L , p a R , p Multi-band imaging ◮ remote sensing 5 , planetology 6 , EELS 7 ,... ◮ evolution parameter: pixel index in the image. Spectrochemical analysis ◮ Raman 8 , NIR 9 ,... ◮ evolution parameter: time, temperature,... Bioinformatics ◮ gene expression analysis 1 0 ◮ evolution parameter: time, subject, treatment,... 5 Keshava et al. , IEEE SP Mag., 2002. 6 Schmidt et al. , IEEE TGRS, 2010. 7 de la Pe˜ na et al. , Ultramicroscopy, 2011. 8 Dobigeon et al. , SP , 2009. 9 Moussaoui, IEEE TSP , 2006. 10 Huang et al. , PLoS Genetics, 2011. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 33 / 77
Linear unmixing Illustrative results Outline Introduction Linear unmixing Endmember extraction Inversion Joint approaches Illustrative results Non-linear unmixing Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 34 / 77
Linear unmixing Illustrative results Experimental results: AVIRIS data [Dobigeon et al. , IEEE Trans. SP , 2010] Simulation parameters ◮ Image: 50 × 50 pixels (Moffett field), L = 224 bands. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 35 / 77
Linear unmixing Illustrative results Experimental results: AVIRIS data [Dobigeon et al. , IEEE Trans. SP , 2010] Simulation parameters ◮ Image: 50 × 50 pixels (Moffett field), L = 224 bands. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 35 / 77
Linear unmixing Illustrative results Experimental results: EELS data [Dobigeon et al. , Ultramiscroscopy , 2012] Simulation parameters ◮ Image: 64 × 64 pixels, L = 1340 energy channels. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 36 / 77
Linear unmixing Illustrative results Experimental results: EELS data [Dobigeon et al. , Ultramiscroscopy , 2012] Simulation parameters ◮ Image: 64 × 64 pixels, L = 1340 energy channels. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 36 / 77
Linear unmixing Illustrative results Experimental results: EELS data [Dobigeon et al. , Ultramiscroscopy , 2012] Simulation parameters ◮ Image: 64 × 64 pixels, L = 1340 energy channels. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 36 / 77
Linear unmixing Illustrative results Experimental results: Mars data [Schmidt et al. , IEEE Trans. GRS , 2010] OMEGA data ◮ L = 184 spectral bands, ≈ 300 × 400 pixels, ◮ 3 materials: CO 2 , dust, H 2 0. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 37 / 77
Linear unmixing Illustrative results Experimental results: Mars data [Schmidt et al. , IEEE Trans. GRS , 2010] OMEGA data ◮ L = 184 spectral bands, ≈ 300 × 400 pixels, ◮ 3 materials: CO 2 , dust, H 2 0. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 37 / 77
Linear unmixing Illustrative results Experimental results: spectrometrics [Dobigeon et al. , Signal Processing , 2009] Chemical mixtures ◮ L = 1000 spectral bands, 10 observed mixtures over time, ◮ 3 materials: chemical species (reactant A, intermediate C, product D). Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 38 / 77
Linear unmixing Illustrative results Experimental results: spectrometrics [Dobigeon et al. , Signal Processing , 2009] Chemical mixtures ◮ L = 1000 spectral bands, 10 observed mixtures over time, ◮ 3 materials: chemical species (reactant A, intermediate C, product D). (a) Reactant A (b) Intermediate C (c) Product D (d) Mixing coefficients Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 38 / 77
Linear unmixing Illustrative results Experimental results: genetics [Huang et al , PLoS Genetics , 2011] Data · 267 blood samples collected for 17 voluntary subjects along time after Influenza A (H3N2) inoculation. · expressions of more than 12000 genes in each sample. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 39 / 77
Linear unmixing Illustrative results Experimental results: genetics [Huang et al , PLoS Genetics , 2011] Data · 267 blood samples collected for 17 voluntary subjects along time after Influenza A (H3N2) inoculation. · expressions of more than 12000 genes in each sample. Results · identification of a group of genes involved in the occurrence of symptoms (inflammatory factor) Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 39 / 77
Linear unmixing Illustrative results Experimental results: genetics [Huang et al , PLoS Genetics , 2011] Data · 267 blood samples collected for 17 voluntary subjects along time after Influenza A (H3N2) inoculation. · expressions of more than 12000 genes in each sample. Results · identification of a group of genes involved in the occurrence of symptoms (inflammatory factor) Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 39 / 77
Linear unmixing Illustrative results Experimental results: genetics [Huang et al , PLoS Genetics , 2011] Data · 267 blood samples collected for 17 voluntary subjects along time after Influenza A (H3N2) inoculation. · expressions of more than 12000 genes in each sample. Results · identification of a group of genes involved in the occurrence of symptoms (inflammatory factor) Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 39 / 77
Non-linear unmixing Outline Introduction Linear unmixing Endmember extraction Inversion Joint approaches Illustrative results Non-linear unmixing Intimate mixing models Bilinear mixing models Post-nonlinear mixing models Robust linear mixing models Illustrative results Linear or nonlinear unmixing? Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 40 / 77
Non-linear unmixing Non-linear mixing models Non-linear Mixing Model: y p = g θ ( a p , m 1 , . . . , m R ) + n p Reference: IEEE Signal Proc. Magazine, Jan. 2002. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 41 / 77
Non-linear unmixing Non-linear mixing models Numerous nonlinear models proposed in the literature Physics-based models ◮ mainly derived from the radiative transfer theory, ◮ require prior knowledge about external parameters related to the studied scene (leaf index area, geometry or illumination incidence,...) ◮ not developed for unmixing purpose � unmixing algorithms not easily available. Bilinear, polynomial and robust models ◮ mainly derived from intuitive considerations, ◮ do not require prior knowledge about external parameters, ◮ developed for unmixing purpose � (various) unmixing algorithms available. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 42 / 77
Non-linear unmixing Non-linear mixing models Numerous nonlinear models proposed in the literature Physics-based models ◮ mainly derived from the radiative transfer theory, ◮ require prior knowledge about external parameters related to the studied scene (leaf index area, geometry or illumination incidence,...) ◮ not developed for unmixing purpose � unmixing algorithms not easily available. Bilinear, polynomial and robust models ◮ mainly derived from intuitive considerations, ◮ do not require prior knowledge about external parameters, ◮ developed for unmixing purpose � (various) unmixing algorithms available. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 42 / 77
Non-linear unmixing Intimate mixing models Outline Introduction Linear unmixing Non-linear unmixing Intimate mixing models Bilinear mixing models Post-nonlinear mixing models Robust linear mixing models Illustrative results Linear or nonlinear unmixing? Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 43 / 77
Non-linear unmixing Intimate mixing models Intimate mixing models Non-linear Mixing Model: y p = g θ ( a p , m 1 , . . . , m R ) + n p Reference: IEEE Signal Proc. Magazine, Sept. 2014. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 44 / 77
Non-linear unmixing Intimate mixing models Intimate mixtures Assumptions ◮ interactions occur at a small scales ◮ abundances associated with the relative mass fractions of the materials ◮ popular models: those promoted by Hapke (1993) based on meaningful/interpretable parameters with physical significance Intimate unmixing: not easy... Strongly depend on external parameters related to the acquisition e.g., geometry of the scene ◮ use of neural-networks (NN) to learn the nonlinear function → training data needed ◮ kernel-based unmixing implicitly relies on the Hapke model Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 45 / 77
Non-linear unmixing Bilinear mixing models Outline Introduction Linear unmixing Non-linear unmixing Intimate mixing models Bilinear mixing models Post-nonlinear mixing models Robust linear mixing models Illustrative results Linear or nonlinear unmixing? Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 46 / 77
Non-linear unmixing Bilinear mixing models Bilinear mixing models Non-linear Mixing Model: y p = g θ ( a p , m 1 , . . . , m R ) + n p Reference: IEEE Signal Proc. Magazine, Sept. 2014. ◮ Possible interactions between the components of the scene, ◮ Nonlinear terms included into the mixing model. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 47 / 77
Non-linear unmixing Bilinear mixing models Bilinear mixing models R R − 1 R � � � y p = a r , p m r + b i , j , p m i ⊙ m j + n p r = 1 i = 1 j = i + 1 � �� � � �� � linear term nonlinear term where b i , j , p tunes the amount of nonlinearity resulting from the interactions between m i and m j . ◮ several bilinear models proposed in the literature ◮ mainly differ by the definition (and related constraints) of b i , j , p Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 48 / 77
Non-linear unmixing Bilinear mixing models Bilinear mixing models R R − 1 R � � � y p = a r , p m r + b i , j , p m i ⊙ m j + n p r = 1 i = 1 j = i + 1 � �� � � �� � linear term nonlinear term where b i , j , p tunes the amount of nonlinearity resulting from the interactions between m i and m j . ◮ several bilinear models proposed in the literature ◮ mainly differ by the definition (and related constraints) of b i , j , p Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 48 / 77
Non-linear unmixing Bilinear mixing models Nascimento’s model (NM) Definition 6 , 7 R R − 1 R � � � y = a r m r + β i , j m i ⊙ m j + n r = 1 i = 1 j = i + 1 � �� � � �� � linear term interaction term m 1 , i m 1 , j . where ⊙ is the termwise (Hadamard) product: m i ⊙ m j = . . . m L , i m L , j Constraints a r ≥ 0 , β i , j ≥ 0 , r , i = 1 , . . . , R j = i + 1 , . . . R positivity: � R r = 1 a r + � R − 1 � R j = i + 1 β i , j = 1 sum-to-one: i = 1 6 Nascimento and Bioucas-Dias, in Proc. SPIE , Sept. 2009. 7 Somers et al. , Remote Sens. Env. , 2009. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 49 / 77
Non-linear unmixing Bilinear mixing models Nascimento’s model (NM) R R − 1 R � � � y = a r m r + β i , j m i ⊙ m j + n � �� � r = 1 i = 1 j = i + 1 new spectrum Properties ◮ defined by the abundances a r , p and the nonlinearity coefficients β i , j ◮ interactions between m i and m j adjusted/quantified with β i , j ◮ if ∀ i , j , β i , j = 0, reduces to the LMM. ◮ if ∃ i , j , β i , j � = 0, abundances do not satisfy � r a r , p = 1 ◮ can be seen as an LMM with R ∗ = 1 2 R ( R + 1 ) correlated endmembers R ∗ � a ∗ r m ∗ y = r + n r = 1 a ∗ r = a r m ∗ r = m r r = 1 , . . . , R , and a ∗ r = β i , j m ∗ r = m i ⊙ m j R + 1 ≤ r ≤ R ∗ . and ◮ Unmixing procedures similar to procedures assuming the LMM. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 50 / 77
Non-linear unmixing Bilinear mixing models Fan’s model (FM) Definition 8 R R − 1 R � � � y = a r m r + a i a j m i ⊙ m j + n r = 1 i = 1 j = i + 1 � �� � � �� � linear term interaction term Constraints positivity: a r ≥ 0 , r = 1 , . . . , R � R sum-to-one: r = 1 a r = 1 ◮ Interaction coefficient in NM fixed as b i , j = a i a j , ◮ Absence of material m i ⇒ a i = 0 ⇒ no nonlinearity terms m i ⊙ m · , ◮ Specific unmixing strategies must be designed. 8 Fan et al., Int. J. Remote Sensing , June 2009. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 51 / 77
Non-linear unmixing Bilinear mixing models Generalized bilinear model (GBM) Definition 9 R R − 1 R � � � y = + γ i , j a i a j m i ⊙ m j + n a r m r r = 1 i = 1 j = i + 1 � �� � � �� � linear term interaction term Constraints ◮ Abundance coefficients a r ≥ 0 , r = 1 , . . . , R positivity: � R sum-to-one: r = 1 a r = 1 ◮ Nonlinearity coefficients 0 ≤ γ i , j ≤ 1 , i = 1 , . . . , R − 1 , j = i + 1 , . . . , R 9 Halimi et al., IEEE Trans. Geosci. and Remote Sensing , 2011. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 52 / 77
Non-linear unmixing Bilinear mixing models Generalized bilinear model (GBM) R R − 1 R � � � y = a r m r + γ i , j a i a j m i ⊙ m j + n r = 1 i = 1 j = i + 1 ◮ Absence of material m i ⇒ a i = 0 ⇒ no nonlinearity terms m i ⊙ m · , ◮ Generalization of existing mixing models ◮ γ i , j = 0, ∀ i , j , → LMM ◮ γ i , j = 1, ∀ i , j , → FM ◮ Interactions between m i and m j adjusted/quantified with γ i , j . ◮ Specific unmixing strategies must be designed. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 53 / 77
Non-linear unmixing Bilinear mixing models Bilinear models: summary Nascimento’s model (NM) R R − 1 R � � � y = a r m r + β i , j m i ⊙ m j + n r = 1 i = 1 j = i + 1 Fan’s model (FM) R R − 1 R � � � y = a r m r + a i a j m i ⊙ m j + n r = 1 i = 1 j = i + 1 Generalized bilinear model (GBM) R R − 1 R � � � y = a r m r + γ i , j a i a j m i ⊙ m j + n r = 1 i = 1 j = i + 1 Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 54 / 77
Non-linear unmixing Bilinear mixing models Bilinear models: summary Clusters of observations generated according to the LMM, the NM, the FM and the GBM (blue) and the corresponding endmembers (red). Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 55 / 77
Non-linear unmixing Bilinear mixing models An extended bilinear model: linear-quadratic mixing model Definition 10 R R R � � � y = a r m r + β i , j m i ⊙ m j + n r = 1 i = 1 j = i � �� � � �� � linear term interaction term Constraints positivity: a r ≥ 0 , r = 1 , . . . , R β i , j ≥ 0 , ∀ i , j � R sum-to-one: r = 1 a r = 1 ◮ Presence of quadratic terms m i ⊙ m i , ◮ Specific unmixing strategies must be designed. 10 Meganem et al., IEEE Trans. Geosci. and Remote Sensing , 2013. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 56 / 77
Non-linear unmixing Post-nonlinear mixing models Outline Introduction Linear unmixing Non-linear unmixing Intimate mixing models Bilinear mixing models Post-nonlinear mixing models Robust linear mixing models Illustrative results Linear or nonlinear unmixing? Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 57 / 77
Non-linear unmixing Post-nonlinear mixing models Post-nonlinear mixing models (PNMM) Definition 11 � R � � y = g a r m r + n r = 1 Constraints positivity: a r ≥ 0 , r = 1 , . . . , R � R sum-to-one: r = 1 a r = 1 ◮ General class of nonlinear models studied for source separation 12 , 13 , ◮ g : unknown linear/nonlinear application. 11 Altmann et al., IEEE Trans. Image Process. , 2013. 12 Jutten and Karhunen, in Proc. 4th ICA Workshop , Apr. 2003. 13 Babaie-Zadeh et al., in Proc. 3rd ICA Workshop , Jun. 2001. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 58 / 77
Non-linear unmixing Post-nonlinear mixing models Post-nonlinear mixing models (PNMM) A particular instance: polynomial PNMM → R L R L g : � � T s 1 + bs 2 1 , . . . , s L + bs 2 s �→ L Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 59 / 77
Non-linear unmixing Post-nonlinear mixing models Post-nonlinear mixing models (PNMM) A particular instance: polynomial PNMM → R L R L g : � � T s 1 + bs 2 1 , . . . , s L + bs 2 s �→ L Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 59 / 77
Non-linear unmixing Post-nonlinear mixing models Post-nonlinear mixing models (PNMM) A particular instance: polynomial PNMM Mathematical formulation y = M a + b ( M a ) ⊙ ( M a ) + n R R � � = M a + b a i a j m i ⊙ m j + n i = 1 j = 1 Remarks ◮ derived from a second-order Taylor expansion of the nonlinearity function: g ( s ) ≈ s + bs 2 + o ( s 2 ) . ◮ defined by the abundances a r and the nonlinearity parameter b . ◮ nonlinear interactions adjusted/quantified with a unique parameter b . ◮ b = 0 ⇒ LMM, b � = 0 ⇒ nonlinear model. ◮ when b � = 0, contains bilinear and quadratic terms m i ⊙ m j similar to other nonlinear models (bilinear and quadratic-linear). ◮ specific unmixing algorithms must be designed. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 60 / 77
Non-linear unmixing Robust linear mixing models Outline Introduction Linear unmixing Non-linear unmixing Intimate mixing models Bilinear mixing models Post-nonlinear mixing models Robust linear mixing models Illustrative results Linear or nonlinear unmixing? Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 61 / 77
Non-linear unmixing Robust linear mixing models Robust linear mixing models (rLMM) Mathematical formulation y = M a + r + n Remarks ◮ r represents any potential deviation from the classical LMM nonlinearity, outliers, spectral variability ◮ r not explicitly parametrized by the endmember spectra or their respective proportions ◮ r = 0 ⇒ LMM, r � = 0 ⇒ nonlinear model. ◮ the rLMM’s differ by the additional constraints/regularizations e.g., spatial sparsity 14 , spatial-spectral sparsity 15 , endmember-driven 16 ... 14 C. Fevotte and N. Dobigeon, IEEE TIP , 2015. 15 G. Newstadt et al. , in IEEE Proc. SSP , 2014. 16 Y. Altmann, N. Dobigeon et al. , IEEE TIP , 2014. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 62 / 77
Non-linear unmixing Illustrative results Outline Introduction Linear unmixing Non-linear unmixing Intimate mixing models Bilinear mixing models Post-nonlinear mixing models Robust linear mixing models Illustrative results Linear or nonlinear unmixing? Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 63 / 77
Non-linear unmixing Illustrative results Spectral unmixing of the “Moffett Field” image with the generalized bilinear model Linear model GBM Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 64 / 77
Non-linear unmixing Illustrative results Spectral unmixing of the “Moffett Field” image with the robust LMM Abundance maps Residual maps Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 65 / 77
Non-linear unmixing Linear or nonlinear unmixing? Outline Introduction Linear unmixing Non-linear unmixing Intimate mixing models Bilinear mixing models Post-nonlinear mixing models Robust linear mixing models Illustrative results Linear or nonlinear unmixing? Conclusion Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 66 / 77
Non-linear unmixing Linear or nonlinear unmixing? To be or not to be... (non-)linear? ◮ PPNMM and rLMM sufficiently flexible to handle linear and nonlinear mixtures... ... at a price of a higher computational cost ◮ LMM-based endmember extraction algorithms can still be valid in case of weak nonlinearities requirement: pure pixels are still endmembers (i.e., extrema) Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 67 / 77
Non-linear unmixing Linear or nonlinear unmixing? To be or not to be... (non-)linear? Figure: Linearly mixed pixels. Nicolas Dobigeon Spectral Mixture Analysis – Linear mixing model and beyond 67 / 77
Recommend
More recommend