of Hyper-Spectral Images via Diffusion Bases The 9th International - PowerPoint PPT Presentation

~ 1 ~ Unsupervised Segmentation of Hyper-Spectral Images via Diffusion Bases The 9th International Joint Conference on Computational Intelligence NCTA 2017, Funchal, Madeira, Portugal Dr. Alon Schclar School of Computer Science, Academic College of Tel-Aviv Yafo Prof. Amir Averbuch School of Computer Science, Tel-Aviv University

~ 2 ~ Introduction • Segmentation – Cluster similar pixels in an image Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 3 ~ Introduction – cont. • Hyper-spectral imagery provides much more information than RGB images • Currently cheaper than before to acquire • However – Large volume – Redundant – Noise • Efficiently utilize the extra information Washington DC Mall Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 4 ~ The Wavelength-wise Global (WWG) Segmentation Algorithm • Diffusion Bases Dimensionality Reduction • Normalization • Quantization • Color frequency calculation • Finding most frequent colors (peaks) • Finding the nearest peak to each quantized color Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 5 ~ Dimensionality reduction • Definition: – Embedding of high-dimensional data into a low- dimensional space that preserves the “ important ” characteristics of the data. – Minimum distortion of the geometry Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 6 ~ Dimensionality reduction • Formal definition – A set of vectors 𝒐 , 𝒚 𝒋 ∈ ℝ 𝑬 𝚫 = 𝒚 𝒋 𝒋=𝟐 is mapped (while satisfying a constraint for example all pair- wise distances preserved ) into a low dimensional space = 𝒚 𝒐 𝒋 ∈ ℝ 𝜽 𝚫 𝒋 𝒋=𝟐 , 𝒚 where D >> 𝜽 Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 7 ~ Dimensionality reduction in Hyper-spectral images • Represent the information in each hyper-pixel by a small number of values Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 8 ~ The Wavelength-wise Global (WWG) Segmentation Algorithm • Diffusion Bases Dimensionality Reduction • Normalization • Quantization • Color frequency calculation • Finding most frequent colors (peaks) • Finding the nearest peak to each quantized color Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 9 ~ Diffusion Bases Dimensionality Reduction Schclar and Averbuch (IJCCI 2015) • Utilizes the similarity among the coordinates of the dataset • When the dimensionality is much smaller than the number of the points – Useful for hyper-spectral images – Point-wise dimensionality reduction is these cases requires sampling and out-of-sample extension (e.g. Nyström ) • Dual to Diffusion Maps ( Coifman and Lafon 2006 ) Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 10 ~ Diffusion Bases Construct the similarity matrix 𝑿 ∈ ℝ 𝑬×𝑬 1. between the coordinates of the dataset – Each waveband image is a coordinate 2. Normalize each row to sum to 1 giving P (Markov matrix) Find the right eigenvectors {𝜊 𝑗 } ∈ ℝ 𝑬 of P and 3. their eigenvalues { λ i } 4. Embedding of every hyper-pixel u is given by Φ: 𝑣 ⟼ 𝜊 1 , 𝑣 , 𝜊 2 , 𝑣 , … , 𝜊 𝜃 , 𝑣 Result: Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 11 ~ The Wavelength-wise Global (WWG) Segmentation Algorithm • Diffusion Bases Dimensionality Reduction • Normalization • Quantization • Color frequency calculation • Finding most frequent colors (peaks) • Finding the nearest peak to each quantized color Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 12 ~ Normalization All pixels in each reduced waveband will be in [0..1] Each coordinate in the reduced representation Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 13 ~ The Wavelength-wise Global (WWG) Segmentation Algorithm • Diffusion Bases Dimensionality Reduction • Normalization • Quantization • Color frequency calculation • Finding most frequent colors (peaks) • Finding the nearest peak to each quantized color Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 14 ~ Quantization • Quantize each pixel coordinate to 𝑚 values where Result: Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 15 ~ The Wavelength-wise Global (WWG) Segmentation Algorithm • Diffusion Bases Dimensionality Reduction • Normalization • Quantization • Color frequency calculation • Finding most frequent colors (peaks) • Finding the nearest peak to each quantized color Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 16 ~ Color frequency calculation • Count the quantized pixels with the same value in the dimension reduced representation – Largest number belongs to the largest segment – In hyper-spectral images it is the most common material in the image – where for every 𝜆 ∈ 1, … , 𝑚 , 𝑔(𝜆) is the number of quantized color vectors that are equal to 𝜆 . Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 17 ~ The Wavelength-wise Global (WWG) Segmentation Algorithm • Diffusion Bases Dimensionality Reduction • Normalization • Quantization • Color frequency calculation • Finding most frequent colors (peaks) • Finding the nearest peak to each quantized color Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 18 ~ Finding most frequent colors (peaks) • In Input • Ou Outp tput • 𝜄 – the number of peaks • Ψ = 𝜍 𝑗 𝑗=1,…,𝜄 𝜃 ∈ ℕ 𝜃 • 𝑔 – the frequency of each color 1 , … , 𝜍 i • 𝜍 𝑗 = 𝜍 i • 𝜊 – neighborhood size of each peak Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 19 ~ The Wavelength-wise Global (WWG) Segmentation Algorithm • Diffusion Bases Dimensionality Reduction • Normalization • Quantization • Color frequency calculation • Finding most frequent colors (peaks) • Finding the nearest peak to each quantized color Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 20 ~ Results 1: Hyper-spectral Microscopy (D=128) Band 50 Band 35 Band 95 𝜽 = 𝟓, 𝜾 = 𝟒, 𝝄 = 𝟒, 𝒎 = 𝟒𝟑 Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 21 ~ Results 2: Remote Sensing (D=100) 𝜽 = 𝟓, 𝜾 = 𝟗, 𝝄 = 𝟖, 𝒎 = 𝟒𝟑 Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 22 ~ Future Work • Finding the parameter values by non- parametric optimization e.g. Nelder-Mead • Find sub-pixel segments (work in progress) Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

~ 23 ~ Thank you  Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017