Image Analysis and Classification in Nanoscale Scanning Tunneling - PowerPoint PPT Presentation

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Image Analysis and Classification in Nanoscale Scanning Tunneling Microscopy Nen Huynh, Tawny Lim, Jonathan Siegel Konstantin Dragomiretskiy, Travis Meyer, Joseph Woodworth Mentors: Dr. Christoph Brune Prof. Andrea Bertozzi California NanoSystems Institute: Dr. Shelley Claridge Miles Silverman Prof. Paul Weiss UCLA REU Final Presentation, August 8, 2012

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Outline 1 Introduction 2 Fourier Transform 3 Preprocessing 4 Segmentation 5 Nonlocal Methods 6 Conclusion

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion What is STM? Scanning Tunneling Microscopy (STM) is a type of Scanning Probe Microscopy. STM generates images based on changes in distance between tip and sample, measured by the current of electrons running from the tip through the sample. Figure: STM Tip

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Scanning Tunneling Microscope Setup Figure: STM Microscope

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion What is our Goal? Use image analysis to extract patterns in data for peptide identification. Figure: Image and Underlying Beta Sheet

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion What types of images are we working with? Figure: Topography and Polarizability Images Subtracting average of image from original can reduce illumination and enhance contrast between light and dark.

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion What types of images are we working with? Figure: Topography and Polarizability Images Subtracting average of image from original can reduce illumination and enhance contrast between light and dark.

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion What types of images are we working with? Figure: Topography and Polarizability Images Subtracting average of image from original can reduce illumination and enhance contrast between light and dark.

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion What types of images are we working with? Figure: Topography and Polarizability Images Subtracting average of image from original can reduce illumination and enhance contrast between light and dark.

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Decomposition First, we must decompose our images into structure and texture. Structure: Piecewise constant approximation to image Texture: Regular periodic patterns TV-min: Ω ( u − f ) 2 dx + λ min u � � Ω |∇ u | dx G-Norm Ω ( f − ( u + v )) 2 dx + λ 1 � � � min u , v Ω |∇ u | + λ 2 Ω � v � G dx Figure: Original Image = Structure + Texture + Noise

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Prior Fourier Transform in Action Prokhorov et al. use the Fourier transform to obtain the orientation and separation between molecular structures on AFM images. [Prokhorov, et al. Langmuir , 2011], [Stabel, et al. J. Phys. Chem., 1995 ]

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Discrete Fourier Transform We advance on the work of Prokhorov et al. by performing the Fourier transform on the texture patch. N − 1 x n · e − 2 π i k � N n X k = n =0 (a) Selected (b) Patch (c) DFT: Top (d) DFT: Side View Patch Close-up View

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Discrete Fourier Transform We can now extract the angles and periods of the patch. Angles Periods 61.6992 5.0119 65.5560 6.1243 60.2551 9.1786 (a) Image (b) DFT: Top 68.1986 13.7415 Patch View 62.2415 3.4465 64.7989 3.9386

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Segmentation Images are divided into regions for further analysis. Segmentation-related techniques: Structure Tensor Principle Component Analysis on Histograms Chan-Vese Model Spectral Clustering

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Structure Tensor The structure tensor gives us orientation information by using the gradient. J ρ = K ρ ∗ � n i =1 ∇ f i ∇ f T Vector-valued structure tensor: i c w = ( λ 1 − λ 2 λ 1 + λ 2 ) 2 Coherency:

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Structure Tensor The structure tensor gives us orientation information by using the gradient. J ρ = K ρ ∗ � n i =1 ∇ f i ∇ f T Vector-valued structure tensor: i c w = ( λ 1 − λ 2 λ 1 + λ 2 ) 2 Coherency:

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Structure Tensor The structure tensor gives us orientation information by using the gradient. J ρ = K ρ ∗ � n i =1 ∇ f i ∇ f T Vector-valued structure tensor: i c w = ( λ 1 − λ 2 λ 1 + λ 2 ) 2 Coherency:

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Structure Tensor The structure tensor shows there are four regions of different orientations in the texture image. (a) Texture (b) Structure Tensor on Texture

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Principle Component Analysis (PCA) on Histogram Transforms Using PCA on the histogram of structure tensor images reduces the significance of the low coherency (gray) areas. Figure: Taking the patch histograms. For each patch, we find the histogram and represent the histogram values as a vector h i .

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion PCA on Histograms Next, apply PCA on the set of histogram vectors to find the direction v 1 with the most variance.

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion PCA on Histograms Vector v 1 allows transformation of structure tensor image. Projecting patch histogram onto v 1 generates scalar value. Figure: Projecting a patch’s histogram onto vector v 1

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion PCA on Histograms The PCA Histogram transform creates a new image better suited for intensity-based segmentation. (a) Image (b) PCA on Structure Tensor Histogram

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Segmentation: Chan-Vese Model Segmentation using the Chan-Vese model: The algorithm divides image into regions by fitting a constant to each of the regions. (a) Image (b) Level Set Function, φ { φ< 0 } ( I ( x ) − c 1 ) 2 dx E ( φ, c 1 , c 2 ) = λ 1 � { φ> 0 } ( I ( x ) − c 2 ) 2 dx + µ Length( { φ =0 } ) � + λ 2

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Chan-Vese (CV) Model The optimal segmentation of an image comes from disjoint sets { φ > 0 } and { φ < 0 } where φ is a level set function minimizing: � � ( I ( x ) − c 1 ) 2 dx + λ 2 ( I ( x ) − c 2 ) 2 dx E ( φ, c 1 , c 2 ) = λ 1 { φ< 0 } { φ> 0 } + µ Length( { φ = 0 } ) and c 1 , c 2 are the fitting constants. (a) (b) (c) Figure: (a-b) Level Set φ . Black line is the zero level set (contour). (c) Image and segmentation contour in red.

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Chan-Vese on PCA (a) Image (b) PCA on Structure (c) Segmentation of Tensor Histogram PCA Image Figure: Example of Chan-Vese on PCA Histogram Transform of image.

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Segmentation: Chan-Vese Model Example for consideration: (a) (b) (c) (d) Figure: (a) Image (b) PCA Hist on texture of image (c) Level Set Function (d) Multiple region segmentation

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Graph-based Methods Segmentation of Graph

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Graph-based Methods Orientation Test Image Segmentation of Graph Spectral Clustering Result

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Non-local Laplacian Continuous Laplacian N ∂ 2 u � △ u := ∂ x 2 i i =1 5-point finite difference approximation to the Laplacian △ u ( x , y ) = u ( x − h , y ) + u ( x , y − h ) − 4 u ( x , y ) + u ( x + h , y ) + u ( x , y + h ) + O ( h 2 ) h 2 Graph Laplacian for a generalized similarity function w (( x 1 , y 1 ) , ( x 2 , y 2 )) � △ w u (( x i , y i )) := ( u (( x j , y j )) − u (( x i , y i ))) · w (( x i , y i ) , ( x j , y j )) ( x j , y j ) ∈ Ω

Introduction Fourier Transform Preprocessing Segmentation Nonlocal Methods Conclusion Metrics Some metrics we used: (a) Image L 2 distance Wasserstein distance (b) L 2 distance (c) Wasserstein distance