Sharpness Search Algorithms for Automatic Focusing in the Scanning - PowerPoint PPT Presentation

Sharpness Search Algorithms for Automatic Focusing in the Scanning Electron Microscope C.F. Batten, D.M. Holburn, B.C. Breton, N.H.M. Caldwell Scientific Imaging Group Cambridge University Scanning May 7 th , 2001

Motivation • There is a growing need for instrument automation – New applications have led to an increase in the number of novice SEM operators – Remote microscopy requires simple commands which perform more work • Focusing is an ideal candidate for automation – Simplifies a common and tedious operation – Helps make remote microscopy practical May 7th, 2001 Scanning 2

Previous Work • Scanning Electron Microscopy – Software solution using image gradient [ Tee79 ] – Hardware solution using image covariance [ Erasmus82 ] – Software solution using frequency domain [ Ong98, Ogasawara99 ] – Use of a general imaging model to predict best focus [ Nicolls95 ] • Optical Microscopy – Survey of sharpness measures [ Groen85, Firestone91 ] – Use of a Fibonacci search to find the best focus [ Yeo93 ] May 7th, 2001 Scanning 3

Our Approach • Traditional autofocusing approaches – Try to integrate additional functionality such as astigmatism correction or topological mapping – Use a fixed stepsize or iterative search and avoid more sophisticated search algorithms due to low SNR and hysteresis concerns • Our approach – Make a dedicated autofocusing search algorithm Increase the Use a more Decrease the SNR for sophisticated number of required each image search algorithm image captures May 7th, 2001 Scanning 4

Outline • Sharpness Measures – Gradient measure – Frequency domain measure – Autocorrelation measures – Variance measure • Sharpness Search Algorithms – Fixed stepsize search – Fixed stepsize search with interpolation – Iterative search – Variable stepsize search – Fibonacci search • Conclusions May 7th, 2001 Scanning 5

Evaluating Sharpness Measures Strictly Unimodal Property 60 Forward Sweep 1 Forward Sweep 2 Forward Sweep 3 Sharpness measure should 50 Variance Sharpness Measure have one peak at the best 40 focus and strictly decrease away from this maximum 30 20 10 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 Focus Step (0.01mm) May 7th, 2001 Scanning 6

Gradient Measure • Sum of the difference between every n th pixel in both the X and Y directions • As image comes into focus, edges become sharper increasing the image gradient • Sharpness Measure Properties – Relatively easy to calculate (one of the first sharpness measures) – Very susceptible to noise – The parameter n acts a low-pass filter in the spatial domain • ( n = 1) Traditional image gradient • ( n = 2) Brenner method • ( n > 2) As long as n < feature size, can increase noise robustness May 7th, 2001 Scanning 7

Frequency Domain Measure • Perform Fourier transform and then sum the frequency components below threshold frequency ( Ω ) • As image comes into focus, edges become sharper which increases the magnitude of medium frequency components • Sharpness Measure Properties – Allows easy integration of astigmatism correction – Fourier transform in software is computationally expensive – The parameter Ω acts as a low-pass filter in frequency domain • Varying Ω produces similar results as varying n • For this work, Ω chosen to be 50 May 7th, 2001 Scanning 8

Auto-correlation Measures • Auto-correlation function is the image convolved with itself and indicates how well neighboring pixels are correlated • Tested two measures using the image auto-correlation – ACFdiff Height of the central ACF peak – ACFsum Area under the central ACF peak • Focused images contain small highly correlated regions that result in a tall sharp central ACF peak • Sharpness Measure Properties – Can calculate ACF efficiently in the frequency domain – Do not need to calculate entire ACF for ACFdiff measure – Correlated noise due to limited bandwidth distortion made using the ACF more difficult May 7th, 2001 Scanning 9

Variance Measure • Sum the square of the difference between each pixel and the mean image intensity • Focused images have greater intensity variation then blurred defocused images • Sharpness Measure Properties – Simple and efficient implementation – Very robust to noise – Strong adherence to the strict unimodality property For these reasons the variance measure was selected as the primary sharpness measure for this work May 7th, 2001 Scanning 10

Comparison of Sharpness Measures (a) f grad (b) f freq (c) f ACFdiff 1 1 1 0.8 0.5 0.8 0.6 0.6 0 0.4 0.2 0.4 −0.5 0 10 20 30 0 10 20 30 0 10 20 30 (d) f ACFsum (e) f var 1 1 Pix Avg: 1 FAvg: 3 Pix Avg: 1 FAvg: 5 0.8 0.8 Pix Avg: 2 FAvg: 3 Pix Avg: 2 FAvg: 10 0.6 0.6 Pix Avg: 4 FAvg: 15 Pix Avg: 8 FAvg: 20 0.4 0.4 Pix Avg: 16 FAvg: 25 0.2 0.2 0 0 0 10 20 30 0 10 20 30 Sharpness Measures at Various Noise Levels May 7th, 2001 Scanning 11

Sharpness Search Algorithms • Investigated five sharpness search algorithms – Fixed stepsize search – Fixed stepsize search with interpolation – Iterative search – Variable stepsize search – Fibonacci search • Notation – l Search interval – α Desired accuracy (How close to optimum is acceptable?) – N Number of required image captures • Goal is to find a search algorithm which minimizes N but still achieves the desired accuracy α May 7th, 2001 Scanning 12

Fixed Stepsize Search • Single sweep over search interval with stepsize = 2 α • Theoretical N given by 1100   2 l = + N 1   α 1000   900 Variance Sharpness Measure • Peak finding reduces N 800 • Developed a novel method 700 to adjust for hysteresis 600 effects based on relative 500 sharpness when returning Variance Moving Average 400 to best focus 4 4.5 5 5.5 6 6.5 Focal Length (mm) Fixed Stepsize Search with Peak Finding May 7th, 2001 Scanning 13

Fixed Stepsize with Interpolation • Interpolation can help reduce the number of image captures while maintaining the desired accuracy • Quadratic and linear interpolation do not perform well on typical variance curves • A New Interpolation Approach – Erasmus and Smith provide a derivation for image variance as a function of defocus [ Erasmus82 ] – Use non-linear regression to curve fit the derived function with the collected data – This allows us to significantly reduce the required number of image captures, but is computationally expensive May 7th, 2001 Scanning 14

Fixed Stepsize with Interpolation (a) Gold on Carbon − 22,200x (b) Gold on Carbon − 75,500x (c) IC Tracks − 3,400x 1.4 1.2 1.2 1.2 1 Normalized Variance Normalized Variance Normalized Variance 1 1 0.8 0.8 0.8 0.6 0.6 0.4 0.4 0.6 0.2 0.2 8.5 8.61 8.72 8.83 8.94 8.56 8.6 8.65 8.69 8.74 8 8.66 9.33 10 10.66 Focal Length (mm) Focal Length (mm) Focal Length (mm) (d) IC Tracks − 3,400x (e) IC Tracks − 9,900x (f) Sublimated Titanium − 1,500x 1.2 1.1 1.4 1.1 1 1.2 Normalized Variance Normalized Variance Normalized Variance 1 0.9 1 0.9 0.8 0.8 0.8 0.7 0.6 0.7 0.6 0.6 0.5 0.4 6 7.55 9.11 10.66 12.22 8.69 8.98 9.27 9.56 9.85 7 8.33 9.66 11 12.33 Focal Length (mm) Focal Length (mm) Focal Length (mm) Derived Variance Function Fitted to Data from Various Specimens May 7th, 2001 Scanning 15

Iterative Search • Several sweeps with gradually smaller stepsizes and search intervals 600 Iteration 1 Iteration 2 • Theoretical N given by Iteration 3 Iteration 4 550   log( α / l ) Variance Sharpness Measure = 500 N η   log( 2 /( η - 1))   450 where η is the number 400 of image captures per iteration 350 300 6 6.5 7 7.5 8 8.5 9 9.5 Focal Length (mm) Online Iterative Focus Sweep ( η =8) May 7th, 2001 Scanning 16

Variable Stepsize Search • Reduce the stepsize as the sharpness increases • A common technique in other maximum search problems, but not used in SEM autofocusing due to low image SNR • We adapt the algorithm as follows – Reduce stepsize based on moving average of variance – Set 2 α as a lower bound on the stepsize – Use peak finding – Perform final fixed stepsize search if stepsize is greater than 2 α once the peak is found • Actual number of image captures varies based on initial stepsize and specific variance curve May 7th, 2001 Scanning 17

Variable Stepsize Search (a) Sharpness During Variable Stepsize Sweep 1100 Variance Sharpness Measure Variance 1000 Moving Average 900 800 700 600 500 400 4 4.5 5 5.5 6 6.5 Focal Length (mm) (b) Stepsize During Variable Stepsize Search 0.35 0.3 Stepsize (mm) 0.25 0.2 0.15 0.1 4 4.5 5 5.5 6 6.5 Focal Length (mm) Example of Variable Stepsize Search May 7th, 2001 Scanning 18

Fibonacci Search • An iterative search where η = 1 • Use previous measurements and one new measurement to narrow search interval • To avoid adverse hysteresis 1100 • 7 • effects, must set instrument 6 1000 • 4 to small focal length before 900 • 2 Sharpness Measure each image capture 800 • 5 (~200ms) 700 • 3 • 1 600 500 a b 400 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Focal Length (arbitrary units) Fibonacci Search Image Captures May 7th, 2001 Scanning 19