Automatic 3D Mapping for Infrared Image Analysis i i r r f f m - PowerPoint PPT Presentation

Automatic 3D Mapping for Infrared Image Analysis i i r r f f m m c c a a d d a a r r a a c c h h e e V. Martin, V. Gervaise, V. Moncada, M.H. Aumeunier, M. Firdaouss, J.M. Travere (CEA) S. Devaux (IPP), G. Arnoux (CCFE) and JET-EFDA contributors Workshop on Fusion Data Processing Validation and Analysis, ENEA Frascati, 26-28 March 2012 V. Martin et al. 1 (19) WFDPVA, ENEA Frascati 28/03/12

3D IR Scene Calibration JET #81313 KL7 (images in DL) Bulk Be W coated CFC Bulk Be Bulk Be Be coated linconel W coated CFC Bulk W V. Martin et al. 2 (19) WFDPVA, ENEA Frascati 28/03/12

3D IR Scene Calibration JET #81313 KL7 • Issue: a complex thermal scene (images in DL) 1. Wide angle views with high geometrical effects: depth of field and curvature 2. Many metallic materials (Be, W) with different and changing optical (reflectance) and thermal (emissivity) properties Bulk Be W coated CFC Bulk Be Bulk Be Be coated linconel W coated CFC Bulk W V. Martin et al. 3 (19) WFDPVA, ENEA Frascati 28/03/12

3D IR Scene Calibration JET #81313 KL7 • Issue: a complex thermal scene (images in DL) 1. Wide angle views with high geometrical effects: depth of field and curvature 2. Many metallic materials (Be, W) with different and changing optical (reflectance) and thermal (emissivity) properties • Objective: Match each pixel with the 3D scene model of in-vessel components for: Bulk 1. getting the real geometry of the viewed objects Be W coated 2. reliable linking between viewed objects and their CFC related properties Bulk Be Bulk Be Be coated linconel W coated CFC Bulk W V. Martin et al. 4 (19) WFDPVA, ENEA Frascati 28/03/12

3D IR Scene Calibration JET #81313 KL7 • Issue: a complex thermal scene (images in DL) 1. Wide angle views with high geometrical effects: depth of field and curvature 2. Many metallic materials (Be, W) with different and changing optical (reflectance) and thermal (emissivity) properties • Objective: Match each pixel with the 3D scene model of in-vessel components for: Bulk 1. getting the real geometry of the viewed objects Be W coated 2. reliable linking between viewed objects and their CFC related properties Bulk Be • Applications Bulk Be Be coated 1. Image processing (event characterization) linconel W coated 2. IR data calibration: T surf = f(material emissivity) CFC Bulk W V. Martin et al. 5 (19) WFDPVA, ENEA Frascati 28/03/12

Methodology • Calibration chain NUC Reference image 2D/3D scene Knowledge base Dead pixel Map models of the thermal scene 2D/3D Scene Registered & Image Image Image Camera Model Calibrated Correction Stabilization Processing Image Mapping V. Martin et al. 6 (19) WFDPVA, ENEA Frascati 28/03/12

Illustration of Motion in Images • Camera vibrations lead to misalignments of ROIs (PFC RT protection) = false alarms or worth missed alarms • Image stabilization is a mandatory step for heat flux deposit analysis based on T surf (t)-T surf (t-1) estimations V. Martin et al. 7 (19) WFDPVA, ENEA Frascati 28/03/12

Image Stabilization • Important factors for method selection • Deformation type: planar (homothety), non-planar • Target application: real-time processing, off-line analysis • Data quality and variability: noise level, pixel intensity changes, image entropy • Required precision level: pixel, sub-pixel • Applications in tokamaks (non-exhaustive list) Motion amplitude Target application Precision Difficulty required JET KL7 5-10 pixels Hot spot detection PFC pixel low image entropy protection wide-angle (camera vibrations) JET KL7 up to 15 pixels Physics analysis (e.g. pixel pixel intensity (disruptions) heat load during changes windowed disruptions…) JET KL9 <1 pixel Physics analysis (power sub-pixel low resolution, slow deposit influx) motion, aliasing divertor tiles (sensor affected by magnetic fields) V. Martin et al. 8 (19) WFDPVA, ENEA Frascati 28/03/12

Image Stabilization See Zitova’s survey, Image and Vision Computing, vol. 21(2003), pp. 977 -1000 V. Martin et al. 9 (19) WFDPVA, ENEA Frascati 28/03/12

Image Stabilization • Classical Methodology See Zitova’s survey, Image and Vision Computing, vol. 21(2003), pp. 977 -1000 V. Martin et al. 10 (19) WFDPVA, ENEA Frascati 28/03/12

Image Stabilization • Classical Methodology 1. Feature Detection • Local descriptors: Harris corners, MSER, codebooks, Gabor wavelets (see Craciunescu talk), SIFT, SURF, FAST… • Global descriptors: Tsallis entropy (see Murari talk), edge detectors… • Fourier analysis: spectral magnitude & phase, pixel gradients, log- polar mapping… See Zitova’s survey, Image and Vision Computing, vol. 21(2003), pp. 977 -1000 V. Martin et al. 11 (19) WFDPVA, ENEA Frascati 28/03/12

Image Stabilization • Classical Methodology 1. Feature Detection • Local descriptors: Harris corners, MSER, codebooks, Gabor wavelets (see Craciunescu talk), SIFT, SURF, FAST… • Global descriptors: Tsallis entropy (see Murari talk), edge detectors… • Fourier analysis: spectral magnitude & phase, pixel gradients, log- polar mapping… 2. Feature Matching • Spatial cross-correlation techniques: normalized cross- correlation, Hausdorff distance… • Fourier domain: normalized cross-spectrum and its extensions See Zitova’s survey, Image and Vision Computing, vol. 21(2003), pp. 977 -1000 V. Martin et al. 12 (19) WFDPVA, ENEA Frascati 28/03/12

Image Stabilization • Classical Methodology 1. Feature Detection • Local descriptors: Harris corners, MSER, codebooks, Gabor wavelets (see Craciunescu talk), SIFT, SURF, FAST… • Global descriptors: Tsallis entropy (see Murari talk), edge detectors… • Fourier analysis: spectral magnitude & phase, pixel gradients, log- polar mapping… 2. Feature Matching • Spatial cross-correlation techniques: normalized cross- correlation, Hausdorff distance… • Fourier domain: normalized cross-spectrum and its extensions 3. Transform Model Estimation • Shape preserving mapping (rotation, translation and scaling only) • Elastic mapping: warping techniques… See Zitova’s survey, Image and Vision Computing, vol. 21(2003), pp. 977 -1000 V. Martin et al. 13 (19) WFDPVA, ENEA Frascati 28/03/12

Image Stabilization • Classical Methodology 1. Feature Detection • Local descriptors: Harris corners, MSER, codebooks, Gabor wavelets (see Craciunescu talk), SIFT, SURF, FAST… • Global descriptors: Tsallis entropy (see Murari talk), edge detectors… • Fourier analysis: spectral magnitude & phase, pixel gradients, log- polar mapping… 2. Feature Matching • Spatial cross-correlation techniques: normalized cross- correlation, Hausdorff distance… • Fourier domain: normalized cross-spectrum and its extensions 3. Transform Model Estimation • Shape preserving mapping (rotation, translation and scaling only) • Elastic mapping: warping techniques… 4. Image transformation • 2D Interpolation: nearest neighboor, bilinear, bicubic… See Zitova’s survey, Image and Vision Computing, vol. 21(2003), pp. 977 -1000 V. Martin et al. 14 (19) WFDPVA, ENEA Frascati 28/03/12

Proposed Algorithm 1. Masked FFT-based image registration [1]  Deterministic computing time  Accelerating hardware compatible algorithm (e.g. FFT on GPU) → real time applications  Local analysis with dynamic intensity-based pixel masking (e.g. mask the divertor bright region) 2. with sub-pixel precision [2]  Slow drift compensation 3. and dynamic update of the reference image  Robust to image intensity changes (context awareness)  Evaluation of registration quality over time [1] D. Padfield, IEEE CVPR’10, pp. 2918 -2925, 2010 [2] M. Guizar-Sicairos et al., Opt. Lett., vol. 33, no. 2, pp. 156-158, 2008 V. Martin et al. 15 (19) WFDPVA, ENEA Frascati 28/03/12

Principle of Fourier-based Correlation V. Martin et al. 16 (19) WFDPVA, ENEA Frascati 28/03/12

Principle of Fourier-based Correlation • Let I ref a reference image, I t an image at time t and DFT the Discrete 2D Fourier transform such as I t ( x , y ) = I ref ( x-x 0 , y-y 0 ) F DFT ( I ) 1 ref F DFT ( I ) 2 t F (.,.) F (.,.) 1 2 F (.,.) NCC F (.,.) F (.,.) 1 2 I ref I t - 1 ( ) NCC DFT F NCC V. Martin et al. 17 (19) WFDPVA, ENEA Frascati 28/03/12