Conceiving a Fast and Practical Multispectral Stereo System Raju Shrestha Supervisors: Jon Yngve Hardeberg, GUC Alamin Mansouri, University of Burgundy
Two I maging Technologies � A Multispectral system captures image data at specific wavelengths across electromagnetic spectrum � A Stereo system records 3D visual information from two different views (like our HVS)
Multispectral I mage Acquisition � Existing state-of-the art multispectral imaging systems are slow, complex and costly
Goal � Fast and Practical Multispectral System � Using Two Digital Cameras � At the same time, capable of acquiring 3D stereo images
Main I dea � Use a pair of digital cameras or a stereo camera; in stereoscopic configurations � Best pair of filters among a number readily available filters � Resulting a Six Channel Multispectral- Stereo System
Six-Channel Multispectral system
Multispectral Camera Model � Camera sensitivities: (s 1 , s 2 ), Illumination: l, Filter transmittances: (t 1 , t 2 ), and Surface reflectance: r � Model:
Estimating Spectral Reflectance � Problem of estimating Nx1 vector R from Kx1 vector of camera response, C [N>K] => Infinite possible solutions. � Linear Model: R’ = QC From training data: Q = R train C + [ regularize ] Based on spectral (RMSE) &colorimetric Experimental ( ∆ E* ab ) evaluation metrics Simulation � Investigated with popular methods: Maloney and Wandell, Imai and Berns, Least Square Wiener, Polynomial and NN � MCC240 - Training; MCC24 - Test targets
Measurements � SPD of the light source & spectral reflectance of color charts - Spectro-radiometer � Fujifilm3D camera has been characterized using Benthem TMc300 monochromator
Simulation Approach � Carried out with several real and imaginary cameras � Nikon D70, Canon 20D, Fujifilm 3D � 265 filters from Omega � Simulated random Gaussian noise (n shot ), and quantization noise (n quant ), - realistic � Selection of Filters: Exhaustive Search that gives minimum estimation error (RMSE, ∆ E* ab ) • Additional constraints: omit <40% max. transm. and <10%of max. in a channel
A paper has been accepted in EUSIPCO2010, Denmark
Experimental Approach � Fujifilm FinePix REAL 3D W1 camera � A multiband filter: Schott BG-36 � Macbeth color charts (MCC240,MCC24) � Camera Settings :
I mage Capture and Corrections � Capture 10 images of each color charts � Linearize with Gamma values computed from central gray patches of MCC240 (C=Y γ ) :1/(1.35,1.37,1.38) � Perform DC and Non-uniformity corrections (10 images)
Stereo Analysis � Qualitative: Observation of 3D images � Quantitative: From Disparity Maps Observations show clear 3D Images with slightly reduced brightness
Stereo Camera Model A scene view is formed by projecting 3D points into the image plane using a perspective transformation: ' = [ | ] sp A R t P bf = Z − x x r l (X,Y,Z) – A 3D point in world coordinates (u,v) – projection point A – Camera matrix (fx, fy) – focal lengths (cx,cy) – principal point [R|t] – Matrix of extrinsic parameters k1, k2, k3 – Radial distortion coeffs. p1, p2 – Tangential ,, ,,
Camera & Stereo Calibration � Using 20 chess board images taken in different orientations � Camera calibration outputs intrinsic parameters (A & dist. coeffs.) � Stereo calibration computes extrinsic parameters ([R|t])
Disparity Maps � Compute rectification transforms – to undistort and simplify correspondence problem (epipolar geometry=>one line search) � Most commonly used Block Matching (BM) algorithm used � Compare disparity maps from our multispectral system with pure stereo
Conclusion and Future Perspectives � Both simulation and experiments show promising results => Proposed system well function as a fast and practical multispectral-stereo system � Choice of filters is important for accuracy � Slight reduction in the brightness of the 3D images and disparity info, However, quality 3D images are still viewable � Relating 3D info with the physical property would be an interesting further work
Thank You!
Comments and Suggestions? Questions,
Recommend
More recommend
