Recovering Cam era Sensitivities using Target-based Reflectances Captured under m ultiple LED-I llum inations Philipp Urban, Michael Desch, Kathrin Happel and Dieter Spiehl
Motivation: Spectral Estim ation Channel m Channel 1 Illumination Scene Camera Response Camera ? Can we calculate spectral reflectances from the camera image? Spectral Image
Motivation: Spectral Estim ation If yes, we could render/ reproduce the image for any illuminant
Motivation: Spectral Estim ation If yes, we could estimate pigments used by an artist Y . Zhao 2008
Spectral Estim ation Approaches c 1 c m Signal Processing Training-based Model-based Spatio-Spectral Multipoint spectral Methods Methods Methods Measurement Methods Camera Sensitivities Required
Model-based Methods Use information to calculate spectral reflectance factor from camera responses Additional Information e.g. Distribution of noise,reflectances
Cam era Model ( Linear, Continuous) c 1 d + = X X = c c m [s 1 s m ] T l r Camera Camera Response Illuminant Reflectance Sensitivities (additive noise) Spectral stimulus Required but Geometry factor* usually unknown *Lambertian surface assumption
Cam era Model ( Linear, Discrete) Sensitivity Range Relative pow er\reflectance Geometry Factor Reflectance vector Diagonal matrix with illuminant as diagonal r = [ 50 65 85 80 60 58 55 50 45 40] T i-th sensitivity vector Noise Spectral representation by N- dimensional vector i-th channel response
Monochrom ator-based Training Stim uli Monochromator Sampling the Sensitivities with monochromatic light + Halon (r 1) Sensitivity Sensitivity Sensitivity Very high effective dimension* (~ N ) Very good sensitivity estimation Wavelength Wavelength Wavelength without any additional assumptions *Hardeberg (2002)
Target-based Training Stim uli Low effective dimension of stimuli for typical targets (broadband) Additional assumptions required • Measure m target reflectances (smoothness, non-negativity,…) • Capture target for one illuminant Large influence of assumptions on estimated sensitivities
LED+ Target-based Training Stim uli High effective dimension of stimuli m x k x n equations for recovering s 1 ,…,s n for typical targets (narrowband) Additional assumptions required • Measure m target reflectances (smoothness, non-negativity,…) • Capture target for LED illuminant 1 … Small influence of assumptions • Capture target for LED illuminant k on estimated sensitivities
Problem : Measurem ent Geom etry Geometry factor differs Different measurement geometries: among patches and illuminants • Distance between camera and patches • Angle between camera and patches • Difference between spectrophotometer Non-linear equation system 45°/0° and camera (sensitivities + geometry factors)
Solution: Use Chrom aticities Divide each equation by the sum of channels (geometry factor can be reduced) Rearrange equations + consider noise where is a ( k · m · n) × (n · N) dimensional matrix that depends on Noise, assumed to be uncorrelated zero-mean Gaussian with covariance matrix
W hat Assum ptions are Reasonable? Sensitivities are non-negative 1 invalid Sensitivities are likely to be smooth: Model wavelength-correlation using a Toeplitz-matrix 2 Unlikely, but possible
W hat Assum ptions are Reasonable? Covariance matrix for can be modeled as block-diagonal 3 (i.e. channel sensitivities are uncorrelated) Likelihood model: where Prior distribution: 2 Posterior distribution: 3 where
Reconstruct Relative Sensitivities Solve Constrained Maximum-A-Posteriori (MAP) Problem: Minimize: 1 subject to: 2 3 • The result are sensitivities that maximize the posterior distribution subject to constraints • Last constraint is a normalization – required to avoid zero sensitivities
Experim ental Setup Camera: 1 Custom made two-stage Six channel camera Sinar 54H-based RGB filter wheel with (12 bit, 22MP) camera (IR cutoff removed) yellow + blue filter 2 Target: Color Checker measured using X-Rite’s EyeOne
Experim ental Setup LED viewing Booth: 3 JUST Normlicht LED Color Viewing Light (six different types of LED) 4 Spectroradiometer to measure LED spectral power distribution (SPD) Konica-Minolta CS1000 5 UV/IR cutoff filter added to limit the sensitivity range of the camera to 400 – 700 nm [EyeOne measurements do not cover the real sensitivity range]
Results Effective dimension: 15 Color Checker only: 9 Training Stimuli Experimental Setup (24 CC* x 6 LED = 144) *CC = Color Checker patches
Results Predicted vs. real channel responses for the training stimuli Reconstructed Sensitivities [Errors: mean ~ 0.3%, 95 th < 0.8%, max < 1.35%]* *Error percentage is relative to the max. channel response
Results ( Test Stim uli) Capture the ESSER TE221 Target under 6 LED illuminant [1698 different test stimuli] Predict channel responses using the reconstructed sensitivities Prediction error histogram* Small mean errors < 0.5%* Small 95 th errors < 1.3%* But large max errors ~7.5%* Possible reason for large max error: Invalid Lambertian surface assumption Effective dimension of stimuli is still to small *Error percentage is relative to the max. channel response
Conclusion Camera sensitivities are required for many reflectance estimation methods If unknown they need to be measured/estimated Sensitivity reconstruction methods use assumptions (non-negativity, smoothness) and training stimuli: Different LED + Target Higher effective dimension of stimuli Smaller influence of assumptions Sensitivity reconstruction using Constrained Maximum-A-Posteriori (MAP) Test: 6-channel camera, 6-LED, Color Checker Results (Esser test target): small mean and 95 th -, large max prediction errors
Thanks for your attention Thanks to Maria Fedutina, Tanja Kaulitz, Henri Kröling, Karsten Rettig, Manfred Jakobi for constructing, manufacturing and assembling the filter wheel and to the Deutsche Forschungsgemeinschaft for the sponsorship of the project Philipp Urban, Michael Desch, Kathrin Happel and Dieter Spiehl http: / / www.idd.tu-darmstadt.de/ color
Recommend
More recommend
