Resolution Limits in Digital Photography Luiz Velho IMPA 1 - PDF document

Resolution Limits in Digital Photography Luiz Velho IMPA 1 Motivation • Longstanding Interest in Photography - Photo Gallery • Current Work on Panoramic Images - Visorama - N-Pan Project • Recent Collaboration - Diego Nehab - Pedro Sander 2 Monday, August 23, 2010

RioHK - Stay tuned... 3 Breaking the Record • Corcovado 67 GP • Sugar Loaf (153 GP) ✴ work in progress 4 Monday, August 23, 2010

Concepts • Resolution - Representation (wikipedia.org) • Frequency # occurencies / unit - Contents • Scale ‣ Waves 5 MTF • Modulation Transfer Function (normankoren.com) 6 Monday, August 23, 2010

Understanding MFT • Modulation - Contrast Attenuation • Spatial Frequencies - Patterns (normankoren.com) (imatest.com) 7 Measuring a System • MTF Curve Object (wikipedia.org) 8 Monday, August 23, 2010

Continuous Image • Frequency Content… ‣ Fourier Theory 9 Discrete Image image frequency contents resolution sampling reconstruction ‣ Shannon - Whittaker Theorem 10 Monday, August 23, 2010

Ideal Optical Image Real Image Projected Image Perfect Lens • Diffraction Limit!!! ‣ Abbe Theorem wavelength numerical aperture resolution lens index of refraction lens aperture angle 11 Diffraction (Circular Aperture) • Large Aperture (cambridgeincolor.com) (agunterphotography.com) • Small Aperture sensor (agunterphotography.com) (cambridgeincolor.com) 12 Monday, August 23, 2010

Resolvable Detail • Airy Disk (Rayleigh Criterium) (cambridgeincolor.com) (agunterphotography.com) 13 Diffraction Limited System • Resolution matches Theoretical Limit - smallest feature ~ size of Airy disk ‣ More on that later... (cambridgeincolor.com) 14 Monday, August 23, 2010

Gigapixel Image Lens Sensor • OBS: Only a single image ... 15 The Gigapixl Project • Big Camera ;-) ‣ Large Format - Film (45 kg) • Portrait of America (gigapixl.org) 16 Monday, August 23, 2010

Gigapixel Mosaic Stitch Lens Sensor Motor • OBS: Many individual images …. 17 Gigapan Project • Gigapan Imager - Robotic Head + Digital Camera • gigapan.org - Worldwide Community (gigapan.org) 18 Monday, August 23, 2010

Our Problem 19 The Real Scenario • Complete Pipeline scene pano atmosphere lens motor sensor stitch • haze • blur • vignetting • noise • ghosting • flute • aberration • quantization weather f-stop shutter iso blending (from Joshi-Cohen 2010, edited) 20 Monday, August 23, 2010

Back to MTF • System MTF - Lens + Sensor + etc.. + (Human Eye) (normankoren.com) 21 Atmospheric Limitations • Turbulence ( Heat ) • Aerosols ( Distance ) (gigapixl.org) 22 Monday, August 23, 2010

Sensor Formats (cambridgeincolor.com) • FX - Full Frame • DX - Digital Camera 23 Lens MTF Parameters • Sagital / Meridional Patterns (cambridgeincolor.com) • Distance from Image Center (cambridgeincolor.com) 24 Monday, August 23, 2010

Lens Performance • the good and the bad ;-) good lens bad lens (cambridgeincolor.com) ‣ MTF-50 - sharpness perception 25 Lens MTF Chart • Manufacturer's Data (cambridgeincolor.com) 26 Monday, August 23, 2010

Aperture x Pixel Size • Optimal Aperture (cambridgeincolor.com) 27 The Sweet Spot MTF-50 (cambridgeincolor.com) • Why ? - Large Apertures (light aberrations) ** - Small Apertures (diffraction effect) 28 Monday, August 23, 2010

Sensor Resolution • Bayer Pattern • Demosaicing original sensor • Moire Artifacts (wikipedia.org) 29 Lens Vignetting • Radial Falloff • 4 types: - Natural - Pixel - Optical - Mechanical (from d'Angelo, 2007) 30 Monday, August 23, 2010

Natural • Due to: - Area of exit pupil: cos(b) - Change in effective area: cos(b) - Larger distance to corner: cos 2 (b) • Law: cos 4 (b) (from d'Angelo, 2007) 31 Pixel • Angular Sensitivity of Sensor - Finite depth of photon wells (dpreview.com) 32 Monday, August 23, 2010

Optical • Entrance Pupil shaded by lens barrel f 1.4 f 5.6 • Depends on Aperture (www.vanwalree.com) 33 Mechanical • Light blocked by lens hood (www.vanwalree.com) 34 Monday, August 23, 2010

Vignetting Correction • Venice panorama (from d'Angelo, 2007) 35 Motion Blur • Object Movement • Camera Motion ‣ De-blurring... 36 Monday, August 23, 2010

The Camera Sensor • Noise • Quantization (from Fredo Durand) 37 Sensor Noise • ISO / Sensor Size Canon EOS-1D Mark IV ISO 6,400 ISO 25,600 ISO 102,400 • Long Exposure • Other Factors 2 seconds ‣ Noise Reduction... (dpreview.com) 38 Monday, August 23, 2010

Sensor Quantization • Response Curve • Analog - Digital (8 to14 bits) (dpreview.com) 39 Camera Processing • Photosites (photons to electrons) • ISO (amplification) • A/D (quantization) RAW • Demosaicing • [denoising] • [color conversion] • Response Curve • Gamma Encoding • JPEG 40 Monday, August 23, 2010

Stitching • Alignment - Projective Transformation • Blending - Linear - Multiband - GraphCut - Poisson 41 The Future • Terapixel Imaging Systems • Metalens • Super-resolution 42 Monday, August 23, 2010

Terapixel Imaging Invited Talk • ICCP 2010 43 Metalens • Breaking the Diffraction Limit (technology-review) capture details of 1/80 the wavelength of light 44 Monday, August 23, 2010

Super Resolution 45 Slide Credits • Fredo Durand • Neel Joshi and Michael Cohen • Pablo d'Angelo - http://dpreview.com - http://www.wikipedia.org/ - http://www.cambridgeincolour.com - http://www.aguntherphotography.com - http://www.normankoren.com/ 46 Monday, August 23, 2010