image processing framework
play

Image Processing Framework FELIX HEIDE 1,2 MARKUS STEINBERGER 3 - PowerPoint PPT Presentation

Mar 29, 2024 β€’478 likes β€’1.33k views

FlexISP: A Flexible Camera Image Processing Framework FELIX HEIDE 1,2 MARKUS STEINBERGER 3 YUN-TA TSAI 1 NASA ROUF 1,2 DAWID PAJK 1 DIKPAL REDDY 1 ORAZIO GALLO 1 JING LIU 4 WOLFGANG HEIDRICH 5 KAREN EGIAZARIAN 1,6 JAN KAUTZ 1 KARI PULLI 1 1

  1. FlexISP: A Flexible Camera Image Processing Framework FELIX HEIDE 1,2 MARKUS STEINBERGER 3 YUN-TA TSAI 1 NASA ROUF 1,2 DAWID PAJΔ„K 1 DIKPAL REDDY 1 ORAZIO GALLO 1 JING LIU 4 WOLFGANG HEIDRICH 5 KAREN EGIAZARIAN 1,6 JAN KAUTZ 1 KARI PULLI 1 1 NVIDIA 2 UBC 3 TU GRAZ 4 UCSC 5 KAUST 6 TUT

  2. CAMERA IMAGE PROCESSING

  4. Image Signal Processor (ISP) 4

  5. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  6. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  7. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  8. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  9. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  10. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  11. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  12. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  13. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  14. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Denoise Demosaic Metering Correction Tone Image Enhancing Mapping

  15. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Denoise Demosaic Metering Correction Tone Image Enhancing Mapping

  16. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Denoise Demosaic Metering Correction Tone Image Enhancing Mapping

  17. NOVEL FEATURES Long Exposure Short Exposure Long Exposure Short Exposure

  18. NOVEL FEATURES Single Exposure Interleaved Exposure

  19. NOVEL FEATURES

  21. Depth Refocusing

  24. Kepler GPU

  25. OBJECTIVES NO PIPELINE FLEXIBILITY PERFORMANCE

  26. PROPOSED SOLUTION FLEXISP Bayer Burst iHDR Others …

  27. IMAGE FORMATION x B D z

  28. IMAGE FORMATION z = DBx x B D z

  29. IMAGE FORMATION z = A x B D

  30. LINEAR SYSTEM 2 π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2

  31. LINEAR SYSTEM 2 π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 errors w.r.t. observation A x

  32. LINEAR SYSTEM 2 π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 errors w.r.t. observation A x

  33. LINEAR SYSTEM 2 + Ξ»(𝑦) π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 regularization

  34. TOTAL VARIATION (Rudin et al. β€˜92)

  35. CROSS-CHANNEL (Heide et al. β€˜13)

  36. COLLABORATIVE DENOISING

  37. COLLABORATIVE DENOISING

  38. LINEAR SYSTEM 2 + Ξ»(𝑦) π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 + + Total variation Cross-channel Collaborative denoising

  39. IMPORTANCE OF PRIORS

  40. MORE ON PRIORS… TV [Rudin et al. β€˜92] EPLL [Zoran and Weiss β€˜11] Curvelets [ CandΓ¨s and Donoho β€˜99 ]

  41. MORE ON PRIORS…

  42. LINEAR SYSTEM 2 + Ξ»(𝑦) π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 Convex Optimization (Mostly)

  43. CONVERGENCE

  44. RESULTS

  45. CAMERA SYSTEMS Bayer Burst iHDR

  46. CAMERA SYSTEMS Bayer Burst iHDR

  47. CAMERA SYSTEMS Bayer Burst iHDR A

  54. CAMERA SYSTEMS Bayer Burst iHDR

  55. CAMERA SYSTEMS Bayer Burst iHDR

  56. CAMERA SYSTEMS Bayer Burst iHDR A

  64. CAMERA SYSTEMS Bayer Burst iHDR

  65. CAMERA SYSTEMS Bayer Burst iHDR

  66. CAMERA SYSTEMS Bayer Burst iHDR A

  73. PERFORMANCE GTX Titan Demosaic (5MP) 2.13 iHDR (13MP) 7.45 Image burst (0.4MPx16) 0.82 TIME IN SECONDS

  74. PERFORMANCE Tegra K1 Demosaic (5MP) 40.5 iHDR (13MP) 174.4 Image burst (0.4MPx16) 16.7 TIME IN SECONDS

  75. MORE ON PRIORS…

  76. PERFORMANCE (BM3D) Tegra K1 Demosaic (5MP) 40.5 iHDR (13MP) 174.4 Image burst (0.4MPx16) 16.7 TIME IN SECONDS

  77. PERFORMANCE (AVERAGING NLM) Tegra K1 Demosaic (5MP) 6.7 iHDR (13MP) 33.2 Image burst (0.4MPx16) 6.6 TIME IN SECONDS

  80. CONCLUSION

  81. Bayer Burst iHDR Color Camera Array + others such as: deconvolution, super-resolution, JPEG compression and de-blocking.

  82. Bayer Burst AOHDR Color Camera Array A FLEXISP

  83. THANK YOU!

Recommend

BBM 413 Today Fundamentals of What is image processing? Image Processing What does it

BBM 413 Today Fundamentals of What is image processing? Image Processing What does it

BBM 413 Today Fundamentals of What is image processing? Image Processing What does it mean, to see? Vision as a computational problem Sample image processing problems Erkut Erdem Dept. of Computer Engineering Hacettepe

399 views β€’ 13 slides
Introduction: What is Image Processing? CS 4640: Image Processing Basics January 10, 2012 What

Introduction: What is Image Processing? CS 4640: Image Processing Basics January 10, 2012 What

Introduction: What is Image Processing? CS 4640: Image Processing Basics January 10, 2012 What is Image Processing? Definition Image processing is the study of any algorithm that takes an image as input and returns an image as output.

530 views β€’ 16 slides
PLT Project SIP(Simplified Image Processing) A Language for image processing Why SIP ??

PLT Project SIP(Simplified Image Processing) A Language for image processing Why SIP ??

PLT Project SIP(Simplified Image Processing) A Language for image processing Why SIP ?? Effectively an image processing language. Concept can be extended for videos Features included to make operations on images short and effective

733 views β€’ 12 slides
Color image processing The use of color in image processing is primarily motivated by two Image

Color image processing The use of color in image processing is primarily motivated by two Image

Color image processing The use of color in image processing is primarily motivated by two Image Analysis factors: Humans can perceive thousands of colors but only about two Color processing dozen shades of gray. Color is a powerful descriptor

1.05k views β€’ 7 slides
BACKGROUND JOB PROCESSING DO'S AND DON'TS BACKGROUND JOB PROCESSING - DO'S AND DON'TS IMAGE

BACKGROUND JOB PROCESSING DO'S AND DON'TS BACKGROUND JOB PROCESSING - DO'S AND DON'TS IMAGE

CHUCK REEVES @MANCHUCK BACKGROUND JOB PROCESSING DO'S AND DON'TS BACKGROUND JOB PROCESSING - DO'S AND DON'TS IMAGE PROCESSOR UPLOAD IMAGE PROCESS IMAGE ENTERS QUEUE ZendCon 2016 BACKGROUND JOB PROCESSING - DO'S AND DON'TS THE PROBLEM

817 views β€’ 35 slides
Yayi A generic framework for morphological image processing IPOL Raffi Enficiaud June 2012

Yayi A generic framework for morphological image processing IPOL Raffi Enficiaud June 2012

Yayi A generic framework for morphological image processing IPOL Raffi Enficiaud June 2012 (Ex.) CMM - Mines Paris Forewords Yayi - A generic framework for morphological image processing http ://raffi.enficiaud.free.fr Recent library (1st

898 views β€’ 41 slides
. Inverse problems in signal and image processing and Bayesian inference framework: from basic to

. Inverse problems in signal and image processing and Bayesian inference framework: from basic to

. Inverse problems in signal and image processing and Bayesian inference framework: from basic to advanced Bayesian computation Ali Mohammad-Djafari Laboratoire des Signaux et Syst` emes (L2S) UMR8506 CNRS-CentraleSup elec-UNIV PARIS SUD

990 views β€’ 77 slides
Digital Image Processing (CS/ECE 545) Lecture 1: Introduction to Image Processing and ImageJ Prof

Digital Image Processing (CS/ECE 545) Lecture 1: Introduction to Image Processing and ImageJ Prof

Digital Image Processing (CS/ECE 545) Lecture 1: Introduction to Image Processing and ImageJ Prof Emmanuel Agu Computer Science Dept. Worcester Polytechnic Institute (WPI) What is an Image? 2 dimensional matrix of Intensity (gray or color)

884 views β€’ 77 slides
Deep Learning in Image Processing Topics: Image Filtering 101 CNNs 101 Image

Deep Learning in Image Processing Topics: Image Filtering 101 CNNs 101 Image

Deep Learning in Image Processing Topics: Image Filtering 101 CNNs 101 Image Processing Pipelines Frank Dellaert CS 4476 Computer Vision Many slides from Stanfords CS231N by Fei-Fei Li, Justin Johnson, Serena Yeung, as well as

1.46k views β€’ 79 slides
Image Processing CS 110 Why Image Processing? Medical Images

Image Processing CS 110 Why Image Processing? Medical Images

Image Processing CS 110 Why Image Processing? Medical Images 2 Why Image Processing? Manufacturing Dig;al Image Processing, Spring 2006 3 What can you

717 views β€’ 24 slides
Lecture 1 Introduction Objectives Digital image processing, Why? Scope of digital image

Lecture 1 Introduction Objectives Digital image processing, Why? Scope of digital image

Lecture 1 Introduction Objectives Digital image processing, Why? Scope of digital image processing Exploration of application areas Components in a typical general-purpose image processing system Directions Digital image

511 views β€’ 30 slides
BLINK: A GPU-Enabled Image Processing Framework Mark Davey Lead HPC Engineer The Foundry The

BLINK: A GPU-Enabled Image Processing Framework Mark Davey Lead HPC Engineer The Foundry The

BLINK: A GPU-Enabled Image Processing Framework Mark Davey Lead HPC Engineer The Foundry The Foundry and HPC The Foundry Founded in 1996 We develop award-winning visual effects, computer graphics and design software used globally

560 views β€’ 21 slides
Image Processing Todays Class Image Representations: Matrices Image Representations: RGB,

Image Processing Todays Class Image Representations: Matrices Image Representations: RGB,

CS6501: Deep Learning for Visual Recognition Image Processing Todays Class Image Representations: Matrices Image Representations: RGB, HSV, etc Image Processing: Brightness Image Filtering: Mean Filter, Median Filter Image Filtering:

959 views β€’ 77 slides
Introduction to Computer Graphics Image Processing (1) June 10, 2020 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 10, 2020 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 10, 2020 Kenshi Takayama Todays topics Edge-aware image processing Gradient-domain image processing 2 Image smoothing using Gaussian Filter Smoothness parameter

841 views β€’ 37 slides
Introduction to Computer Graphics Image Processing (1) July 4, 2019 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) July 4, 2019 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) July 4, 2019 Kenshi Takayama Todays topics Edge-aware image processing Gradient-domain image processing 2 Image smoothing using Gaussian Filter Smoothness parameter

1.02k views β€’ 36 slides
Introduction to Computer Graphics Image Processing (1) June 8, 2017 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 8, 2017 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 8, 2017 Kenshi Takayama Todays topics Edge-aware image processing Gradient-domain image processing 2 Image smoothing using Gaussian Filter Smoothness parameter

1.25k views β€’ 38 slides
Introduction to Computer Graphics Image Processing (1) June 9, 2016 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 9, 2016 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 9, 2016 Kenshi Takayama Todays topics Edge-aware image processing Gradient-domain image processing 2 Image smoothing using Gaussian Filter Smoothness parameter

1k views β€’ 36 slides
An pre-processing toolbox for image processing and image segmentation Project Characteristics

An pre-processing toolbox for image processing and image segmentation Project Characteristics

Technische Universitt Mnchen Chair for Computation in Engineering An pre-processing toolbox for image processing and image segmentation Project Characteristics Modeling: Mathematics: Geometric Modeling for biomechanical applications:

308 views β€’ 3 slides
Introduction to Digital Image Processing Asim Banerjee IEEE Workshop on Image Processing. 1 st

Introduction to Digital Image Processing Asim Banerjee IEEE Workshop on Image Processing. 1 st

Introduction to Digital Image Processing Asim Banerjee IEEE Workshop on Image Processing. 1 st March 2009. Objective To provide an introduction to basic concepts and methodologies of Digital Image Processing To familiarize one

1.16k views β€’ 53 slides
High Quality Real Time Image Processing Framework on Mobile Platforms using Tegra K1 Eyal Hirsch

High Quality Real Time Image Processing Framework on Mobile Platforms using Tegra K1 Eyal Hirsch

High Quality Real Time Image Processing Framework on Mobile Platforms using Tegra K1 Eyal Hirsch SagivTech Snapshot Established in 2009 and headquartered in Israel Core domain expertise: GPU Computing and Computer Vision What we

489 views β€’ 30 slides
David Tschumperl Image Team, GREYC / CNRS (UMR 6072) IPOL Workshop on Image Processing

David Tschumperl Image Team, GREYC / CNRS (UMR 6072) IPOL Workshop on Image Processing

David Tschumperl Image Team, GREYC / CNRS (UMR 6072) IPOL Workshop on Image Processing Libraries, Cachan/France, June 2012 Presentation layout Image Processing : Get the Facts 1 The CImg Library : C++ Template Image Processing Library 2

715 views β€’ 54 slides
Baseband Signal Processing Framework Baseband Signal Processing Framework for the OsmocomBB GSM

Baseband Signal Processing Framework Baseband Signal Processing Framework for the OsmocomBB GSM

Baseband Signal Processing Framework Baseband Signal Processing Framework for the OsmocomBB GSM Protocol Stack Harald Krll, Christian Benkeser, Stefan Zwicky, Benjamin Weber, Qiuting Huang Integrated Systems Laboratory, ETH Zurich d S b i h

306 views β€’ 26 slides
Image Transforma1ons image filtering : change range of image Image Processing : g(x) =

Image Transforma1ons image filtering : change range of image Image Processing : g(x) =

Image Transforma1ons image filtering : change range of image Image Processing : g(x) = h(f(x)) Transforma1ons, Warping and Morphing h Last Week Courtesy: Irfan Essa, Georgia Tech James Hayes, Georgia Tech Alexei Efros, UC Berkeley

308 views β€’ 6 slides
Exploring image processing pipelines with scikit-image, joblib, ipywidgets and dash A bag of

Exploring image processing pipelines with scikit-image, joblib, ipywidgets and dash A bag of

Exploring image processing pipelines with scikit-image, joblib, ipywidgets and dash A bag of tricks for processing images faster Emmanuelle Gouillart joint Unit CNRS/Saint-Gobain SVI and the scikit-image team @EGouillart From images to science

446 views β€’ 22 slides

More recommend