richardson lucy deblurring for moving light field cameras
play

Richardson-Lucy Deblurring for Moving Light Field Cameras Donald - PowerPoint PPT Presentation

Jan 12, 2024 •129 likes •597 views

Richardson-Lucy Deblurring for Moving Light Field Cameras Donald Dansereau 1 , Anders Eriksson 2 and Jrgen Leitner 2,3 1 Stanford University, 2 Queensland University of Technology, 3 ARC Centre of Excellence for Robotic Vision CVPR:LF4CV

  1. Richardson-Lucy Deblurring for Moving Light Field Cameras Donald Dansereau 1 , Anders Eriksson 2 and Jürgen Leitner 2,3 1 Stanford University, 2 Queensland University of Technology, 3 ARC Centre of Excellence for Robotic Vision CVPR:LF4CV Workshop 2017 July 26

  2. 3D Motion Complicates Vision Scene-dependent nonuniform apparent motion 2 ComputationalImaging.org

  3. 3D is Easier in 4D We have 6-DOF virtual camera control Video Stabilization [video] http://pages.cs.wisc.edu/~lizhang/projects/lfstable/ [Smith2009] 3 ComputationalImaging.org

  4. 3D is Easier in 4D We can fix the camera’s position Closed-Form Change Detection Per-pix ixel l stil still- l-ca camera meth thods ● Change detection ● Tracking/segmentation ● Velocity & temporal filtering [dansereau2016] http://dgd.vision/Projects/LFChangeDet/ 4 ComputationalImaging.org

  5. 3D is Easier in 4D We can fix the camera’s position Closed-Form Change Detection Per-pix ixel l stil still- l-ca camera meth thods ● Change detection ● Tracking/segmentation ● Velocity & temporal filtering [dansereau2016] http://dgd.vision/Projects/LFChangeDet/ 5 ComputationalImaging.org

  6. 3D is Easier in 4D We can fix the camera’s position Closed-Form Change Detection Per-pix ixel l stil still- l-ca camera meth thods ● Change detection ● Tracking/segmentation ● Velocity & temporal filtering [dansereau2016] http://dgd.vision/Projects/LFChangeDet/ 6 ComputationalImaging.org

  7. 3D is Easier in 4D Lukas-Kanade optical flow generalizes to 6-DOF Linearize Apparent Motion Closed-form 6-DOF Odometry L x L y T r u e P l e n o p t i c P o i n t w i s e F e a t u r e 2 ) m ( − 2 z L y L x 1 0 1 4 1 2 5 1 0 0 8 − 5 6 4 − 1 0 y ( m ) x ( m ) 2 − 1 5 L z L z [Neumann2002, Dansereau2011, Dong2013] 7 ComputationalImaging.org

  8. 3D is Easier in 4D Lukas-Kanade optical flow generalizes to 6-DOF Linearize Apparent Motion Closed-form 6-DOF Odometry L x L y T r u e P l e n o p t i c P o i n t w i s e F e a t u r e 2 ) m ( − 2 z L y L x 1 0 1 4 1 2 5 1 0 0 8 − 5 6 4 − 1 0 y ( m ) x ( m ) 2 − 1 5 L z L z [Neumann2002, Dansereau2011, Dong2013] 8 ComputationalImaging.org

  9. Blur in 3D Scenes Convolution models blurring in 2D… Can we replace convolution with LF rendering in 3D scenes? 9 ComputationalImaging.org

  10. Related Work “Light Field Blind Motion Deblurring” [Srinivasan 2017] ● 3-DOF ● Insights on blur manifestation in LF ● Blind ● Modern optimization (ADAM) LF-RL ● Requires extension to be blind ● 6-DOF ● Proof of convergence to ML estimate (see paper) ● New LF equiparallax regularizer 10 ComputationalImaging.org

  11. Related Work “Richardson-Lucy Deblurring for Scenes under a Projective Motion Path” [Tai et al. 2011] 11 ComputationalImaging.org

  12. Richardson-Lucy Deblurring IN 12 ComputationalImaging.org

  13. Light Field Richardson-Lucy IN FWD BLUR REV BLUR Still LF 13 ComputationalImaging.org

  14. Light Field Richardson-Lucy IN FWD BLUR REV BLUR Still LF Simulated Motion 14 ComputationalImaging.org

  15. Light Field Richardson-Lucy IN FWD BLUR REV BLUR Still LF Simulated Motion Simulated Blur 15 ComputationalImaging.org

  16. Regularization Anisotropic total variation Favour textural edges [Goldluecke & Wanner 2013, Heber2013] Equiparallax Favour equal slopes in s,u and t,v 16 ComputationalImaging.org

  17. Rendered Results: Rot about y Ori riginal Ori riginal Ori riginal 17 ComputationalImaging.org

  18. Rendered Results: Rot about y Blurre Ori riginal red Ori Bl Blurre riginal red Ori Bl Blurre riginal red 18 ComputationalImaging.org

  19. Rendered Results: Rot about y Deblurre Blurre Ori riginal red rred Deblurr Ori Bl Blurre riginal red rred Deblurr Blurre Bl Ori riginal red rred 19 ComputationalImaging.org

  20. Rendered Results: Rot about z rotz Bl Blurre rred Bl Blurr rred Bl Blurr rred 20 ComputationalImaging.org

  21. Rendered Results: Rot about z rotz Bl Blurre Deblurre rred red Blurr Deblurre Bl rred rred Deblurre Bl Blurr rred rred 21 ComputationalImaging.org

  22. Rendered Results: Trans along x Bl Blurre rred Bl Blurr rred Bl Blurr rred 22 ComputationalImaging.org

  23. Rendered Results: Trans along x Bl Blurre Deblurre rred red Blurr Deblurre Bl rred rred Deblurre Blurr Bl rred rred 23 ComputationalImaging.org

  24. Rendered Results: Rot about z rotz Bl Blurre rred Bl Blurr rred Bl Blurr rred 24 ComputationalImaging.org

  25. Rendered Results: Rot about z rotz Bl Blurre Deblurre rred red Blurr Deblurre Bl rred rred Deblurre Bl Blurr rred rred 25 ComputationalImaging.org

  26. Rendered Results: Trans along z Bl Blurre rred Blurr Bl rred Bl Blurr rred 26 ComputationalImaging.org

  27. Rendered Results: Trans along z Deblurre Bl Blurre rred red Bl Blurr Deblurre rred rred Bl Blurr Deblurre rred rred 27 ComputationalImaging.org

  28. Rendered Results: Trans along z Deblurre Blurre Bl rred red Without Regulariza Bl Blurr Deblurre rred rred zation Without Regulariza Deblurre Blurr Bl rred rred zation 28 ComputationalImaging.org

  29. Results: Captured Quantitative Evaluation Repeatable camera motion Isolated dimensions Known magnitudes George 29 ComputationalImaging.org

  30. Validating Calibration & Rendering Still Camera calibration, rectification Metric blur simulation Metric robot motion Camera-to-robot calibration 30 ComputationalImaging.org

  31. Validating Calibration & Rendering Measured Still Blur Camera calibration, rectification Metric blur simulation Metric robot motion Camera-to-robot calibration 31 ComputationalImaging.org

  32. Validating Calibration & Rendering Simulated Blur Measured Still Blur Camera calibration, rectification Metric blur simulation Metric robot motion Camera-to-robot calibration 32 ComputationalImaging.org

  33. Validating Calibration & Rendering Simulated Blur Measured Still Still Blur Camera calibration, rectification Metric blur simulation Metric robot motion Camera-to-robot calibration 33 ComputationalImaging.org

  34. Results: Captured No increase in noise Large increase in sharpness Regularization is helping 34 ComputationalImaging.org

  35. Results: Captured Measured Blur 35 ComputationalImaging.org

  36. Results: Captured Measured Blur Deblurred 36 ComputationalImaging.org

  37. Results: Captured 37 ComputationalImaging.org

  38. Results: Captured 38 ComputationalImaging.org

  39. Results: Captured 39 ComputationalImaging.org

  40. Results: Captured 40 ComputationalImaging.org

  41. Results: Captured 41 ComputationalImaging.org

  42. Results: Captured 42 ComputationalImaging.org

  43. Summary & Future Work Generaliz lized co convolu lutio tional l blu lur usin sing LF Renderin ing Applied to RL deblurring 3D scenes, 6-DOF camera motion Proof of convergence to ML estimate Equiparallax regularization Next: Equiparallax regularization: applications Beyond 6-DOF, defocus Blind deblurring 43 ComputationalImaging.org

  44. Acknowledgments QUT HPC Group George 44 ComputationalImaging.org

  45. Light Field T oolbox for MATLAB Load Gantry and Lytro imagery Calibrate and rectify Lytro imagery Linear depth, volume fjlters Denoising: low-light, fog, dust, murky water Occluder removal: rain, snow, silty water LF Synth: Bare-Bones Rendering 45 ComputationalImaging.org

Recommend

1 Prior Work: Depth from Defocus Prior Work: Modifying Cameras DSLR with a focusing mechanism

1 Prior Work: Depth from Defocus Prior Work: Modifying Cameras DSLR with a focusing mechanism

Light fields Light Fields: From Shape Recovery to Sparse Reconstruction Ravi Ramamoorthi University of California, San Diego Refocusing Viewpoint Change L4CV Keynote, Jul 26, 2017 Consumer light field cameras Outline Motivation Light

113 views • 10 slides
Image Deblurring Seungyong Lee POSTECH 1 Contents Fast Motion Deblurring (Siggraph Asia

Image Deblurring Seungyong Lee POSTECH 1 Contents Fast Motion Deblurring (Siggraph Asia

Image Deblurring Seungyong Lee POSTECH 1 Contents Fast Motion Deblurring (Siggraph Asia 2009) Non-uniform Motion Deblurring for Camera Shakes using Image Registration (Siggraph 2011 Talks) Text Deblurring (an ongoing project) 2

1.35k views • 87 slides
Light Field Depth Estimation Challenge LF4CV@CVPR2017 Katrin Honauer, Ole Johannsen, Bastian

Light Field Depth Estimation Challenge LF4CV@CVPR2017 Katrin Honauer, Ole Johannsen, Bastian

Light Field Depth Estimation Challenge LF4CV@CVPR2017 Katrin Honauer, Ole Johannsen, Bastian Goldluecke July 26, 2017 4D Light Fields 2D grid of 2D views all cameras are parallel and identical (except shift: negative disparity) [Figure:

479 views • 22 slides
Field of a moving particle in a dielectric The E- and B-field associated to a uniformly

Field of a moving particle in a dielectric The E- and B-field associated to a uniformly

Field of a moving particle in a dielectric The E- and B-field associated to a uniformly moving particle in vacuum are given by and where Previously we considered the case b <<1 P. Piot, PHYS 571 Fall 2007 Field

504 views • 14 slides
Spinning Parallelogram Operator (SPO) for Light Field Depth Estimation 4D Light Field Benchmark

Spinning Parallelogram Operator (SPO) for Light Field Depth Estimation 4D Light Field Benchmark

Spinning Parallelogram Operator (SPO) for Light Field Depth Estimation 4D Light Field Benchmark Challenge at 2 nd Workshop on LF4CV, CVPR 2017 Shuo Zhang 1 , Hao Sheng 1 , Chao Li 1 , Jun Zhang 2 , Zhang Xiong 1 1 Beihang University 2

419 views • 12 slides
High-Fidelity Light Field VR Playback Using NVIDIA GPUs Tim Milliron, Vice President of

High-Fidelity Light Field VR Playback Using NVIDIA GPUs Tim Milliron, Vice President of

High-Fidelity Light Field VR Playback Using NVIDIA GPUs Tim Milliron, Vice President of Engineering Nikhil Karnad, Architect for Image-Based Rendering 1 An Introduction to Light Field 2 THE LIGHT FIELD HOW IT WORKS KEY BENEFITS Capture

449 views • 34 slides
Cameras, Light and Shading CS 543 / ECE 549 Saurabh Gupta Spring 2020, UIUC

Cameras, Light and Shading CS 543 / ECE 549 Saurabh Gupta Spring 2020, UIUC

Cameras, Light and Shading CS 543 / ECE 549 Saurabh Gupta Spring 2020, UIUC http://saurabhg.web.illinois.edu/teaching/ece549/sp2020/ Many slides adapted from S. Seitz, L. Lazebnik, D. Hoiem, D. Forsyth Recap P Y Z f O y &' ( ,

765 views • 48 slides
What is Light ? Discussion Questions: 1) What is light? 2) How fast does light travel? 3) What

What is Light ? Discussion Questions: 1) What is light? 2) How fast does light travel? 3) What

What is Light ? Discussion Questions: 1) What is light? 2) How fast does light travel? 3) What is visible light? 4) What is our planets main source of light? electric field travels in a vertical plane What is Light? direction of travel

523 views • 4 slides
Ford Site Design Standards April 23, 2019 City of Saint Paul Mike Richardson Lucy Thompson

Ford Site Design Standards April 23, 2019 City of Saint Paul Mike Richardson Lucy Thompson

Ford Site Design Standards April 23, 2019 City of Saint Paul Mike Richardson Lucy Thompson Project Manager Principal City Planner Planning & Economic Development Menaka Mohan Ford Site Planner LHB, Inc. Michael Lamb Lydia Major

372 views • 19 slides
Cameras EECS 442 Prof. David Fouhey Winter 2019, University of Michigan

Cameras EECS 442 Prof. David Fouhey Winter 2019, University of Michigan

Cameras EECS 442 Prof. David Fouhey Winter 2019, University of Michigan http://web.eecs.umich.edu/~fouhey/teaching/EECS442_W19/ Next Few Classes Tuesday: Cameras (Projective Geometry) Thursday: Cameras (Light, Lenses) Next

1.41k views • 95 slides
Intelligent Transportation Systems James Marty Farris P.E. ODOT ITS Field Equipment

Intelligent Transportation Systems James Marty Farris P.E. ODOT ITS Field Equipment

Intelligent Transportation Systems James Marty Farris P.E. ODOT ITS Field Equipment Operations Center Websites Cities Field Equipment DMS Cameras RWIS Snowplow Cameras Portable DMS Dynamic Message Signs

856 views • 23 slides
Insights from PhenoCam g Andrew D. Richardson Harvard University I thank my PhenoCam

Insights from PhenoCam g Andrew D. Richardson Harvard University I thank my PhenoCam

Using digital cameras to monitoring vegetation phenology: Insights from PhenoCam g Andrew D. Richardson Harvard University I thank my PhenoCam collaborators for their contributions to this work. I gratefully acknowledge funding support from

344 views • 22 slides
Gesture Detection A camera is used to detect gestures made in the cameras field of view using

Gesture Detection A camera is used to detect gestures made in the cameras field of view using

Gesture Detection A camera is used to detect gestures made in the cameras field of view using infrared LEDs. Detected gestures will be used to control a music player. Andrew Zhong Patrick Kuczera Shahzeb Asif 1 Motivation/Goals The

230 views • 10 slides
belly buttons, absent cameras, Linda C.H. LAI moving house, The City University of Hong Kong

belly buttons, absent cameras, Linda C.H. LAI moving house, The City University of Hong Kong

Documenting Sentiments in Hong Kong 1997 Video Diaries: belly buttons, absent cameras, Linda C.H. LAI moving house, The City University of Hong Kong November 3, 2012 fooling around 1997 A relatively recent conclusion of HK's British

443 views • 32 slides
Origin of Magnetic field A current (or moving charge) experience a magnetic force when it is in a

Origin of Magnetic field A current (or moving charge) experience a magnetic force when it is in a

Origin of Magnetic field A current (or moving charge) experience a magnetic force when it is in a magnetic field. The magnetic field is the result of another current (or moving charge). If electric field describes the interaction between two

344 views • 8 slides
Body Worn Cameras and Transparency Increase in use of cameras Expectation of

Body Worn Cameras and Transparency Increase in use of cameras Expectation of

10/10/2016 Body Cameras: The Intersection of Public Records and Law Enforcement Frayda Bluestein, Jeff Welty, Tom Carruthers Body Worn Cameras and Transparency Increase in use of cameras Expectation of accountability and transparency

348 views • 9 slides
Chapter 6 6.1 Eyes and Cameras: Everyday Light Sensors Telescopes: Portals of Discovery Our

Chapter 6 6.1 Eyes and Cameras: Everyday Light Sensors Telescopes: Portals of Discovery Our

Chapter 6 6.1 Eyes and Cameras: Everyday Light Sensors Telescopes: Portals of Discovery Our goals for learning How does your eye form an image? How do we record images? How does your eye form an Refraction image? Refraction

250 views • 9 slides
Chapter 6 Telescopes: Portals of Discovery 6.1 Eyes and Cameras: Everyday Light Sensors Our

Chapter 6 Telescopes: Portals of Discovery 6.1 Eyes and Cameras: Everyday Light Sensors Our

Chapter 6 Telescopes: Portals of Discovery 6.1 Eyes and Cameras: Everyday Light Sensors Our goals for learning How does your eye form an image? How do we record images? How does your eye form an image? Refraction Refraction

850 views • 53 slides
Sources of the Magnetic Field We have been dealing with problems involving magnetic force on

Sources of the Magnetic Field We have been dealing with problems involving magnetic force on

Sources of the Magnetic Field We have been dealing with problems involving magnetic force on charged particles moving in a magnetic field Where do magnetic field come from? Moving charges or electric currents TimelineI

658 views • 18 slides
Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for Optical See-Through Head-Mounted

Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for Optical See-Through Head-Mounted

Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for Optical See-Through Head-Mounted Displays Yuta Itoh, Toshiyuki Amano, Daisuke Iwai, and Gudrun Klinker IEEE ISMAR 2016, Tue. 20th Sep. 2016 Gaussian Light Field: Estimation of

718 views • 51 slides
Range gated cameras technology and its applications Friday, October 10, 14 Range gated cameras

Range gated cameras technology and its applications Friday, October 10, 14 Range gated cameras

Range gated cameras technology and its applications Friday, October 10, 14 Range gated cameras can have several names * Time resolved cameras * Range gated cameras * Laser tomography * Grindad laserkamera / avbildning ( SV) All systems

647 views • 25 slides
Order One 46 Classrooms Cameras only: $56,108.00 Order Two 8 Classrooms Cameras

Order One 46 Classrooms Cameras only: $56,108.00 Order Two 8 Classrooms Cameras

S WEARING I N N EW 2020-2021 B OARD OF E DUCATION S TUDENT R EPRESENTATIVE Order One 46 Classrooms Cameras only: $56,108.00 Order Two 8 Classrooms Cameras and Other Equipment (Touchboard, etc.): $32,160.00 To Cover All Staff

654 views • 30 slides
Cameras Cameras Good cameras need focusing, cheap ones dont Good cameras need focusing,

Cameras Cameras Good cameras need focusing, cheap ones dont Good cameras need focusing,

Cameras 1 Cameras 2 Observations about Cameras Observations about Cameras They record a scene on an image sensor They record a scene on an image sensor Cameras Cameras Good cameras need focusing, cheap ones dont Good cameras need

484 views • 4 slides
Light Field Structure Analysis With material courtesy of Jaakko

Light Field Structure Analysis With material courtesy of Jaakko

Light Field Structure Analysis With material courtesy of Jaakko Lehtinen Key observation Light rays are highly coherent Rays originating from same surface point

623 views • 27 slides

More recommend