Using sparse optical flow for multiple Kinect applications 27.6.2013 Stefan Guthe 1
Microsoft Kinect – Recap and Impact Launched October 2010 Consumer-Grade RGB-D Sensor Over 3000 papers in the last 3 years related to the Kinect Scientific interest beyond Monocular Motion Capturing Multiple Kinect Scenarios Face Tracking, SLAM, Hand pose estimation 2
Outline Scenario: Gas capturing Related Work Our approach Algorithmic details Results Outlook 3
Scenario: Gas Capturing Classical Approach: Schlieren Solved for 2d (Qualitative, Quantitative) 3D still relies on particles or Laser-Doppler Our scenario: Propane Gas travelling in air [Settles 2008] 4
Related Work First approach to 3d Schlieren: Atcheson et al. 2008 Wavelet Noise Background Horn-Schunck Optical Flow to Detect Pixel deviations Reconstruction with Diffusion Tensor and Poisson Integration to 3d gradients in constrained Voxel volume [Atcheson 2008] 5
Our approach Multiple Kinects 6
Our approach – the setup Convergent multiple Kinect setup, common world space Projection walls reduce scene depth Index gradient by propane flowing with 4 bar in air occluders with different aerodynamics 7
Our approach – gas detection in Kinect streams First approach: Difference in depth image stream [Berger 2011] Second approach: IR stream + tailored optical flow 8
Algorithmic details Mean spot intensity distribution of Kinect pattern resembles a aaussian Significant differences at nodge and in the tail regions 9
Algorithmic details 1) Find spots in reference image Bwmorph + brightness threshold 2) Fit mean spot kernel Least squares approach 3) For each frame and each spot Determine best fit of kernel within range from starting position Use least squares approach 4) Interpolate difference vectors to fill holes 5) Iteratively reconstruct volume based on difference vectors 10
Results Outperformed State of the Art 2012 The PSNR of Kinect 1.0 is believed to be in the range 35dB - 20dB 11
Results – reconstructed gas flows 12
Conclusion and outlook Employed multiple Kinects in convergent setup Reconstruction of Propane Flow in air around occluders Sparse Optical Flow based on Least-Squares Fit to spots Future work Compare Time-Of-Flight approaches to solution in Kinect 2.0 setups Mobile ad-hoc gas capturing (e.g. DARPA robotics challenge) 13
Fin. Thank you for your attention 14
Multiple Kinect Studies Project Sumary Schematic of Multiple Hardware Shutter and Kinect Kinect Setup ! Capture scene motion data from multiple unsynchronized RGB-D footage ! Heterogenous Sensor Setup ! Opaque and Transparent Object Motion ! Summarization in 3D space Published in VMV 2011, CDC4CV 15
Multiple Kinect Studies – Heterogeneous Sensor Calibration 16
Tomographic Gas reconstruction Project Sumary ! Record heated air flow atop camping stove ! Rely on Background-oriented Schlieren ! Masking on optical flow ! Reconstruction based on radial basis functions and diffusion tensors ! Poisson-Integration leads to Refractive Index field Published in Springer Lecture Series 17
Tomographic Gas reconstruction 18
Multiple Kinect Studies – NiTe vs. Ours 1/2 19
Multiple Kinect Studies – NiTe vs. Ours 2/2 20
Recommend
More recommend
