Monocular Vision Based Obstacle Avoidance: A Literature Review - PowerPoint PPT Presentation

Monocular Vision Based Obstacle Avoidance: A Literature Review

Outline Introduction ● Problem ● ● Relevant Work ○ Monocular Cues ○ Machine Learning ○ Expansion ○ Optical Flow ○ SLAM ○ GPU Processing ● Results Conclusion ● Q & A ●

Introduction ● Autonomous UAV operation in complex environments requires: Navigation ○ ○ Homing Obstacle detection/avoidance ○ ● Detection and avoidance will be the focus of this presentation http://www.ros.org/news/robots/uavs/

Obstacle Detection/Avoidance ● What is obstacle detection/avoidance? ● Why is obstacle detection/avoidance useful? The Problem ● ○ UAV drones are limited by: carrying capacity ■ ■ computational power on board battery ■ ○ Solutions are either heavy, computationally http://www.dronemagazine.it/1013-black-hornet-nano-drone-dellesercito-inglese-video/ intensive, or energy inefficient

Previous Solutions ● MAVs require: ○ Light weight ○ Computationally Inexpensive ○ Energy efficient Past solutions include: ● ○ Sensor Arrays, such as: LiDAR, radar, and sonar Stereo Vision ○

Monocular Vision ● Use one camera to view the world ● Techniques for monocular vision Monocular cues ○ ○ Perception changes Optical flow ○ ● Less weight and energy consumption than stereo https://www.youtube.com/watch?v=V4r2HXGA8jw

Problem with Monocular Vision ● Single camera for vision ○ No inherent depth perception ○ Previous stereovision algorithms don’t apply ● Must rely on clever algorithms to make up for its shortcoming

Proposed Solutions ● Researchers have attempted to tackle drawbacks of monocular vision ● Categories of research in this talk: Monocular Cues ○ ○ Machine Learning Expansion ○ ○ Optical Flow SLAM ○ ○ GPU Processing

Monocular Cues ● Use cues from scene in image to estimate depth Cue types: ● ○ Accommodation ○ Motion parallax ○ Size constancy https://www.pinterest.com/pin/46513808625067354/

Related Works: Monocular Cues ● Michels et al. - 2005 ○ Monocular depth cues using a portable laser measuring system ● Ross et al. - 2013 Feature extraction in wooden environments for object detection ○ ● Bills et al. - 2011 ○ Specialized monocular cue algorithms for different environments Wu et al. - 2014 ● ○ Synopsis of different monocular vision techniques ● Croon et al. - 2010 ○ Texture variation for obstacle detection

Machine Learning ● Use learning capable computers to: ○ Improve accuracy of a detection and avoidance system ○ Basis of an detection and avoidance system ● Artificial Neural Networks is a common machine learning technique used https://www.quora.com/

Related Works: Machine Learning ● Michels et al. - 2005 ○ Supervised and reinforcement learning on real and synthetic images for training depth Ross et al. - 2013 ● ○ Iterative supervised learning using DAgger and an expert pilot’s movements Bills et al. - 2011 ● ○ Classifier for identifying different indoor environments ● Oh et al. - 2004 Automation of gain tuning to improve light intensity variations using a neural network ○ ● Smolyanskiy et al. - 2017 ○ Two deep neural networks for navigation in nature trails

Expansion ● Use expansion of objects to detect obstacles Feature points such as SIFT and ● SURF are used: ○ Approximate object location and dimension ○ Quickly expanding feature points belong to obstacles http://www.mdpi.com/1424-8220/17/5/1061/htm

Related Works: Expansion ● Aguilar et al. - 2017 ○ Matching features from known objects to determine change in scale Mori and Scherer - 2013 ● ○ Scale change of SURF features for time to impact Al-Kaff et al. - 2016 ● ○ Expansion of features and area of objects ● Chavez and Gustafson - 2009 Feature expansion using SIFT ○

Optical Flow ● Use movement in consecutive images to understand environment Two versions - dense and sparse ● Dense ● ○ Calculates vector of displacement for each pixel Sparse ● ○ Calculates vector of displacement for selected features ● Most research for MAVs use sparse OF http://drstyle.me/honey-bee-optical-flow-landing-airplane-plane/

Related Works: Optical Flow ● Sagar - 2014 ○ Use OF on clusters of features to distinguish near and far obstacles Wu et al. - 2014 ● ○ Detailed background, uses, and disadvantages of OF Oh et al. - 2004 ● ○ OF sensitivity to illumination variation improved by using neural network ● Chavez and Gustafson - 2009 Use Lucas-Kanade OF to autonomously navigate a flapping wing MAV indoors ○

SLAM ● Simultaneous Localization and Mapping (SLAM) Maps environment around UAV and ● estimates location within it ○ Valuable information for detection/avoidance ○ Traditionally accomplished by complex sensor or stereo systems ● Monocular SLAM (MonoSLAM) http://www.asctec.de/en/uav-drone/ascending-technologies/ ○ Use one camera to perform SLAM asctec-researchline/uav-uas-drone-computer-vision-slam/

Related Works: SLAM ● Park et al. - 2011 ○ Novel parallel MonoSLAM method using GPU SIFT ○ Required fewer features than comparative methods ● Ha and Sattigeri - 2012 Combined image segmentation and MonoSLAM to avoid obstacles ○ ○ Features from MonoSLAM linked to objects in segmented image Simulation used as test bed ○

GPU Processing ● Improved embedded GPU capabilities enables: Increased detection/avoidance performance ○ ○ Parallel execution of various detection/avoidance techniques ● These improvements equip researchers the means to create more robust algorithms and execute algorithms in real-time https://www.amazon.com/NVIDIA-Jetson-TX1-Development-Kit/dp/B017NWO6LG

Related Works: GPU Processing ● Ready and Taylor - 2007 ○ GPU could track 500 features at 70% utilization vs CPU 40 features at 90% utilization Park et al. - 2011 ● ○ MonoSLAM with GPU 5 to 9 times faster than CPU implementation Smolyanskiy et al. - 2017 ● ○ GPU ran two neural networks and visual odometry to navigate trail and avoid obstacles

Results ● Most research discussed has focused on feature detection ● However: Computation time is high ○ ○ Usually offloaded to ground station for computing Relatively small number of features ○ ● GPUs could be used to address these issues: ○ Provide computational speed up for parallelizable algorithms ○ Embedded GPU’s can be used to do onboard computing ○ Increase in tracking ability of features

Our Research ● Assumption: Closer objects expand faster than farther away objects ● The metric used: area of the features ○ ○ area of convex hull produced from feature points Improvement of feature detection using GPUs ●

Detection Algorithm 1. Collect and match features between consecutive frames 2. Keep expanding features 3. Cluster neighboring features 4. Construct convex hulls from clusters 5. Keep expanding convex hulls 6. Issue avoidance command, if required

Avoidance Algorithm 1. Draw bounding rectangles around convex hulls 2. Determine quadrants the rectangles occupy 3. Determine path of avoidance a. Calculate inverse vector sum of rectangles b. Navigate towards inverse vector sum

Conclusion ● Introduction to Monocular Vision ● Highlighted various approaches to using Monocular Vision Discussed our research ● Walk-through of our algorithms for detection and avoidance ●

Questions? http://clipartall.com/clipart/11375-clipart-question.html