Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Deep Neural Network Enhanced VSLAM Landmark Selection Dr. Patrick Benavidez University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Overview 1 Introduction 2 Background on methods used in VSLAM 3 Proposed Method 4 Testbed 5 Preliminary Results University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results What is VSLAM? Visual Simultaneous Localization and Mapping Use of vision and depth sensors to acquire features from an environment, map them and to navigate with the map University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Motivation to use VSLAM Similar to methods used by humans GPS-denied and contested environments Spoofing attacks on GPS Cloud-based robotics Data simplification and organization University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results What processes are involved in VSLAM? Sensors capture properties of the surrounding environment Operations to transform captured environmental data with robot pose data Algorithms to place transformed environmental data into existing map Methods to determine whether the robot has already visited a particular location Operations to update existing data in the map Loop closure operations to constrain the bounds of a map University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Typical Scenes for VSLAM Indoors Outdoors University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results VSLAM Mapping Process feature detectors – find useful feature rich points in an image feature descriptors – describe sets of features feature matching – match features into map University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Feature Detection Corner detectors - Harris, Shi Tomasi Scale Invariant Feature Transform (SIFT) – better than Harris Corner detector Speeded-Up Robust Features (SURF) – faster version of SIFT Features from Accelerated Segment Test (FAST) – ”fast enough for SLAM” [ http://docs.opencv.org/3.0- beta/doc/py tutorials/py feature2d/py sift intro/py sift intro.html #sift-intro ] University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Feature Descriptors Feature descriptors describe sets of features Feature descriptors are saved in a database or similar structure Binary Robust Independent Elementary Features (BRIEF) Oriented FAST and Rotated BRIEF (ORB) SIFT & SURF are patented, ORB is free Fusion of FAST Keypoint Detector and BRIEF descriptor methods with increased performance [ http://docs.opencv.org/3.0- beta/doc/py tutorials/py feature2d/py orb/py orb.html#orb ] University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Feature Matching Commonly used methods are brute force and FLANN matching algorithms These methods match feature descriptors of a newly acquired image to those saved in the database University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Bag of Words A bag of words in natural language processing is the decomposition of a sentence into its constituent components (words) and storing them in a container (bag) Example [ https://en.wikipedia.org/wiki/Bag-of-words model ]: Sentence 1 – John likes to watch movies. Mary likes movies too. Sentence 2 – John also likes to watch football games. Bag of words used to describe sentence 1 and sentence 2 –[”John”, ”likes”, ”to”, ”watch”, ”movies”, ”also”, ”football”, ”games”, ”Mary”, ”too”] University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Visual Bag of Words A visual bag of words is where an image is broken down into its component regions of interest (ROI) in an image (words) and stored in a collection (bag) Labels can be applied to the ROI in a bag Example: [ https://gilscvblog.com/2013/08/23/bag-of-words-models- for-visual-categorization/ ]: University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Application of Visual Bag of Words – Scene Identification A visual bag of words (collection of known images) is created to describe particular components for each scene Features taken from the latest camera image are compared to those in the visual bag of words for each scene A collection of the most relevant words (images) in each bag matching the input image are generated The bag most closely matching the current image identifies the scene University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Application of Visual Bag of Words – Mapping A visual bag of words (collection of unknown images) is created at runtime to describe particular components discovered by a robot Locations where the words have been discovered are input into the map Features taken from the latest camera image are compared to those in the visual bag of words to determine if the object has been seen before New objects are added to the bag Objects already in the bag are used to identify where the agent is in a map if it has been to that location before University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Problems with existing methods Features are made too general by design Recall that features are simple components of an image: corners, edges, intersections, etc. Almost every type of object can contain these features Problem: Both static and dynamic objects in the environment are registered in the map in the same context University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Problems with existing methods (continued) Objects with freedom to move around the environment Examples: people, animals, robots, mobile carts, etc. Problem: Traditional SLAM/VSLAM will fail to produce meaningful maps with multiple agents working in the same environment University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Problems with existing methods (continued) Time varying objects that do not travel around the environment Examples: trees, plants, tracking solar panels, windmills, flags, banners, televisions, digital billboards Problem: Features should not be taken off of the dynamic portions of these items Features can be acquired from their static components (planted/grounded base, static frame) [ http://www.swri.org/3pubs/ttoday/Summer12/images/IMG0340- 250x167.jpg ] University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Introduction Background on methods used in VSLAM Proposed Method Testbed Preliminary Results Problems with existing methods (continued) Loop closure – revisiting the same place twice produces multiple paths due to odometry errors University of Texas at San Antonio - Department of Electrical and Computer Engineering Deep Neural Network Enhanced VSLAM Landmark Selection
Recommend
More recommend
