an introduction to
play

An Introduction to Eric Rosen, Kaiyu Zheng Semantic Mapping in - PowerPoint PPT Presentation

Nov 15, 2023 •230 likes •1k views

An Introduction to Eric Rosen, Kaiyu Zheng Semantic Mapping in Robotics 3/22/2019 Outline Timeline of Semantic mapping Why semantic mapping (in robotics)? How? Problem definition Literature review What? (Our

  1. An Introduction to Eric Rosen, Kaiyu Zheng Semantic Mapping in Robotics 3/22/2019

  2. Outline • Timeline of “Semantic mapping” • Why semantic mapping (in robotics)? • How? • Problem definition • Literature review • What? (Our research) • TopoNets, GraphSPNs • Action-oriented semantic mapping

  3. Timeline “Semantic Mapping” • Originated in linguistics - 1960s • Correspondence of hierarchies in languages For example: phonogram “/ h əˈ l ō /” → word “hello” On the Uniqueness of Semantic Mapping [Householder, 1962] • In literacy (vocabulary instruction) • Structure of knowledge in graphic form Semantic Mapping [Johnson et.al., 1986]

  5. Timeline “Semantic Mapping” • Appearance in CS (NLP) - 1989 • Neural network (!) • Semantic relations between symbolic data. Self-Organizing Semantic Maps [Ritter and Kohonen, 1989] (training data) “ geometrically or topologically organized maps”

  6. These concepts are central to semantic mapping in robotics! Self-Organizing Semantic Maps [Ritter and Kohonen, 1989]

  7. Timeline “Semantic Mapping” (robotics) • Semantic hierarchy of spatial representations • [sensorimotor → control] → topology → geometry A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations [Kuipers and Byun, 1991] • Definition of terms “metric map”, “topological map”, “hybrid map” (no semantic map  ) Some Notes on the Use of Hybrid Maps for Mobile Robots [Buschka and Saffotti, 2004] • Hybrid metric-topological-semantic map Multi-Hierarchical Semantic Maps for Mobile Robotics [Galindo, Buschka et.al. , 2005]

  8. Multi-Hierarchical Semantic Maps for Mobile Robotics [Galindo, Buschka et.al. , 2005]

  9. Timeline “Semantic Mapping” (robotics) • Semantic hierarchy of spatial representations • [sensorimotor → control] → topology → geometry A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations [Kuipers and Byun, 1981] • Definition of terms “metric map”, “topological map”, “hybrid map” (no semantic map  ) Some Notes on the Use of Hybrid Maps for Mobile Robots [Buschka and Saffotti, 2004] • Hybrid metric-topological-semantic map Multi-Hierarchical Semantic Maps for Mobile Robotics [Galindo, Buschka et.al. , 2005] • 3D laser-based SLAM with scene interpretation Towards semantic maps for mobile robots [Nüchter, Hertzberg et.al. , 2008]

  10. Recent successor (also 3D SLAM) Meaningful Maps With Object-Oriented Semantic Mapping [Sünderhauf, et.al. , 2016] Towards semantic maps for mobile robots [Nüchter, Hertzberg et.al. , 2008]

  11. Timeline “Semantic Mapping” (robotics) • Semantic hierarchy of spatial representations • [sensorimotor → control] → topology → geometry A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations [Kuipers and Byun, 1981] • Definition of terms “metric map”, “topological map”, “hybrid map” (no semantic map  ) Some Notes on the Use of Hybrid Maps for Mobile Robots [Buschka and Saffotti, 2004] • Hybrid metric-topological-semantic map Multi-Hierarchical Semantic Maps for Mobile Robotics [Galindo, Buschka et.al. , 2005] • 3D laser-based SLAM with scene interpretation Towards semantic maps for mobile robots [Nüchter, Hertzberg et.al. , 2008] • Semantic Mapping with Mobile Robots [Influencer for us] PhD thesis [Pronobis, 2011]

  12. Why semantic mapping? Robots Are Getting Better Hardware ↑ Sensing ↑ Control ↑ Robots are getting better at single tasks Mobility Manipulation

  13. Why semantic mapping? Moving Forward Multiple tasks Automated planning and scheduling

  14. Why semantic mapping? Moving Forward Multiple tasks Automated planning and scheduling State representation World states description Semantic maps

  15. How? Problem definition Semantic Mapping Input PhD thesis [Pronobis, 2011] Sensory observations and odometry Prior knowledge of semantic information Output Semantic maps (what is this?) Capable to facilitate planning Shared understanding in literature

  16. Problem definition Semantic Maps • Historically informal Self-Organizing Semantic Maps Towards semantic maps for mobile robots [Ritter and Kohonen, 1989] [Nüchter, Hertzberg et.al. , 2008] Multi-Hierarchical Semantic Maps for Mobile Robotics [Galindo, Buschka et.al. , 2005] and PhD thesis [Pronobis, 2011] did not provide a formal definition either.

  17. Problem definition Semantic Maps • Formal, but ignored (and hard to find) Vague enough Somewhat complicated Semantic Maps for Robotics [Lang and Paulus, 2014]

  18. Problem definition Semantic Maps • The attempt in [Zheng et.al.’19 In submission] 𝑁 = (𝑈, 𝒀, 𝒁) 𝑁 : semantic map 𝑈 = (𝑾, 𝑭) : topological graph 𝒀 = {𝒀 𝑗 : 𝑗 ∈ 𝑾} : local observations 𝒁 = {𝒁 𝑗 : 𝑗 ∈ 𝑾} : semantic attributes Each node 𝑗 is a place (defined later) • Did not include “metric map” • No semantics in edges • Suitable for their research problem

  19. Problem definition Summary • Semantic mapping is conceptually clear • Formally define semantic mapping? • Semantic maps may vary • Yet, one should at least include: • Spatial information (contained in map) • Maps: Metric, topological • Anchoring of spatial concepts • Place classification (simplification) (We are talking about mobile robots)

  20. Literature review Maps Metric map • Metric map • Created by SLAM • Captures geometry of the world • Topological map • A graph ( V,E ) • Each node in V is a place [Sünderhauf et.al., ICRA’16] • Each edge in E indicates navigability Topological maps • Captures structure of the world [Zheng., senior thesis’17]

  21. [Pronobis , et. al. ICAPS Workshop’17]

  22. Literature review Local Place Classification • Classification from local sensory information • Through detecting objects in the environment (less common) [Viswanathan et.al., CRV’10] [Li et.al., ECCV Workshops’10] • Through robot sensory observations • RGB-D (Visual scene classification) [Zhu et.al., CVPR’16] • Laser-range [Pronobis et.al., IROS’17] [Friedman et.al., IJCAI’07] • Multi-modal (RGB+Laser+Odometry) [Pronobis et.al., IJRR’10]

  23. Literature review Mainstream: Place classification on a map Structured prediction • Relations between places • Probabilistic inference • Boost classification results

  24. Literature review Place Classification on a Map Map → Planning for robots (Mobility) Metric Maps [Friedman et.al., IJCAI’07] [Goeddel et.al., IROS’16] [Sünderhauf et.al., ICRA’16]

  25. Literature review Place Classification on a Map Map → Planning for robots (Mobility) Topological Maps [Friedman et.al., IJCAI’07] [Pronobis et.al., ICRA’12] [Zheng et.al.’19 In submission]

  26. Literature review Structured Prediction Voronoi Random Field (VRF) in [Friedman et.al., IJCAI’07] Graph-Structured Sum-Product Networks (GraphSPNs [Zheng et.al. AAAI’18] ) Factor graph (i.e. MRF) in [Pronobis et.al., ICRA’12]

  27. What? Our Research • TopoNets, GraphSPNs • Action-oriented semantic mapping

  28. What? Our Research • TopoNets, GraphSPNs • Action-oriented semantic mapping

  29. TopoNets

  30. Video link: https://www.youtube.com/watch?v=JrXeRsnJin0

  31. Take-away I End-to-end Unified Deep Model Semantics in Unified Model global context Global topology Local place semantics Sensory information Figure adapted from [Pronobis , et. al. ICAPS Workshop’17]

  32. Take-away II Tractable Exact Inference • TopoNets are Sum-Product Networks [Poon&Domingos , UAI’11] • Viewed in 2 ways: • Deep architecture • Graphical model • Structure semantics: • Hierarchical mixture of parts Latent Variable Input Variables

  33. Sum-Product Networks P(X 1 , X 2 ) Sum (Mixture Model) 0.5 Weights (Priors) 0.3 0.2 Product (Compositions of Parts) Low-level Features 0.8 0.7 0.5 0.4 0.2 0.3 0.5 0.6 Naïve Bayes Mixture Model • 3 components X 1 X 1 X 2 X 2 Input Variables • 2 binary variables [Poon & Domingos , UAI’11, Friesen & Domingos , ICML’16]

  34. Sum-Product Networks • Learn conditional or joint distributions • Tractable partition function, exact inference [Poon & Domingos , UAI’11, Friesen & Domingos , ICML’16]

  35. Take-away III Template-based Method • Learn a set of template networks • Templates can decompose graphs • At inference time, form a single network • Adapts structure to topology of the environment The end-to-end model is called TopoNets . The SPN-based structured prediction method is called GraphSPN

  36. TopoNets: Template-based Method Learning

  37. TopoNets: Template-based Method Inference

  38. Experiments Dataset • Collected by mobile robot • 40 semantic maps on 4 floors • Built from laser-range and odometry data • Two place category setups • Cross-validation: • Trained on graphs from 3 floors • Tested on graphs from remaining floor

  39. Experiments Setup 𝑁 𝑢𝑓𝑡𝑢 = (𝑈, 𝒀, 𝒁) 𝑁 𝑢𝑓𝑡𝑢 : test semantic map 𝑈 : topological graph 𝒀 : local observations 𝒁 : semantic attributes 𝒁 = 𝒁 𝑞𝑚𝑏𝑑𝑓 ∪ 𝒁 𝑞𝑚𝑏𝑑𝑓ℎ𝑝𝑚𝑒𝑓𝑠

Recommend

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

TRAINING COURSE ON SOCIAL PROTECTION & FORMALIZATION TRINIDAD AND TOBAGO MARCH 15, 2017 INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION Design of the NIS Assistance from the

547 views • 23 slides
INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

Patterns of variation in the expression of case and agreement Andrs Brny Leiden University Centre for Linguistics 9 November 2019, BaSIS Workshop a.barany@hum.leidenuniv.nl INTRODUCTION Introduction 2/42 INTRODUCTION

1.37k views • 42 slides
Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures Definition and concepts Process and product properties Principles SE Approaches Motivation Software and the economy The economies of

1.03k views • 49 slides
Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1 https://www.cdc.gov/std/stats17/infographic.htm Introduction Epidemiology in Western Colorado CPHE Surveillance Data Report, May 2018 Chlamydia Epidemiology in Western

977 views • 39 slides
DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every meeting (2) Some ideas of diagrams for section 7.1 (Introduction) of TP. Giles Barr 25 Jan 2018, Pembroke College On DUNE, the reality is there is a

474 views • 10 slides
INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

ADVANCED COMPUTER SECURITY INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT IS SECURITY? A systems security policies describe What the system is supposed to do Store and provide access

479 views • 16 slides
Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout World Wide

387 views • 14 slides
14 Introduction Introduction Bad guys can put malware into Example: hosts

14 Introduction Introduction Bad guys can put malware into Example: hosts

Introduction Introduction source Encapsulation ISO/OSI reference model message M application segment H t H t M transport network datagram H n H n H t M presentation: allow applications to frame link H l H n H t M interpret

583 views • 4 slides
Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout

322 views • 8 slides
Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Introduction Zynq Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL Processing System Processor Peripherals Data Buses AXI Bus Conclusion BTE5380 - Embedded Systems Octobre 2014 Andreas Habegger

1.81k views • 30 slides
2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04 Project References S E 5 PART.1 SMILE5 Introduction M I L Introduction STORION-SMILE5 Residential Energy Storage System Inverter 5kW Output Input

1.06k views • 25 slides
Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an application server? Why use an application server? Course introduction Services provided by CICS How CICS applications are defined Scaling CICS to meet

1.84k views • 146 slides
Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic building blocks Node Links (relationship between nodes) Spatial information Network data

584 views • 9 slides
Introduction Introduction (1) Main argument of the paper: universities are not only

Introduction Introduction (1) Main argument of the paper: universities are not only

Le jugement des pairs comme outil de management des universits Peer-review as a management tool Christine Musselin (CSO, Sciences Po et CNRS) Avril 2013, Lirrsistible ascension du capitalisme acadmique Introduction Introduction (1)

590 views • 28 slides
Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Moreno Baricevic Gilberto Daz Axel Kohlmeyer Stefano Cozzini ULA ICTP CNR-IOM DEMOCRITOS Merida, VENEZUELA Trieste, ITALY Trieste, ITALY Introduction Introduction to storage and to storage and filesystems filesystems Introduction

1.19k views • 51 slides
JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of the brand new JA-100 alarm system Visions Basic architecture System components Setting Help us with final validations You are the FIRST ones

1.69k views • 111 slides
1 Introduction Introduction Communication System Baseband an adjective that describes

1 Introduction Introduction Communication System Baseband an adjective that describes

Introduction Introduction Physical Layer Mobile network Global ISP Provides the means to transmit bits Basic Concepts of from sender to receiver, Data Transmission Defines the mechanical, electrical, and Home network timing

463 views • 7 slides
Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical

Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical

INFOGR Computer Graphics Jacco Bikker - April-July 2015 - Lecture 1: Introduction Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical Details Assignments Field Study State of

869 views • 62 slides
Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction

Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction

Virtual and Augmented Reality Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction Who am I? What do I do here? 2 About me Short research bio: - 2014 - finished PhD at FMFI UK - 2014 2016 -

1.3k views • 39 slides
PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience

PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience

PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience for your presentation. It tells them what your presentation is about, why it is important, and finally, what to listen for in your speech. Get the

505 views • 3 slides
How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer

How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer

Introduction to Beamer Hafida Benhidour How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer Introduction to Beamer Hafida Benhidour Department of computer science King Saud University November

732 views • 49 slides
Shenzhen Cuilu jewelry Co., Ltd was founded in 1996 and its a large private enterprise

Shenzhen Cuilu jewelry Co., Ltd was founded in 1996 and its a large private enterprise

Introduction Introduction - - Brief Brief Introduction Introduction of of Cuilu Cuilu Group Group Introduction Introduction - - Brief Brief Introduction Introduction of of Cuilu Cuilu Group Group Shenzhen Cuilu jewelry Co., Ltd

147 views • 14 slides
Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

V olunteer C ommunications N etwork (V-C-N.org) Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org Academy Course - 2012-02.24-2208 262: Introduction to ICS-214 : V-C-N.org : Slide: 1 V olunteer C

735 views • 34 slides
Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

V olunteer C ommunications N etwork (V-C-N.org) Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org Academy Course - 2012-02.24-2208 262: Introduction to ICS-214 : V-C-N.org : Slide: 1 V olunteer C

558 views • 18 slides

More recommend