distributed multi sensor data fusion for 3d mapping ai
play

Distributed multi-sensor data fusion for 3D mapping & AI - PowerPoint PPT Presentation

Mar 23, 2024 •161 likes •310 views

Prof. Rozenn Dahyot Associate Professor in Statistics School of Computer Science & Statistics Trinity College Dublin https://www.scss.tcd.ie/Rozenn.Dahyot/ Distributed multi-sensor data fusion for 3D mapping & AI applications

  1. Prof. Rozenn Dahyot Associate Professor in Statistics School of Computer Science & Statistics Trinity College Dublin https://www.scss.tcd.ie/Rozenn.Dahyot/ Distributed multi-sensor data fusion for 3D mapping & AI applications https://www.meetup.com/Machine-Learning-Dublin/events/251878496/ #MLDublin meets Valeo @ DogPatch Labs 2018/06/25 The ADAPT Centre is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.

  2. Why do we need a Digital twin of the world? www.adaptcentre.ie • City planning (Future cities) • Virtual visits • Games, education • Data visualisation • Navigation (cars, drones) • Creating data for training AI ! • Driverless cars • Autonomous drones • Asset management • Environment monitoring (e.g. vegetation, Multiple source of Data pollution, ..) • Drones • Disaster prevention & emergency response • Social media (floods, earthquakes,…) • Google Street view • OpenStreetMap • Satellite imagery • Lidar • …

  3. Importance of virtual environments in AI www.adaptcentre.ie Closing the Simulation-to-Reality Gap for Deep Robotic Learning https://arxiv.org/abs/1709.07857 (2017, Google Brain)

  4. Digital Twin of the world using online data www.adaptcentre.ie Google Street View navigation TCD AI: using GSV imagery and OSM in game engine EU FP7 GRAISearch: https://www.youtube.com/channel/UCY_bkYqwO4K_jZzodE-LP5Q • Social Media based 3D Visual Popularity A. Bulbul and R. Dahyot, Computer & Graphics (2017) DOI:10.1016/j.cag.2017.01.005 • Populating Virtual Cities using Social Media A. Bulbul and R. Dahyot, Computer Animation and Virtual Worlds journal (2016) DOI:10.1002/cav.1742

  5. Digital Twin of the world with Drones www.adaptcentre.ie Trinity College Dublin Drone Survey Dataset J. Byrne, J. Connelly, J. Su, V. Krylov, M. Bourke, D. Moloney & R. Dahyot, Tech Report and Dataset (2017). http://www.tara.tcd.ie/handle/2262/81836 Applications of the VOLA Format for 3D Data Knowledge Discovery. Byrne, Jonathan & al (2017). International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery Interactive Demo Virtual visit at https://www.scss.tcd.ie/Rozenn.Dahyot/TCDCampusWebGL2/

  6. Digital Twin of the world with Semantic labelling www.adaptcentre.ie EU project VARCITY (Variation & the City) (2012-2017, PI: Luc S. J. Van Gool) https://varcity.ethz.ch/ Target . Automatic discovery and geotagging of recurring stationary objects from Street Level imagery: • Utility companies’ assets • Street signage • Vegetation Info @ https://www.adaptcentre.ie/case-studies/aimapit " Automatic Discovery and Geotagging of Objects from Street View Imagery ". V. A. Krylov, E. Kenny, R. Dahyot. Remote Sensing (2018). https://doi.org/10.3390/rs10050661

  7. Digital Twin of the world with sensor data fusion www.adaptcentre.ie  Vision module: fully convolutional neural networks (FCNN) for semantic segmentation and monocular depth estimation Mapillary Vistas Dataset Average processing speed of 200km/h on desktop PC  Sensor fusion module: Markov Random Fields (MRF) V. A. Krylov, R. Dahyot. "Object Geolocation using MRF-based Multi-sensor Fusion", IEEE International Conference on Image Processing (ICIP 2018 - to appear )

  8. Bringing AI on the edge www.adaptcentre.ie https://www.bonseyes.com/ BONSEYES: Platform for Open Development of Systems of Artificial Intelligence T. Llewellynn et al., ACM International Conference on Computing Frontiers 2017 DOI:10.1145/3075564.3076259

  9. Mathematics of AI www.adaptcentre.ie y machine g(y) x f(x) • Supervised scenario: When observations are paired • (𝒋) (𝒋) • Unsupervised scenario: When observations are not paired (𝒋) (𝒌) •

  10. Mathematics of AI www.adaptcentre.ie Example f 𝑦 ∈ ℝ � and g 𝑧 ∈ ℝ � 𝑧 � � = 𝜒 � � 𝑦 𝑧 � � = 𝜒 � � 𝑦 Shape Registration with Directional Data M. Grogan & R. Dahyot, Pattern Recognition (2018) DOI:10.1016/j.patcog.2018.02.021

  11. Mathematics of AI www.adaptcentre.ie Video credit @gabrielpeyre https://youtu.be/FLhabslT4y4 Automated Colour Grading using Colour Distribution Transfer F. Pitie, A. Kokaram & R. Dahyot, Computer Vision and Image Understanding (2007) DOI:10.1016/j.cviu.2006.11.011 Robust Registration of Gaussian Mixtures for Colour Transfer User Interaction for Image Recolouring using L2 M. Grogan and R. Dahyot M. Grogan, R. Dahyot and A. Smolic, in Conference on Visual Media arxiv.org/abs/1705.06091 (2017) Production (CVMP), London, December 2017 DOI:10.1145/3150165.3150171

  12. Expertise for designing intelligent systems www.adaptcentre.ie Computer Vision Signal processing Computer Graphics Machine Learning Statistics Engineering Algorithms Mathematics Environment, Energy, Geospatial Data Analytics Smart cities,… …

  13. www.adaptcentre.ie Questions? Thank you! Prof. Rozenn Dahyot & Co https://www.scss.tcd.ie/Rozenn.Dahyot/Research.html#Team

Recommend

Distributed Fusion in Sensor Distributed Fusion in Sensor Networks Networks Jie Gao Computer

Distributed Fusion in Sensor Distributed Fusion in Sensor Networks Networks Jie Gao Computer

Distributed Fusion in Sensor Distributed Fusion in Sensor Networks Networks Jie Gao Computer Science Department Stony Brook University Papers Papers [Xiao04] Lin Xiao, Stephen Boyd, Fast Linear Iterations for Distributed Averaging ,

659 views • 50 slides
ASYNCHRONOUS MULTI-SENSOR FUSION FOR 3D MAPPING AND LOCALIZATION Patrick Geneva, Kevin Eckenhoff,

ASYNCHRONOUS MULTI-SENSOR FUSION FOR 3D MAPPING AND LOCALIZATION Patrick Geneva, Kevin Eckenhoff,

ASYNCHRONOUS MULTI-SENSOR FUSION FOR 3D MAPPING AND LOCALIZATION Patrick Geneva, Kevin Eckenhoff, and Guoquan Huang Presented by Yulin Yang September 24, 2017 Department of Mechanical Engineering, University of Delaware, USA MOTIVATIONS

235 views • 10 slides
Introduction to Sensor Data Fusion Methods and Applications Last lecture: Why Sensor Data

Introduction to Sensor Data Fusion Methods and Applications Last lecture: Why Sensor Data

Introduction to Sensor Data Fusion Methods and Applications Last lecture: Why Sensor Data Fusion? Motivation, general context Discussion of examples Today: Steep climb to a first algorithm. oral examination: 6 credit points

1.02k views • 56 slides
Introduction to Sensor Data Fusion Methods and Applications Last lecture: Why Sensor Data

Introduction to Sensor Data Fusion Methods and Applications Last lecture: Why Sensor Data

Introduction to Sensor Data Fusion Methods and Applications Last lecture: Why Sensor Data Fusion? Motivation, general context Discussion of examples Today: Steep climb to a first algorithm. oral examination: 6 credit points

995 views • 42 slides
Sequential Monte Carlo Methods Particle Filter Martin Ulmke Head of Research Group

Sequential Monte Carlo Methods Particle Filter Martin Ulmke Head of Research Group

Sequential Monte Carlo Methods Particle Filter Martin Ulmke Head of Research Group Distributed Sensor Systems Sensor Data and Information Fusion Fraunhofer FKIE Lecture: Introduction to Sensor Data Fusion 12 December 2018 Inhalt

1.27k views • 74 slides
Software-Defined Multi-Spectral Imaging System Image and Information Fusion Experiments for

Software-Defined Multi-Spectral Imaging System Image and Information Fusion Experiments for

Software-Defined Multi-Spectral Imaging System Image and Information Fusion Experiments for Aviation and Marine Sensor Networks Sam Siewert, AIAA SciTech 2017 - January 10, 2017 Fusion of Networked Sensor Systems Goals and Objectives

429 views • 32 slides
vil : Dri Drift ft with th De Devi Security of Multi-Sensor Fusion based Localization in

vil : Dri Drift ft with th De Devi Security of Multi-Sensor Fusion based Localization in

vil : Dri Drift ft with th De Devi Security of Multi-Sensor Fusion based Localization in High-Level Autonomous Driving under GPS Spoofing Junjie Shen , Jun Yeon Won, Zeyuan Chen, Qi Alfred Chen ASGuard A utonomous S ystem Gu Guard

394 views • 20 slides
On-line Fusion of Functional Knowledge Within Distributed Sensor Networks Dominik Fisch, Bernhard

On-line Fusion of Functional Knowledge Within Distributed Sensor Networks Dominik Fisch, Bernhard

On-line Fusion of Functional Knowledge Within Distributed Sensor Networks Dominik Fisch, Bernhard Sick Intelligent Embedded Systems Group University of Kassel www.ies-research.de Final Colloquium of the DFG Priority Program 1183 Organic

575 views • 21 slides
On-line Fusion of Functional Knowledge Within Distributed Sensor Networks Dominik Fisch, Bernhard

On-line Fusion of Functional Knowledge Within Distributed Sensor Networks Dominik Fisch, Bernhard

On-line Fusion of Functional Knowledge Within Distributed Sensor Networks Dominik Fisch, Bernhard Sick Research Group Computationally Intelligent Systems University of Applied Sciences Deggendorf www.cis-research.de 11th Colloquium of

435 views • 22 slides
Exploration of a Multi-Sensor Approach for the Detection and Mapping of Coal Mine Fires in the

Exploration of a Multi-Sensor Approach for the Detection and Mapping of Coal Mine Fires in the

Office of Surface Mining Reclamation and Enforcement Geospatial Information Services Exploration of a Multi-Sensor Approach for the Detection and Mapping of Coal Mine Fires in the United States Alex Sivitskis AmeriCorps Member GIS and Remote

214 views • 20 slides
Efficient Algorithm for Serial Data Fusion in Wireless Sensor

Efficient Algorithm for Serial Data Fusion in Wireless Sensor

Efficient Algorithm for Serial Data Fusion in Wireless Sensor Networks A. Mostefaoui 1 , M. Melkemi 2 and A. Boukerche 3 1 University of Franche-Comt, France

701 views • 51 slides
Figure 1: Visual representation of M 3 Fusion . II. D ATA The study was carried out on Reunion

Figure 1: Visual representation of M 3 Fusion . II. D ATA The study was carried out on Reunion

M 3 Fusion : A Deep Learning Architecture for Multi- { Scale/Modal/Temporal } satellite data fusion P. Benedetti, D. Ienco, R. Gaetano, K. Os e, R. Pensa and S. Dupuy Abstract Modern Earth Observation systems provide sensing data at different

107 views • 9 slides
Data Fusion in Sensor Networks Hugh Durrant-Whyte ARC Federation Fellow, Research Director ARC

Data Fusion in Sensor Networks Hugh Durrant-Whyte ARC Federation Fellow, Research Director ARC

Data Fusion in Sensor Networks Hugh Durrant-Whyte ARC Federation Fellow, Research Director ARC Centre of Excellence for Autonomous Systems The University of Sydney, Australia hugh@cas.edu.au ARC=Australian version of NSF Slide 1 IPAM Sensor

805 views • 36 slides
Data Assimilation and Detection in Multi-Sensor & Multi-Scale Environments N. Sri

Data Assimilation and Detection in Multi-Sensor & Multi-Scale Environments N. Sri

Data Assimilation and Detection in Multi-Sensor & Multi-Scale Environments N. Sri Namchchivaya Andrea Barreiro, Shanshan Liu, Kristjan Onu, Jun Park Pete Sauer, and Richard Sowers Department of Aerospace Engineering University of Illinois

379 views • 34 slides
A method of personal positioning based on sensor data fusion of wearable camera and

A method of personal positioning based on sensor data fusion of wearable camera and

A method of personal positioning based on sensor data fusion of wearable camera and self-contained sensors. *1 *2 *1 Makasakatsu Kourogi and Takeshi Kurata *1 National Institute of Advanced Industrial Science and Technology (AIST) *2 HIT

483 views • 24 slides
Multi-scale Geometric Summaries for Similarity-based Upstream Sensor Fusion Christopher Tralie,

Multi-scale Geometric Summaries for Similarity-based Upstream Sensor Fusion Christopher Tralie,

Multi-scale Geometric Summaries for Similarity-based Upstream Sensor Fusion Christopher Tralie, Paul Bendich, John Harer Duke University, ECE / Math 3/6/2019 Christopher Tralie, Paul Bendich, John Harer Multi-scale Geometric Summaries for

292 views • 28 slides
How to Get a Map? Basic Mapping Task By hand? Slow, expensive and imprecise

How to Get a Map? Basic Mapping Task By hand? Slow, expensive and imprecise

5/5/20 What is Mapping? Mapping and SLAM: A (High-Level) Survey Building a mo model of the robots environme ment Based on sensor data obtained in that environment Sebastian Thrun, 2002 Many different environment representations

105 views • 7 slides
Blue Sensors, Sensor Correlation, and Alert Fusion Al Valdes 10/4/00 Event Processing,

Blue Sensors, Sensor Correlation, and Alert Fusion Al Valdes 10/4/00 Event Processing,

Blue Sensors, Sensor Correlation, and Alert Fusion Al Valdes 10/4/00 Event Processing, Monitoring, and Correlation Note: Event counts Represent 6 weeks of Meta Alerts Meta Alert CSL Data (500) Fusion Other Monitors Alerts (1.5K)

842 views • 15 slides
Prize-Collecting Data Fusion for Cost-Performance Tradeoff in Distributed Inference Anima

Prize-Collecting Data Fusion for Cost-Performance Tradeoff in Distributed Inference Anima

Prize-Collecting Data Fusion for Cost-Performance Tradeoff in Distributed Inference Anima Anandkumar 1 Meng Wang 1 Lang Tong 1 Ananthram Swami 2 1 School of ECE, Cornell University, Ithaca, NY. 2 Army Research Laboratory, Adelphi MD. INFOCOM 2009

671 views • 51 slides
Multi- -dimensional Data and dimensional Data and Spatial Range Spatial Range Multi Query in

Multi- -dimensional Data and dimensional Data and Spatial Range Spatial Range Multi Query in

Multi- -dimensional Data and dimensional Data and Spatial Range Spatial Range Multi Query in Sensor Networks Query in Sensor Networks Jie Gao Computer Science Department Stony Brook University 1 Papers Papers [Li03a] X. Li, Y. J.

615 views • 40 slides
Data Collection Infrastructure for Location- Location-Unaware Sensor Networks Distributed coding

Data Collection Infrastructure for Location- Location-Unaware Sensor Networks Distributed coding

Data Collection Infrastructure for Location- Location-Unaware Sensor Networks Distributed coding protocols for data storage Silvija Kokalj-Filipovic Predrag Spasojevic Roy Yates Talk OutLine Data Collection from a

660 views • 20 slides
Data Collection and Aggregation 1 Challenges: data Data type: numerical sensor readings.

Data Collection and Aggregation 1 Challenges: data Data type: numerical sensor readings.

Data Collection and Aggregation 1 Challenges: data Data type: numerical sensor readings. Rich and massive data, spatially distributed and correlated. Data dynamics: data streaming and aging. Uncertainty, noise, erroneous

1.45k views • 99 slides
Chapter 5 Generating Linked Data in Real- time from Sensor Data Streams NIKOLAOS KONSTANTINOU

Chapter 5 Generating Linked Data in Real- time from Sensor Data Streams NIKOLAOS KONSTANTINOU

Chapter 5 Generating Linked Data in Real- time from Sensor Data Streams NIKOLAOS KONSTANTINOU DIMITRIOS-EMMANUEL SPANOS Materializing the Web of Linked Data Outline Introduction Fusion The Data layer Rule-based Reasoning Complete Example

1.02k views • 83 slides
Motion Interpolation & Sensor Fusion Final presentation Lennart Bastian Antoine Keller Sofia

Motion Interpolation & Sensor Fusion Final presentation Lennart Bastian Antoine Keller Sofia

Introduction Background Interpolation and Approximation Interpolation and Approximation Optimization Sensor Fusion Motion Interpolation & Sensor Fusion Final presentation Lennart Bastian Antoine Keller Sofia Morales Santiago TUM

821 views • 44 slides

More recommend