Persistent self-supervised learning principle: from stereo to - PowerPoint PPT Presentation

Dec 30, 2023 •359 likes •503 views

Persistent self-supervised learning principle: from stereo to monocular vision for obstacle avoidance by Kevin van Hecke, Guido de Croon, Laurens van der Maaten, Daniel Hennes, and Dario Izzo presented by Mike Kalaitzakis What is Persistent

Persistent self-supervised learning principle: from stereo to monocular vision for obstacle avoidance by Kevin van Hecke, Guido de Croon, Laurens van der Maaten, Daniel Hennes, and Dario Izzo presented by Mike Kalaitzakis
What is Persistent Self-Supervised Learning? ● What is Self- Supervised Learning ● What makes this persistent ● Differences and Similarities between other ML methods ● What are the advantages https://xkcd.com/1838/
What is Persistent Self-Supervised Learning? ● Self-Supervised Learning is a mechanism that uses a trusted sensor cue for training to recognize a complementary sensor cue
What is Persistent Self-Supervised Learning? ● Persistent Self- Supervised Learning is a Self- Supervised Learning method where the goal is to be able to replace the original trusted sensor cue if necessary
What is Persistent Self-Supervised Learning? ● Shares some similarities with Unsupervised Learning (UL) and Learning from Demonstration (LfD) ● Does not need labeled data like UL ● Does not need an external “teacher” as LfD ● Does not need a reward function and many trial-and-error sessions as Reinforcement Learning ● Can autonomously decide which sensor cue to use or if re-training is needed
Going from Stereo Vision to Monocular Vision ● The disparity is the difference between the same image feature in the two images of a calibrated stereo pair ● It is used to calculate the distance of an object from the cameras Stefano Mattoccia, University of Bolona ● A 4-gram custom made stereo camera is used (128x96 pixel, 10fps) ● The PSSL tries to estimate the average disparity and not the whole map TU Delft, MAV-Lab
Visual Bag of Words (VBoW) and Textons ● In VboW a complex image is split in small image patches ● A dictionary is created by clustering the patches called textons ● When processing an image each patch is compared to the dictionary creating a texton occurrence histogram ● Both intensity and gradient textons are used
Why Visual Bag of Words? ● Short Answer: Computationally efficient and limited computational power platform (Parrot AR Drone 2, 1GHz ARM cortex A8, 128MB RAM) ● Compared to the best available CNN the results were comparable and depended on the dataset
System Overview ● Binary classification ● Determine the threshold (t λ ) from the ROC curve ● The threshold is chosen to minimize the probability of collision ● Since the data cannot be assumed identically and independently distributed a Markov process was used to model the system ● PSSL limitation factor: By design the system will encounter very few occasions where obstacles are detected
System Overview
Simulation Results ● SmartUAV simulation environment was used to choose regression method and learning scheme ● kNN, Linear regression, Gaussian process and Neural Networks were tested ● “Cold Turkey”, DAgger and “Training wheels” approaches to learning were tested
Experiment Results
ISS Video

Recommend

Depth from Stereo Dominic Cheng February 7, 2018 Agenda 1. Introduction to stereo 2.

Depth from Stereo Dominic Cheng February 7, 2018 Agenda 1. Introduction to stereo 2. Efficient Deep Learning for Stereo Matching (W. Luo, A. Schwing, and R. Urtasun. In CVPR 2016.) 3. Cascade Residual Learning: A Two-stage Convolutional

647 views • 30 slides

Learning frameworks Self-supervised learning: (Auto)encoder networks Supervised learning Network

Learning frameworks Self-supervised learning: (Auto)encoder networks Supervised learning Network must copy inputs to outputs through Assumes environment specifies correct output (targets) for each input a bottleneck (fewer hidden units)

407 views • 5 slides

Self-Supervised Feature Learning by Learning to Spot Artifacts Wonbin Kim Self-Supervised

Self-Supervised Feature Learning by Learning to Spot Artifacts Wonbin Kim Self-Supervised Learning To exploit different labelings that are freely available besides or within visual data, and to use them as intrinsic reward signals to

387 views • 24 slides

Efficient Deep Learning for Stereo Matching Wenjie Luo, Alex Schwing and Raquel Urtasun W. Luo

Efficient Deep Learning for Stereo Matching Wenjie Luo, Alex Schwing and Raquel Urtasun W. Luo et.al. (UofT) Stereo Matching 1 / 6 Stereo Estimation Desired Properties: Good enough to detect obstacles precisely Fast: real time Robust to:

119 views • 8 slides

Stacking for supervised learning Stacking for supervised learning Niall Rooney, NIKEL,

Stacking for supervised learning Stacking for supervised learning Niall Rooney, NIKEL, University of Ulster 1 Ensemble learning Ensemble learning l Postulate multiple hypotheses to explain the data l Shortcomings of single model learning

681 views • 19 slides

Machine Learning for NLP Supervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Supervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain Sciences University of Trento 1 Supervised learning Supervised: you know the result of the task you want to perform. Supervised learning mostly

832 views • 61 slides

PCA CS 446 Supervised learning So far, weve done supervised learning: Given (( x i , y i )) ,

PCA CS 446 Supervised learning So far, weve done supervised learning: Given (( x i , y i )) , find f with f ( x i ) y i . k -nn, decision trees, . . . 1 / 18 Supervised learning So far, weve done supervised learning: Given (( x i , y

1.01k views • 56 slides

Today Recap: epipolar constraint Stereo image rectification Stereo: Stereo

3/29/2011 Today Recap: epipolar constraint Stereo image rectification Stereo: Stereo solutions Correspondence and Calibration Computing correspondences Non-geometric stereo constraints Mon, March 28 Calibration

342 views • 13 slides

3D Vision: Stereo Marc Pollefeys, Torsten Sattler Spring 2016

3D Vision: Stereo Marc Pollefeys, Torsten Sattler Spring 2016 http://cvg.ethz.ch/teaching/3dvision/ Dense Correspondence / Stereo Tsukuba dataset http://cat.middlebury.edu/stereo/ Stereo Standard stereo geometry Stereo matching

478 views • 35 slides

Semi-Supervised Learning Maria-Florina Balcan 03/30/2015 Readings: Semi-Supervised Learning.

Semi-Supervised Learning Maria-Florina Balcan 03/30/2015 Readings: Semi-Supervised Learning. Encyclopedia of Machine Learning. Jerry Zhu, 2010 Combining Labeled and Unlabeled Data with Co- Training. Avrim Blum, Tom Mitchell. COLT

953 views • 45 slides

RECSM Summer School: Machine Learning for Social Sciences Session 1.3: Supervised Learning and

RECSM Summer School: Machine Learning for Social Sciences Session 1.3: Supervised Learning and Model Accuracy Reto West Department of Political Science and International Relations University of Geneva 1 Supervised Learning Supervised

436 views • 42 slides

CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression What is supervised learning? Supervised learning is the process of trying to infer from labeled data the underlying function that produced the labels

784 views • 33 slides

CSE 158 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

751 views • 25 slides

CSE 258 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression

CSE 258 Lecture 1.5 Web Mining and Recommender Systems Supervised learning Regression What is supervised learning? Supervised learning is the process of trying to infer from labeled data the underlying function that produced the labels

599 views • 33 slides

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering and nearest neighbour classifiers. Justin Pearson 1 2020 1 http://user.it.uu.se/~justin/Teaching/MachineLearning/index.html 1 / 27 Supervised

813 views • 27 slides

1 Basic Stereo Derivations Correspondence It is fundamentally ambiguous, even with stereo

Stereo Vision Reading: Chapter 11 Stereo matching computes depth from two or more images Subproblems: Calibrating camera positions. Finding all corresponding points (hardest part) Computing depth or surfaces. Slide credits

107 views • 7 slides

Web Mining and Recommender Systems Supervised learning Regression Learning Goals Introduce

Web Mining and Recommender Systems Supervised learning Regression Learning Goals Introduce the concept of Supervised Learning Understand the components (inputs and outputs) of supervised learning problems Introduce linear

1.36k views • 122 slides

3D Photography: Stereo Matching Kevin Kser, Marc Pollefeys Spring 2012

3D Photography: Stereo Matching Kevin Kser, Marc Pollefeys Spring 2012 http://cvg.ethz.ch/teaching/2012spring/3dphoto/ Stereo & Multi-View Stereo Tsukuba dataset http://cat.middlebury.edu/stereo/ Stereo Standard stereo geometry

1.63k views • 124 slides

Weakly Supervised Classification Weakly Supervised Classification and Robust Learning and Robust

IIT-H and RIKEN-AIP Joint Workshop on March 15, 2019 Machine Learning and Applications, Hyderabad, India Weakly Supervised Classification Weakly Supervised Classification and Robust Learning and Robust Learning --- Overview of Our Recent

1.06k views • 25 slides

THE SUPERVISED LEARNING PROBLEM THE SUPERVISED LEARNING PROBLEM Matthieu R Bloch January 7, 2020

THE SUPERVISED LEARNING PROBLEM THE SUPERVISED LEARNING PROBLEM Matthieu R Bloch January 7, 2020 1 WHY ML ? WHY ML ? Traditional engineering is top-down We use fundamental principles (mathematics, physics) to build models and abstractions

243 views • 13 slides

WHY SUPERVISED LEARNING MAY WORK WHY SUPERVISED LEARNING MAY WORK Matthieu R Bloch Tuesday,

WHY SUPERVISED LEARNING MAY WORK WHY SUPERVISED LEARNING MAY WORK Matthieu R Bloch Tuesday, January 14, 2020 1 LOGISTICS LOGISTICS TAs and Office hours Monday: Mehrdad (TSRB) - 2pm-3:15pm Tuesday: TJ (VL) - 1:30pm - 2:45pm Wednesday:

378 views • 13 slides

Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site

CS11-747 Neural Networks for NLP Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site https://phontron.com/class/nn4nlp2018/ Supervised, Unsupervised, Semi-supervised Most models handled here are supervised learning

670 views • 32 slides

Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site

CS11-747 Neural Networks for NLP Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site https://phontron.com/class/nn4nlp2020/ Supervised, Unsupervised, Semi-supervised Most models handled here are supervised learning

580 views • 39 slides

Supervised Learning: The Setup Machine Learning 1 Last lecture We saw What is learning?

Supervised Learning: The Setup Machine Learning 1 Last lecture We saw What is learning? Learning as generalization The badges game 2 This lecture More badges Formalizing supervised learning Instance space and features

1.04k views • 80 slides