learning 2d linear dynamics in image space using neural
play

Learning 2D Linear Dynamics in Image Space Using Neural - PowerPoint PPT Presentation

Aug 30, 2022 •190 likes •342 views

Learning 2D Linear Dynamics in Image Space Using Neural Networks Jeffrey Mahler, Michael Laskey, John Schulman, Sergey Levine, Jeff Donahue, Sachin Pa:l,

  1. Learning ¡2D ¡Linear ¡Dynamics ¡in ¡ Image ¡Space ¡Using ¡Neural ¡Networks ¡ Jeffrey ¡Mahler, ¡Michael ¡Laskey, ¡John ¡Schulman, ¡Sergey ¡Levine, ¡Jeff ¡ Donahue, ¡Sachin ¡Pa:l, ¡Pieter ¡Abbeel, ¡and ¡Ken ¡Goldberg ¡ UC ¡Berkeley ¡EECS ¡ RSS ¡Workshop ¡on ¡Informa:on-­‑Based ¡Grasp ¡and ¡Manipula:on ¡Planning ¡ July ¡13 th , ¡2014 ¡ ¡

  2. Mo:va:on ¡ • Planning ¡for ¡grasping ¡and ¡ manipula:on ¡tasks ¡with ¡ unknown ¡objects ¡ • Only ¡sensor ¡data ¡and ¡ applied ¡controls ¡available ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 2 ¡

  3. Objec:ve ¡ • Predict ¡future ¡observa:ons ¡from ¡current ¡ observa:ons ¡and ¡control ¡inputs ¡ Current ¡Image ¡ Future ¡Image? ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 3 ¡

  4. Approach ¡ • Learn ¡the ¡parameters ¡of ¡a ¡dynamical ¡system: ¡ – Sensory ¡data ¡(e.g., ¡images) ¡ ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 4 ¡

  5. Approach ¡ • Learn ¡the ¡parameters ¡of ¡a ¡dynamical ¡system: ¡ – Sensory ¡data ¡(e.g., ¡images) ¡ – Learned ¡feature ¡representa:ons ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 4 ¡

  6. Approach ¡ • Learn ¡the ¡parameters ¡of ¡a ¡dynamical ¡system: ¡ – Sensory ¡data ¡(e.g., ¡images) ¡ – Learned ¡feature ¡representa:ons ¡ – Control ¡inputs ¡(e.g., ¡torques) ¡ ¡ ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 4 ¡

  7. Approach ¡ • Learn ¡the ¡parameters ¡of ¡a ¡dynamical ¡system: ¡ – Sensory ¡data ¡(e.g., ¡images) ¡ – Learned ¡feature ¡representa:ons ¡ – Control ¡inputs ¡(e.g., ¡torques) ¡ – Linear ¡dynamics ¡ ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 4 ¡

  8. Approach ¡ • Learn ¡the ¡parameters ¡of ¡a ¡dynamical ¡system: ¡ – Sensory ¡data ¡(e.g., ¡images) ¡ – Learned ¡feature ¡representa:ons ¡ – Control ¡inputs ¡(e.g., ¡torques) ¡ – Linear ¡dynamics ¡ ¡ – Neural ¡network-­‑generated ¡image ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 4 ¡

  9. Algorithm ¡ • Expecta:on ¡maximiza:on ¡for ¡(approximate) ¡ maximum-­‑likelihood ¡parameters ¡ … ¡ E-­‑Step ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 5 ¡

  10. Algorithm ¡ • Expecta:on ¡maximiza:on ¡for ¡(approximate) ¡ maximum-­‑likelihood ¡parameters ¡ … ¡ E-­‑Step ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 5 ¡

  11. Algorithm ¡ • Expecta:on ¡maximiza:on ¡for ¡(approximate) ¡ maximum-­‑likelihood ¡parameters ¡ … ¡ E-­‑Step ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 5 ¡

  12. Algorithm ¡ • Expecta:on ¡maximiza:on ¡for ¡(approximate) ¡ maximum-­‑likelihood ¡parameters ¡ … ¡ E-­‑Step ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 5 ¡

  13. Algorithm ¡ • Expecta:on ¡maximiza:on ¡for ¡(approximate) ¡ maximum-­‑likelihood ¡parameters ¡ … ¡ E-­‑Step ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 5 ¡

  14. Algorithm ¡ • Expecta:on ¡maximiza:on ¡for ¡(approximate) ¡ maximum-­‑likelihood ¡parameters ¡ … ¡ E-­‑Step ¡ M-­‑Step ¡ Neural ¡Network ¡ 5 ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡

  15. Discussion ¡ • Ini:al ¡results ¡ T = 0 1 2 3 4 5 6 7 8 9 ¡ Predicted ¡ Ground Truth ¡ • Future ¡Work ¡ – Natural ¡images ¡and ¡depth ¡images ¡ – Planning ¡a ¡manipula:on ¡task ¡ – Alterna:ve ¡dynamics ¡func:ons ¡ Mahler, ¡Laskey, ¡Abbeel, ¡Goldberg ¡ 6 ¡

Recommend

Image Space Embeddings and Generalized Convolutional Neural Networks Nate Strawn September

Image Space Embeddings and Generalized Convolutional Neural Networks Nate Strawn September

Image Space Embeddings and Generalized Convolutional Neural Networks Nate Strawn September 20th, 2019 Georgetown University Table of Contents 1. Introduction 2. Smooth Image Space Embeddings 3. Example: Dictionary Learning 4. Convolutional

530 views • 49 slides
The Space of Faces = + An image is a point in a high dimensional space An N x M image is

The Space of Faces = + An image is a point in a high dimensional space An N x M image is

The Space of Faces = + An image is a point in a high dimensional space An N x M image is a point in R NM 1 Linear Subspaces Dimensionality Reduction convert x into v 1 , v 2 coordinates What does the v 2 coordinate measure? - distance

106 views • 9 slides
Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Kalev

Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Kalev

+ Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Kalev Kask Linear classifiers (perceptrons) Linear Classifiers a linear classifier is a mapping which partitions feature space using a linear

511 views • 39 slides
1 The Space of Faces = + An image is a point in a high dimensional space An N x M image

1 The Space of Faces = + An image is a point in a high dimensional space An N x M image

1 The Space of Faces = + An image is a point in a high dimensional space An N x M image is a point in R NM 2 Linear Subspaces convert x into v 1 , v 2 coordinates What does the v 2 coordinate measure? - distance to line - use it for

810 views • 18 slides
Neural LMs Image: (Bengio et al, 03) One Hot Vectors Neural LMs (Bengio et al, 03)

Neural LMs Image: (Bengio et al, 03) One Hot Vectors Neural LMs (Bengio et al, 03)

Neural LMs Image: (Bengio et al, 03) One Hot Vectors Neural LMs (Bengio et al, 03) Low-dimensional Representations Learning representations by back-propagating errors Rumelhart, Hinton & Williams, 1986 A neural

1.64k views • 114 slides
Learning and transferring mid-level image representions using convolutional neural networks

Learning and transferring mid-level image representions using convolutional neural networks

Willow project-team Learning and transferring mid-level image representions using convolutional neural networks Maxime Oquab, Lon Bottou, Ivan Laptev, Josef Sivic 1 mardi 5 aot 14 Image classification (easy) Is there a car ? Source :

776 views • 61 slides
Adversarial Learning Bounds for Linear Classes and Neural Nets Understanding Adversarial Learning

Adversarial Learning Bounds for Linear Classes and Neural Nets Understanding Adversarial Learning

Adversarial Learning Bounds for Linear Classes and Neural Nets Understanding Adversarial Learning through Rademacher Complexity Pranjal Awasthi, Natalie Frank, Mehryar Mohri Google Research & Courant Institute August 14, 2020 1 / 17

397 views • 17 slides
Using Learning Dynamics to Understand Neural Network Generalisation Arnu Pretorius Supervisor:

Using Learning Dynamics to Understand Neural Network Generalisation Arnu Pretorius Supervisor:

Using Learning Dynamics to Understand Neural Network Generalisation Arnu Pretorius Supervisor: Dr. Steve Kroon (Computer Science ) Co-supervisor: Dr. Herman Kamper (EE Engineering + ) 20 October 2017 Maties Machine Learning, Stellenbosch

447 views • 11 slides
Neural Networks for Machine Learning Lecture 3a Learning the weights of a linear neuron Geoffrey

Neural Networks for Machine Learning Lecture 3a Learning the weights of a linear neuron Geoffrey

Neural Networks for Machine Learning Lecture 3a Learning the weights of a linear neuron Geoffrey Hinton with Nitish Srivastava Kevin Swersky Why the perceptron learning procedure cannot be generalised to hidden layers The perceptron

738 views • 34 slides
Low-Dimensional Dynamics of Encoding and Learning in Recurrent Neural Networks Stefan Horoi 1,2 ,

Low-Dimensional Dynamics of Encoding and Learning in Recurrent Neural Networks Stefan Horoi 1,2 ,

Low-Dimensional Dynamics of Encoding and Learning in Recurrent Neural Networks Stefan Horoi 1,2 , Victor Geadah 1,2 , Guy Wolf 1,2,* and Guillaume Lajoie 1,2,* * Equal senior authorship contributions 1 Department of Mathematics and Statistics 2

290 views • 14 slides
Neural networks and Reinforcement learning review CS 540 Yingyu Liang Neural Networks Outline

Neural networks and Reinforcement learning review CS 540 Yingyu Liang Neural Networks Outline

Neural networks and Reinforcement learning review CS 540 Yingyu Liang Neural Networks Outline Building unit: neuron Linear perceptron Non-linear perceptron The power/limit of a single perceptron Learning of a single

479 views • 27 slides
CSE 152: Computer Vision Hao Su Lecture 7: Neural Networks Review of Filters: From Linear to

CSE 152: Computer Vision Hao Su Lecture 7: Neural Networks Review of Filters: From Linear to

CSE 152: Computer Vision Hao Su Lecture 7: Neural Networks Review of Filters: From Linear to Non-linear Image filtering (Linear case) 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

1.14k views • 90 slides
Linear Regression 4/14/17 Hypothesis Space Supervised learning For every input in the data

Linear Regression 4/14/17 Hypothesis Space Supervised learning For every input in the data

Linear Regression 4/14/17 Hypothesis Space Supervised learning For every input in the data set, we know the output Regression Outputs are continuous A number, not a category label The learned model: A linear function mapping

400 views • 12 slides
Convolutional Neural Networs for Image Classification Marcello Pelillo University of Venice,

Convolutional Neural Networs for Image Classification Marcello Pelillo University of Venice,

Convolutional Neural Networs for Image Classification Marcello Pelillo University of Venice, Italy Image and Video Understanding a.y. 2018/19 The Age of Deep Learning The Deep Learning Philosophy Learn a feature hierarchy all

1.14k views • 54 slides
Learning From Data Lecture 10 Nonlinear Transforms The Z -space Polynomial transforms Be

Learning From Data Lecture 10 Nonlinear Transforms The Z -space Polynomial transforms Be

Learning From Data Lecture 10 Nonlinear Transforms The Z -space Polynomial transforms Be careful M. Magdon-Ismail CSCI 4100/6100 recap: The Linear Model linear in x : gives the line/hyperplane separator s = w t x linear in w : makes

196 views • 18 slides
recap: Linear Models, RBFs, Neural Networks Linear Model with Nonlinear Transform Neural Network

recap: Linear Models, RBFs, Neural Networks Linear Model with Nonlinear Transform Neural Network

recap: Linear Models, RBFs, Neural Networks Linear Model with Nonlinear Transform Neural Network k -RBF-Network d m k Learning From Data h ( x ) = w 0 + w j j ( x ) h ( x ) = w 0 +

336 views • 5 slides
Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural

Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural

Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural circuits David Sussillo with Valerio Mante and Bill Newsome Table of contents Introduc(on Training recurrent neural networks(RNNs) Understanding how

1.57k views • 105 slides
If It's Not Deterministic, It's Crap: Deterministic Machine Learning and Molecular Dynamics

If It's Not Deterministic, It's Crap: Deterministic Machine Learning and Molecular Dynamics

If It's Not Deterministic, It's Crap: Deterministic Machine Learning and Molecular Dynamics Spoilers GPUs/FPGAs/CPUs/ASICs ad nauseum AMBER Molecular Dynamics Determinism Matters Multi-GPU Servers Neural Networks

1.04k views • 68 slides
Image Space Occlusion Culling Image Space Occlusion Culling 1 Hudson et al, SoCG 97 Occluder A

Image Space Occlusion Culling Image Space Occlusion Culling 1 Hudson et al, SoCG 97 Occluder A

Image Space Occlusion Culling Image Space Occlusion Culling 1 Hudson et al, SoCG 97 Occluder A Viewpoint umbra B C 2 What Methods are Called Image-Space? Those where the decision to cull or render is done after projection (in image

566 views • 24 slides
Vision and Language Learning with Graph Neural Networks Linchao Zhu 22 Apr, 2020 Recognition,

Vision and Language Learning with Graph Neural Networks Linchao Zhu 22 Apr, 2020 Recognition,

Vision and Language Learning with Graph Neural Networks Linchao Zhu 22 Apr, 2020 Recognition, LEarning, Reasoning UTS CRICOS 00099F Overview RNNs for Image Captioning Transformer for Image Captioning Graph Network for Visual

385 views • 19 slides
Extending Unsupervised Neural Image Compression With Supervised Multitask Learning David Tellez,

Extending Unsupervised Neural Image Compression With Supervised Multitask Learning David Tellez,

Extending Unsupervised Neural Image Compression With Supervised Multitask Learning David Tellez, Diederik Hoppener, Cornelis Verhoef, Dirk Grunhagen, Pieter Nierop, Michal Drozdzal, Jeroen van der Laak, Francesco Ciompi david.tellezm@gmail.com

302 views • 17 slides
Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference ICTP, Trieste, Jan 29-31 2018 Deep Neural Networks When properly trained, deep NN are capable of learning the complex functional dependence of

442 views • 17 slides
Non-linear hypotheses Machine Learning Non-linear Classification x 2 x 1 size # bedrooms #

Non-linear hypotheses Machine Learning Non-linear Classification x 2 x 1 size # bedrooms #

Neural Networks: Representation Non-linear hypotheses Machine Learning Non-linear Classification x 2 x 1 size # bedrooms # floors age Andrew Ng What is this? You see this: But the camera sees this: Andrew Ng Computer Vision: Car

470 views • 42 slides
Machine Learning for Sequence Learning Learning in an All-Subsequence Space Severin Gsponer,

Machine Learning for Sequence Learning Learning in an All-Subsequence Space Severin Gsponer,

Machine Learning for Sequence Learning Learning in an All-Subsequence Space Severin Gsponer, Georgiana Ifrim, Barry Smyth January 20, 2016 Outline Background Linear Classifiers for Sequences SEQL Approach Contribution Future

298 views • 18 slides

More recommend