Unsupervised Learning of 3D Structure from Images - PowerPoint PPT Presentation

Mar 21, 2023 •31 likes •191 views

Unsupervised Learning of 3D Structure from Images https://goo.gl/8pGKOG October 21, 2016 Objective Find a statistical representation of data (images or volume data) that is generalizable to new unseen data (new views of an object). We want to

Unsupervised Learning of 3D Structure from Images https://goo.gl/8pGKOG October 21, 2016
Objective Find a statistical representation of data (images or volume data) that is generalizable to new unseen data (new views of an object). We want to infer a 3D representation (polyhedral mesh or dense volumes of voxels) from 2D images that we can render new instances of and reason about. Other approaches rely heavily on visual feature engineering, the paper’s approach is to learn to infer 3D representation directly from 2D images.
Applications 3D representations allows easier downstream properties of objects to use for: Physical reasoning of objects including interaction, and navigation, ● Scene completion, ● ● Denoising, ● Compression, and Generative virtual reality ●
Challenges 1. Inherently ill-posed: an infinite number of possible 3D structures to give rise to a particular 2D observation. Learn statistical models to find most likely representation ○ 2. Inference is intractable: mapping image pixels to 3D representations, handling multi-modality of representations 3. Unclear how to best represent 3D structures ○ Polyhedral mesh and dense volume of voxels are used in the paper
3D Representations Dense Volume Polyhedral of Voxels Mesh
Model
Conditional Generative Model Given observed volume or image x and a context c , infer 3D representation h , and render 2D image from either a Neural Net or OpenGL engine. Context c is either: nothing, object class label, or one or more views of a scene. The generative models with latent variables describe probability densities p( x ) though marginalization of the set of latent variables z . Bound Marginal likelihood p(x) by
Architectures Applied recent work on sequential generative models (similar to RNN) to capture complex distributions of 3D structures. Sequentially transform independent Gaussian latent variables into refinements of h (the “canvas”).
Architectures
Neural Framework
Results - Mesh
Results - Mesh
Results - Volume
Results - Volume
Conclusion The advantage of forcing inference into a specific format is that the NN can be chained together to perform various tasks.
Questions?

Recommend

Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

UNSUPERVISED LEARNING IN R Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of clustering finding groups of homogeneous items Next up, dimensionality reduction Find structure in features

904 views • 29 slides

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild Shangzhe Wu Christian Rupprecht Andrea Vedaldi V ISUAL G EOMETRY G ROUP , U NIVERSITY OF O XFORD Agenda v Problem Introduction v Method Overview v Results

3.49k views • 59 slides

Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site

CS11-747 Neural Networks for NLP Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site https://phontron.com/class/nn4nlp2018/ Supervised, Unsupervised, Semi-supervised Most models handled here are supervised learning

670 views • 32 slides

Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site

CS11-747 Neural Networks for NLP Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site https://phontron.com/class/nn4nlp2020/ Supervised, Unsupervised, Semi-supervised Most models handled here are supervised learning

580 views • 39 slides

Unsupervised Learning Principal Component Analysis CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu

Unsupervised Learning Principal Component Analysis CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Slides credit: Maria-Florina Balcan Unsupervised Learning Discovering hidden structure in data Last time: K-Means Clustering What is the

502 views • 15 slides

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no output classifications (labels) l Clustering is an important type of unsupervised learning PCA was another type of unsupervised learning l The

858 views • 38 slides

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on supervised learning methods such as regression and classification. In that setting we observe both a set of features X 1 , X 2 , . . . , X p for each

934 views • 50 slides

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

721 views • 60 slides

Unsupervised Learning Andrea Passerini passerini@disi.unitn.it Machine Learning Unsupervised

Unsupervised Learning Andrea Passerini passerini@disi.unitn.it Machine Learning Unsupervised Learning Unsupervised Learning Setting Supervised learning requires the availability of labelled examples Labelling examples can be an extremely

752 views • 27 slides

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Wanli Ouyang, Zsolt Kira, Lawrence Neal, Raymond Yeh, Junting Lou and Teck-Yian Lim Unsupervised Learning in General Unsupervised learning is

1.48k views • 115 slides

Unsupervised Le Learning of Video Representations using LS LSTMs Srivastava et al. University

Unsupervised Le Learning of Video Representations using LS LSTMs Srivastava et al. University of Toronto Presented by Shyam Tailor The Overall Idea Take a sequence of images and encode into a fixed size latent representation Decode

210 views • 20 slides

Unsupervised Learning of Visual Structure Using Predictive Generative Networks William Lotter,

Unsupervised Learning of Visual Structure Using Predictive Generative Networks William Lotter, Gabriel Kreiman & David Cox Harvard University, Cambridge, USA Article overview by Ilya Kuzovkin Computational Neuroscience Seminar University

964 views • 45 slides

Machine Learning for NLP Unsupervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Unsupervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain Sciences University of Trento 1 Unsupervised learning In unsupervised learning, we learn without training data. The idea is to find a

998 views • 67 slides

Unsupervised, Fast and Precise Recognition of Digital Arcs in Noisy Images T. P . NGUYEN, B.

Introduction Arc segmentation Unsupervised Noise Detection A framework for arc recognition along noisy curves Experimentations Conclusions and futur work Unsupervised, Fast and Precise Recognition of Digital Arcs in Noisy Images T. P .

550 views • 51 slides

Unsupervised Learning Introduction Nakul Verma Unsupervised Learning What can we learn from

Unsupervised Learning Introduction Nakul Verma Unsupervised Learning What can we learn from data when label information is not available? Supervised learning framework Supervised learning Data: Assumption: there is a (relatively simple)

442 views • 16 slides

Practical Unsupervised Learning INFO/CS 4300, Spring 2016 Jack Hessel Unsupervised Learning is

Practical Unsupervised Learning INFO/CS 4300, Spring 2016 Jack Hessel Unsupervised Learning is Cool! But how can we use this in our projects? 1. 2. But first, lets look at our dataset... Data Dimensionality! Probability of Lung Cancer

999 views • 55 slides

Automatic Learning of a Morphological Model Theory and Unsupervised Approaches Unsupervised

Automatic Learning of a Morphological Model Theory and Unsupervised Approaches Unsupervised Learning - ToC Foundations Problem description General architecture Papers: Goldsmith 2001 and 2006 Goldsmith and Hu 2004

1.23k views • 43 slides

Mixture Models David M. Blei COS424 Princeton University March 2, 2012 Unsupervised learning

Mixture Models David M. Blei COS424 Princeton University March 2, 2012 Unsupervised learning Unsupervised learning is about taking data and finding structure in it. It is about finding patterns without knowing what we are looking for.

416 views • 28 slides

Unsupervised learning introduction October 7, 2019 Unsupervised learning introduction

Unsupervised learning introduction October 7, 2019 Unsupervised learning introduction October 7, 2019 1 / 39 Intro General statement of the problem One has a set of N observations ( x 1 , x 2 , ..., x N ) of a random p -vector X having

495 views • 32 slides

Unsupervised learning General introduction to unsupervised learning PCA Special directions

Unsupervised learning General introduction to unsupervised learning PCA Special directions These are special directions we will try to find. Best direction u : |u| 2 = 1 2 1. Minimize : d i X i T u is the projection length x i u d i T

1.01k views • 29 slides

THE MAGIC OF UNSUPERVISED LEARNING Agustinus Nalwan Head of AI Carsales.com.au A LITTLE BIT

THE MAGIC OF UNSUPERVISED LEARNING Agustinus Nalwan Head of AI Carsales.com.au A LITTLE BIT ABOUT MYSELF AI 4 Years SAVING OUR FOREST BUSHFIRE DETECTION BACK TO THE MAIN TOPIC THE MAGIC OF UNSUPERVISED LEARNING SUPERVISED LEARNING

1.11k views • 82 slides

Lecture 13: From Unsupervised to Reinforcement Learning (Chapters 8-10) R. Rao, 528: Lecture 13

CSE/NB 528 Lecture 13: From Unsupervised to Reinforcement Learning (Chapters 8-10) R. Rao, 528: Lecture 13 1 Todays Agenda: All about Learning F Unsupervised Learning Sparse Coding Predictive Coding F Supervised learning Perceptrons and

479 views • 24 slides

Unsupervised Deep Learning Tutorial - Part 2 Alex Graves MarcAurelio Ranzato

Unsupervised Deep Learning Tutorial - Part 2 Alex Graves MarcAurelio Ranzato gravesa@google.com ranzato@fb.com NeurIPS, 3 December 2018 Overview Practical Recipes of Unsupervised Learning Learning representations Learning to

1.16k views • 103 slides