Practical Challenges of Gaussian Processes Marc Deisenroth - PowerPoint PPT Presentation

Apr 08, 2024 •197 likes •264 views

Practical Challenges of Gaussian Processes Marc Deisenroth Statistical Machine Learning Group Department of Computing Imperial College London New Directions for Learning with Kernels and Gaussian Processes, Dagstuhl November 29, 2016 Data

Practical Challenges of Gaussian Processes Marc Deisenroth Statistical Machine Learning Group Department of Computing Imperial College London New Directions for Learning with Kernels and Gaussian Processes, Dagstuhl November 29, 2016
Data Efficiency in Decision-Making Systems § Trial-and-error learning from a small number of samples § Careful treatment of uncertainty (robust decision making and targeted exploration) § Incorporation of useful priors § Transfer learning § Bayesian experimental design / Bayesian optimization 2 GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016
Fast Approximate Inference in RL/Robotics 3 t=T 2 x t+1 x t+1 x (2) 1 t=5 t=0 t=1 − 1 − 0.5 0 0.5 1 0 0.5 1 1.5 0 p(x t+1 ) t=2 p(x t , u t ) 1 − 1 − 3 − 2 − 1 0 1 2 3 0 x (1) − 1 − 0.5 0 0.5 1 (x t , u t ) § In model-based RL, we need to perform approximate inference (e.g., moment matching) efficiently (see also EP for Deep GPs) § Training is cheap compared to (repeated) inference: Moment matching scales in O p N 2 D 3 q Even with sparse approximations, we are limited to « 3, 000 data points More scalable models and inference 3 GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016
Learning Simulators § Learn parameters of a simulator of a very expensive experiment § Few thousand outcomes of real experiments § Learn (parameters of) a simulator for these experiments Bayesian optimization or GP regression § Medium/large-scale GP models: § Very fast to evaluate § Scalable to reasonably high dimensions 4 GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016
Infrastructure § Scale-free and distributed model architectures § Scalable probabilistic models and inference GPFlow is a good start 5 GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016

Recommend

Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian

Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian posteriors Approximate methods Laplace approximation Variational bayes Expectation propagation Comparisons GP regression - recap so far Model the

1.41k views • 113 slides

Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes

Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes Pierre-Franois Lget, Stanford University - KIPAC Gaussian process (GP) interpolation: Gaussian processes are a non-parametric way to interpolate data

281 views • 7 slides

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian Processes Fancier Gaussian Processes The current DFF. ( de facto fanciness) Uses for: Regression Classification

754 views • 50 slides

State Space Gaussian Processes with Non-Gaussian Likelihoods Hannes Nickisch 1 Arno Solin 2

State Space Gaussian Processes with Non-Gaussian Likelihoods Hannes Nickisch 1 Arno Solin 2 Alexander Grigorievskiy 2 , 3 1 Philips Research, 2 Aalto University, 3 Silo.AI ICML2018 July 13, 2018 Outline Gaussian Processes Temporal GPs as

285 views • 17 slides

Another introduction to Gaussian Processes Richard Wilkinson School of Maths and Statistics

Another introduction to Gaussian Processes Richard Wilkinson School of Maths and Statistics University of Sheffield GP summer school September 2017 Why use Gaussian processes? Why use Gaussian processes? A stochastic process is a collection

1.69k views • 152 slides

Gaussian Processes Dan Cervone NYU CDS November 10, 2015 Dan Cervone (NYU CDS) Gaussian

Gaussian Processes Dan Cervone NYU CDS November 10, 2015 Dan Cervone (NYU CDS) Gaussian Processes November 10, 2015 1 / 22 What are Gaussian processes? GPs let us do Bayesian inference on functions . Using GPs we can: Interpolate spatial

728 views • 54 slides

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian Processes Outline Parametric

572 views • 40 slides

Modern Gaussian Processes: Scalable Inference and Novel Applications (Part III) Applications,

Modern Gaussian Processes: Scalable Inference and Novel Applications (Part III) Applications, Challenges & Opportunities Edwin V. Bonilla and Maurizio Filippone CSIROs Data61, Sydney, Australia and EURECOM, Sophia Antipolis, France July 14

551 views • 38 slides

Gaussian Processes Seung-Hoon Na Chonbuk National University Gaussian Process Regression

Gaussian Processes Seung-Hoon Na Chonbuk National University Gaussian Process Regression Predictions using noisy observations The case of a single test input: Gaussian Process Regression Computational and numerical issues It is

444 views • 29 slides

My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an

My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an PhD student. Teacher Assistant. Universidad Aut onoma de Madrid 1/13 Index Introduction: Bayesian Optimization and Gaussian Processes. 2/13

923 views • 70 slides

Gaussian Processes for Big Data James Hensman joint work with Nicol o Fusi, Neil D. Lawrence

Gaussian Processes for Big Data James Hensman joint work with Nicol o Fusi, Neil D. Lawrence Overview Motivation Sparse Gaussian Processes Stochastic Variational Inference Examples Overview Motivation Sparse Gaussian Processes

533 views • 36 slides

Scalable Gaussian processes with a twist of Probabilistic Numerics Kurt Cutajar EURECOM, Sophia

Scalable Gaussian processes with a twist of Probabilistic Numerics Kurt Cutajar EURECOM, Sophia Antipolis, France Data Science Meetup - October 30 th 2017 Agenda Kernel Methods Scalable Gaussian Processes (using Preconditioning)

481 views • 46 slides

Introduction to Gaussian Processes Stephen Keeley and Jonathan Pillow Princeton Neuroscience

Introduction to Gaussian Processes Stephen Keeley and Jonathan Pillow Princeton Neuroscience Institute Princeton University skeeley@princeton.edu March 28, 2018 Gaussian Processes (GPs) are a flexible and general way to parameterize functions

174 views • 5 slides

Gaussian Processes Covariance Functions and Classification Carl Edward Rasmussen Max Planck

Gaussian Processes Covariance Functions and Classification Carl Edward Rasmussen Max Planck Institute for Biological Cybernetics T ubingen, Germany Gaussian Processes in Practice, Bletchley Park, July 12th, 2006 Carl Edward Rasmussen

696 views • 30 slides

CSC 411 Lecture 20: Gaussian Processes Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 20: Gaussian Processes Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 20-Gaussian Processes 1 / 24 Overview Last lecture: Bayesian linear regression, a parametric model This

835 views • 24 slides

Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu L Murray, Iain

Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu L Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo for expensive Bayesian

357 views • 14 slides

Probabilistic Graphical Models Lecture 21: Advanced Gaussian Processes Andrew Gordon Wilson

Probabilistic Graphical Models Lecture 21: Advanced Gaussian Processes Andrew Gordon Wilson www.cs.cmu.edu/~andrewgw Carnegie Mellon University April 1, 2015 1 / 100 Gaussian process review Definition A Gaussian process (GP) is a collection

1.36k views • 100 slides

Gaussian Random Variables and Processes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department

Gaussian Random Variables and Processes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 1, 2012 1 / 33 Gaussian Random Variables Gaussian Random Variable

753 views • 33 slides

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes Mauricio A. Zhenwen Dai Alvarez Neil D. Lawrence Gaussian Process Approximation Workshop, 2017 Motivation I Machine learning has been very

685 views • 21 slides

Persistence of Gaussian Stationary Processes: a spectral perspective Naomi Feldheim (Stanford)

Persistence of Gaussian Stationary Processes: a spectral perspective Naomi Feldheim (Stanford) Joint work with Ohad Feldheim (Stanford) Shahaf Nitzan (GeorgiaTech) February, 2017 Gaussian stationary processes (GSP) For T { R , Z } , a

1.2k views • 94 slides

Persistence of Gaussian Stationary Processes Ohad Feldheim (Stanford) Joint work with Naomi

Persistence of Gaussian Stationary Processes Ohad Feldheim (Stanford) Joint work with Naomi Feldheim (Stanford) Shahaf Nitzan (GeorgiaTech) UBC, Vancouver January, 2017 Gaussian stationary processes (GSP) For T { R , Z } , a random

908 views • 80 slides

DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

Introduction Recap Kernels Gaussian Processes References DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal Institute of Technology November 5th, 2015 Ek KTH DD2434 - Advanced Machine Learning

1.91k views • 170 slides

Adversarially Robust Optimization with Gaussian Processes Ilija Bogunovic, Jonathan Scarlett,

Adversarially Robust Optimization with Gaussian Processes Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, Volkan Cevher Conference on Neural Information Processing Systems (Dec 2018) Gaussian Process Optimization Optimum Non-robust

393 views • 12 slides

Recurrent Gaussian processes Andreas Damianou with C esar Lincoln Mattos, Zhenwen Dai, Neil

Robust, Deep Recurrent Gaussian processes Andreas Damianou with C esar Lincoln Mattos, Zhenwen Dai, Neil Lawrence, Jeremy Forth, Guilherme Barreto Royal Society , 06 June 2016 Challenge: Learn patterns from sequences Recurrent Gaussian

622 views • 25 slides