introduction to gaussian processes
play

Introduction to Gaussian Processes Neil D. Lawrence GPMC 6th - PowerPoint PPT Presentation

Apr 19, 2023 •125 likes •1.95k views

Introduction to Gaussian Processes Neil D. Lawrence GPMC 6th February 2017 Book Rasmussen and Williams (2006) Outline The Gaussian Density Covariance from Basis Functions Outline The Gaussian Density Covariance from Basis Functions The

  1. Stages to Derivation of the Posterior ◮ Multiply likelihood by prior ◮ they are “exponentiated quadratics”, the answer is always also an exponentiated quadratic because exp( a 2 ) exp( b 2 ) = exp( a 2 + b 2 ). ◮ Complete the square to get the resulting density in the form of a Gaussian. ◮ Recognise the mean and (co)variance of the Gaussian. This is the estimate of the posterior.

  2. Multivariate Regression Likelihood ◮ Noise corrupted data point y i = w ⊤ x i , : + ǫ i

  3. Multivariate Regression Likelihood ◮ Noise corrupted data point y i = w ⊤ x i , : + ǫ i ◮ Multivariate regression likelihood:  n  1  − 1 � 2 �  �  y i − w ⊤ x i , : p ( y | X , w ) = (2 πσ 2 ) n / 2 exp       2 σ 2    i = 1

  4. Multivariate Regression Likelihood ◮ Noise corrupted data point y i = w ⊤ x i , : + ǫ i ◮ Multivariate regression likelihood:  n  1  − 1 � 2 �  �  y i − w ⊤ x i , : p ( y | X , w ) = (2 πσ 2 ) n / 2 exp       2 σ 2    i = 1 ◮ Now use a multivariate Gaussian prior: 1 − 1 � � 2 α w ⊤ w p ( w ) = exp p (2 πα ) 2

  5. Two Dimensional Gaussian ◮ Consider height, h / m and weight, w / kg . ◮ Could sample height from a distribution: p ( h ) ∼ N (1 . 7 , 0 . 0225) ◮ And similarly weight: p ( w ) ∼ N (75 , 36)

  6. Height and Weight Models p ( w ) p ( h ) h / m w / kg Gaussian distributions for height and weight.

  7. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  8. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  9. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  10. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  11. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  12. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  13. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  14. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  15. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  16. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  17. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  18. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  19. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  20. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  21. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  22. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  23. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  24. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  25. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  26. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  27. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  28. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  29. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  30. Independence Assumption ◮ This assumes height and weight are independent. p ( h , w ) = p ( h ) p ( w ) ◮ In reality they are dependent (body mass index) = w h 2 .

  31. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  32. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  33. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  34. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  35. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  36. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  37. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  38. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  39. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  40. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  41. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  42. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  43. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  44. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  45. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  46. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  47. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  48. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  49. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  50. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  51. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

Recommend

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian Processes Fancier Gaussian Processes The current DFF. ( de facto fanciness) Uses for: Regression Classification

754 views • 50 slides
Another introduction to Gaussian Processes Richard Wilkinson School of Maths and Statistics

Another introduction to Gaussian Processes Richard Wilkinson School of Maths and Statistics

Another introduction to Gaussian Processes Richard Wilkinson School of Maths and Statistics University of Sheffield GP summer school September 2017 Why use Gaussian processes? Why use Gaussian processes? A stochastic process is a collection

1.69k views • 152 slides
Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian

Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian

Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian posteriors Approximate methods Laplace approximation Variational bayes Expectation propagation Comparisons GP regression - recap so far Model the

1.41k views • 113 slides
Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes

Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes

Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes Pierre-Franois Lget, Stanford University - KIPAC Gaussian process (GP) interpolation: Gaussian processes are a non-parametric way to interpolate data

281 views • 7 slides
Introduction to Gaussian Processes Stephen Keeley and Jonathan Pillow Princeton Neuroscience

Introduction to Gaussian Processes Stephen Keeley and Jonathan Pillow Princeton Neuroscience

Introduction to Gaussian Processes Stephen Keeley and Jonathan Pillow Princeton Neuroscience Institute Princeton University skeeley@princeton.edu March 28, 2018 Gaussian Processes (GPs) are a flexible and general way to parameterize functions

174 views • 5 slides
State Space Gaussian Processes with Non-Gaussian Likelihoods Hannes Nickisch 1 Arno Solin 2

State Space Gaussian Processes with Non-Gaussian Likelihoods Hannes Nickisch 1 Arno Solin 2

State Space Gaussian Processes with Non-Gaussian Likelihoods Hannes Nickisch 1 Arno Solin 2 Alexander Grigorievskiy 2 , 3 1 Philips Research, 2 Aalto University, 3 Silo.AI ICML2018 July 13, 2018 Outline Gaussian Processes Temporal GPs as

285 views • 17 slides
My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an

My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an

My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an PhD student. Teacher Assistant. Universidad Aut onoma de Madrid 1/13 Index Introduction: Bayesian Optimization and Gaussian Processes. 2/13

923 views • 70 slides
Gaussian Processes Dan Cervone NYU CDS November 10, 2015 Dan Cervone (NYU CDS) Gaussian

Gaussian Processes Dan Cervone NYU CDS November 10, 2015 Dan Cervone (NYU CDS) Gaussian

Gaussian Processes Dan Cervone NYU CDS November 10, 2015 Dan Cervone (NYU CDS) Gaussian Processes November 10, 2015 1 / 22 What are Gaussian processes? GPs let us do Bayesian inference on functions . Using GPs we can: Interpolate spatial

728 views • 54 slides
Introduction to Gaussian Processes Iain Murray murray@cs.toronto.edu CSC2515, Introduction to

Introduction to Gaussian Processes Iain Murray murray@cs.toronto.edu CSC2515, Introduction to

Introduction to Gaussian Processes Iain Murray murray@cs.toronto.edu CSC2515, Introduction to Machine Learning, Fall 2008 Dept. Computer Science, University of Toronto The problem Learn scalar function of vector values f ( x ) 1 f(x) 0.5 y i

837 views • 39 slides
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian Processes Outline Parametric

572 views • 40 slides
Advanced Introduction to Machine Learning CMU-10715 Gaussian Processes Barnabs Pczos

Advanced Introduction to Machine Learning CMU-10715 Gaussian Processes Barnabs Pczos

Advanced Introduction to Machine Learning CMU-10715 Gaussian Processes Barnabs Pczos Introduction http://www.gaussianprocess.org/ Some of these slides in the intro are taken from D. Lizotte, R. Parr, C. Guesterin 3 Contents

873 views • 72 slides
Gaussian Processes Seung-Hoon Na Chonbuk National University Gaussian Process Regression

Gaussian Processes Seung-Hoon Na Chonbuk National University Gaussian Process Regression

Gaussian Processes Seung-Hoon Na Chonbuk National University Gaussian Process Regression Predictions using noisy observations The case of a single test input: Gaussian Process Regression Computational and numerical issues It is

444 views • 29 slides
DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

Introduction Recap Kernels Gaussian Processes References DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal Institute of Technology November 5th, 2015 Ek KTH DD2434 - Advanced Machine Learning

1.91k views • 170 slides
Introduction to Gaussian Processes Iain Murray School of Informatics, University of Edinburgh

Introduction to Gaussian Processes Iain Murray School of Informatics, University of Edinburgh

Introduction to Gaussian Processes Iain Murray School of Informatics, University of Edinburgh The problem Learn scalar function of vector values f ( x ) 1 f(x) 0.5 y i 5 0 0 f 0.5 5 0 1 1 0.5 1.5 0.5 0 0.2 0.4 0.6

767 views • 42 slides
Gaussian Processes for Big Data James Hensman joint work with Nicol o Fusi, Neil D. Lawrence

Gaussian Processes for Big Data James Hensman joint work with Nicol o Fusi, Neil D. Lawrence

Gaussian Processes for Big Data James Hensman joint work with Nicol o Fusi, Neil D. Lawrence Overview Motivation Sparse Gaussian Processes Stochastic Variational Inference Examples Overview Motivation Sparse Gaussian Processes

533 views • 36 slides
Scalable Gaussian processes with a twist of Probabilistic Numerics Kurt Cutajar EURECOM, Sophia

Scalable Gaussian processes with a twist of Probabilistic Numerics Kurt Cutajar EURECOM, Sophia

Scalable Gaussian processes with a twist of Probabilistic Numerics Kurt Cutajar EURECOM, Sophia Antipolis, France Data Science Meetup - October 30 th 2017 Agenda Kernel Methods Scalable Gaussian Processes (using Preconditioning)

481 views • 46 slides
Practical Challenges of Gaussian Processes Marc Deisenroth Statistical Machine Learning Group

Practical Challenges of Gaussian Processes Marc Deisenroth Statistical Machine Learning Group

Practical Challenges of Gaussian Processes Marc Deisenroth Statistical Machine Learning Group Department of Computing Imperial College London New Directions for Learning with Kernels and Gaussian Processes, Dagstuhl November 29, 2016 Data

264 views • 5 slides
Gaussian Processes Covariance Functions and Classification Carl Edward Rasmussen Max Planck

Gaussian Processes Covariance Functions and Classification Carl Edward Rasmussen Max Planck

Gaussian Processes Covariance Functions and Classification Carl Edward Rasmussen Max Planck Institute for Biological Cybernetics T ubingen, Germany Gaussian Processes in Practice, Bletchley Park, July 12th, 2006 Carl Edward Rasmussen

696 views • 30 slides
CSC 411 Lecture 20: Gaussian Processes Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 20: Gaussian Processes Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 20: Gaussian Processes Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 20-Gaussian Processes 1 / 24 Overview Last lecture: Bayesian linear regression, a parametric model This

835 views • 24 slides
Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu L Murray, Iain

Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu L Murray, Iain

Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu L Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo for expensive Bayesian

357 views • 14 slides
Probabilistic Graphical Models Lecture 21: Advanced Gaussian Processes Andrew Gordon Wilson

Probabilistic Graphical Models Lecture 21: Advanced Gaussian Processes Andrew Gordon Wilson

Probabilistic Graphical Models Lecture 21: Advanced Gaussian Processes Andrew Gordon Wilson www.cs.cmu.edu/~andrewgw Carnegie Mellon University April 1, 2015 1 / 100 Gaussian process review Definition A Gaussian process (GP) is a collection

1.36k views • 100 slides
Gaussian Random Variables and Processes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department

Gaussian Random Variables and Processes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department

Gaussian Random Variables and Processes Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 1, 2012 1 / 33 Gaussian Random Variables Gaussian Random Variable

753 views • 33 slides
Outline 1. Motivation 2. Gaussian process introduction 3. Change surface model 4. Analysis of

Outline 1. Motivation 2. Gaussian process introduction 3. Change surface model 4. Analysis of

5/22/2016 Scalable Gaussian Processes for Characterizing Multidimensional Change Surfaces April 18, 2016 William Herlands Committee: Daniel Neill, Alex Smola, Wilbert Van Panhuis Chair: Dave Choi Outline 1. Motivation 2. Gaussian process

1k views • 21 slides
Persistence of Gaussian stationary processes Naomi D. Feldheim Joint work with Ohad N. Feldheim

Persistence of Gaussian stationary processes Naomi D. Feldheim Joint work with Ohad N. Feldheim

Introduction Persistence Probability Ideas from the Proofs Persistence of Gaussian stationary processes Naomi D. Feldheim Joint work with Ohad N. Feldheim Department of Mathematics Tel-Aviv University Darmstadt July, 2014 Introduction

1.03k views • 69 slides

More recommend