Model-based Machine Learning Chris Bishop Microsoft Research - PowerPoint PPT Presentation

Jun 14, 2023 •106 likes •374 views

Model-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012 Traditional machine learning Logistic regression Neural networks K-means, mixture of Gaussians PCA, kernel PCA, ICA, FA Support vector machines

Model-based Machine Learning Chris Bishop Microsoft Research Cambridge Royal Society, March 2012
Traditional machine learning Logistic regression Neural networks K-means, mixture of Gaussians PCA, kernel PCA, ICA, FA Support vector machines Deep belief networks Decision trees and random forests … many others …
Model-based machine learning Goal : a single modelling framework which supports a wide range of models Traditional: “how do I map my problem onto a standard algorithm”? Model-based: “what is the model that represents my problem”?
Realisation of model-based ML Bayesian framework Probabilistic graphical models Efficient deterministic inference
Movie recommender demo
Probabilistic graphical models Maths (M) Geometry (G) Algebra (A) P (M, G, A) = P (M) P (G|M) P (A|M) Graph structure captures domain knowledge
Efficient inference
Local message-passing Maths (M) ? Geometry (G) Algebra (A)
What if distributions are intractable? True distribution Monte Carlo Variational Message Passing Loopy belief propagation Expectation propagation ⁞
Algorithms  Models M. E. Tipping and C. M. Bishop (1997) C. M. Bishop (1999)
Childhood Asthma
Allergic Sensitisation Model
Comparison with traditional ML Separation of model and training algorithm Auto-generated inference algorithm Easy extension to more complex situations Modify model, use the same inference algorithms Flexible as requirements change Compact code Easy to write and maintain Transparent functionality Many traditional methods are special cases One simple framework for newcomers to the field
“Big data” Computational size vs. statistical size ? length temperature
Noisy ranking Conventional approach to ranking: “ Elo ” single strength value for each player cannot handle teams, or more than 2 players
Bayesian Ranking: TrueSkill TM s 1 s 2  1  2 y 12 R. Herbrich, T. Minka, and T. Graepel; NIPS (2006)
Multi-player multi-team model s 1 s 2 s 3 s 4 t 1 t 2 t 3 y 12 y 23
^ ^ s 1 s 2 s 1 s 2  1  2  1  2 ^ ^ ^ y 12 y 12
TrueSkill TM Sept. 2005; 10s of millions of users; millions of matches per day
Convergence 40 35 30 25 Level 20 15 char (TrueSkill ™ ) 10 SQLWildman (TrueSkill ™ ) char (Elo) 5 SQLWildman (Elo) 0 0 100 200 300 400 Number of Games
Infer.NET 1. Specify your machine learning problem as a probabilistic model in a .NET program (typically 10-20 lines of code). 2. Use Infer.NET to compile the model into optimized runtime code. 3. Run the code to make inferences on your data automatically. research.microsoft.com/infernet
research.microsoft.com/~ cmbishop

Recommend

1 Model-Based Classification Model-Based Classification Model-based approach Build a

Machine Learning CSE 473: Artificial Intelligence Nave Bayes Up until now: how use a model to make optimal decisions Machine learning: how to acquire a model from data / experience Learning parameters (e.g. probabilities)

628 views • 8 slides

Machine Learning CS 188: Artificial Intelligence Up until now: how to reason in a model and

Machine Learning CS 188: Artificial Intelligence Up until now: how to reason in a model and how to make optimal decisions Review of Machine Learning (ML) Machine learning: how to acquire a model DISCLAIMER: It is insufficient to

258 views • 7 slides

Outline Model Based Metaheuristics DM812 METAHEURISTICS Lecture 10 1. Machine Learning

Machine Learning Outline Model Based Metaheuristics DM812 METAHEURISTICS Lecture 10 1. Machine Learning Machine Learning and Estimation of Distribution Algorithms 2. Model Based Metaheuristics Estimation of Distribution Algorithms Marco

443 views • 6 slides

Machine Learning CS 188: Artificial Intelligence Nave Bayes Up until now: how use a model

Machine Learning CS 188: Artificial Intelligence Nave Bayes Up until now: how use a model to make optimal decisions Machine learning: how to acquire a model from data / experience Learning parameters (e.g. probabilities) Learning

405 views • 11 slides

MCMC based machine learning a . (Bayesian Model Averaging) Nicos Angelopoulos

MCMC based machine learning a . (Bayesian Model Averaging) Nicos Angelopoulos n.angelopoulos@ed.ac.uk School of Biological Sciences Biochemistry Group University of Edinburgh, Scotland, UK. a Collaborative work with James Cussens, York

393 views • 21 slides

An Enhanced Ride Sharing Model Based on Human Characteristics and Machine Learning Recommender

An Enhanced Ride Sharing Model Based on Human Characteristics and Machine Learning Recommender System Govind Yatnalkar, Husnu S. Narman, Haroon Malik The 3rd International Conference on Emerging Data and Industry 4.0 (EDI40) April 6 - 9, 2020,

622 views • 29 slides

Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh | http://reza-zadeh.com Machine Learning

Scaled Machine Learning at Matroid Reza Zadeh @Reza_Zadeh | http://reza-zadeh.com Machine Learning Pipeline Learning Replicate Algorithm model Data Trained Serve Model Model Repeat entire pipeline Scaling Machine Learning Datasets and

890 views • 21 slides

OUTLINE THEORETICAL IMPEDIMENTS TO MACHINE LEARNING Model-blind machine learning is a

NIPS-2017 OUTLINE THEORETICAL IMPEDIMENTS TO MACHINE LEARNING Model-blind machine learning is a curve-fitting exercise slow and dumb WITH SEVEN SPARKS FROM The Causal hierarchy THE CAUSAL REVOLUTION What we miss by

143 views • 3 slides

CWoLa Hunting: Extending the Bump Hunt with Machine Learning Based on: Phys. Rev. Lett. 121,

CWoLa Hunting: Extending the Bump Hunt with Machine Learning Based on: Phys. Rev. Lett. 121, 241803 (2018) [1805.02664] Jack Collins, Kiel Howe, Ben Nachman 1 CWoLa Hunting Outline 1) Machine Learning 2) Model Unspecific Searches 3) CWoLa

668 views • 65 slides

Introduction to Machine Learning 3. Instance Based Learning Alex Smola Carnegie Mellon

Introduction to Machine Learning 3. Instance Based Learning Alex Smola Carnegie Mellon University http://alex.smola.org/teaching/cmu2013-10-701 10-701 Outline Parzen Windows Kernels, algorithm Model selection Crossvalidation, leave

1.1k views • 68 slides

Measuring model performance or error Introduction to Machine Learning Is our model any good?

INTRODUCTION TO MACHINE LEARNING Measuring model performance or error Introduction to Machine Learning Is our model any good? Context of task Accuracy Computation time Interpretability 3 types of tasks

1.31k views • 77 slides

Edge-based Discovery of Training Data for Machine Learning CMU authors Deep Learning Recipe

CSci 8980 Edge-based Discovery of Training Data for Machine Learning CMU authors Deep Learning Recipe Collect a large amount of data and label it Select a model and train a DNN Deploy the DNN for inference Labelled Data Some

409 views • 14 slides

10a Machine Learning: Symbol-based 10.0 Introduction 10.5 Knowledge and Learning 10.1 A

10a Machine Learning: Symbol-based 10.0 Introduction 10.5 Knowledge and Learning 10.1 A Framework for 10.6 Unsupervised Learning Symbol-based Learning 10.7 Reinforcement Learning 10.2 Version Space Search 10.8 Epilogue and 10.3 The

935 views • 54 slides

10d Machine Learning: Symbol-based 10.0 Introduction 10.5 Knowledge and Learning 10.1 A

10d Machine Learning: Symbol-based 10.0 Introduction 10.5 Knowledge and Learning 10.1 A Framework for 10.6 Unsupervised Learning Symbol-based Learning 10.7 Reinforcement Learning 10.2 Version Space Search 10.8 Epilogue and 10.3 The

1.05k views • 29 slides

10c Machine Learning: Symbol-based 10.0 Introduction 10.5 Knowledge and Learning 10.1 A

10c Machine Learning: Symbol-based 10.0 Introduction 10.5 Knowledge and Learning 10.1 A Framework for 10.6 Unsupervised Learning Symbol-based Learning 10.7 Reinforcement Learning 10.2 Version Space Search 10.8 Epilogue and 10.3 The

605 views • 35 slides

10b Machine Learning: Symbol-based 10.0 Introduction 10.5 Knowledge and Learning 10.1 A

10b Machine Learning: Symbol-based 10.0 Introduction 10.5 Knowledge and Learning 10.1 A Framework for 10.6 Unsupervised Learning Symbol-based Learning 10.7 Reinforcement Learning 10.2 Version Space Search 10.8 Epilogue and 10.3 The

866 views • 50 slides

Genetics-based Machine Learning and Behaviour Based Robotics: A New Synthesis

Genetics-based Machine Learning and Behaviour Based Robotics: A New Synthesis Genetics-based Machine Learning and Behavior Based Robotics: A New Synthesis, Marco Dorigo, Uwe Schnepf, IEEE Transactions on System, MAn, and Cybernetics,

348 views • 34 slides

Cross-Validation Machine Learning 1 Model selection Very broadly: Choosing the best model using

Cross-Validation Machine Learning 1 Model selection Very broadly: Choosing the best model using given data What makes a model Features Hyper-parameters that control the hypothesis space Example: depth of a decision tree, neural

328 views • 12 slides

Learning to Inflate Tom Rudelius IAS Based on 1810.05159/hep-th Outline Machine Learning

Learning to Inflate Tom Rudelius IAS Based on 1810.05159/hep-th Outline Machine Learning and hep-th Introduction to Inflation Learning to Inflate Algorithm Results Possible Next Steps Machine Learning and hep-th Machine

709 views • 34 slides

Thinking about Genomics through a Machine Learning Lens: Basics of how several DNA- and RNA-based

Thinking about Genomics through a Machine Learning Lens: Basics of how several DNA- and RNA-based problems translate into commonly-studied areas of machine learning Faculty Lead Discussion (short version) 26 June 2018 GAUSSI Summer Retreat

323 views • 21 slides

Overview Learning phrases from alignments A phrase-based model 6.864 (Fall 2007)

Overview Learning phrases from alignments A phrase-based model 6.864 (Fall 2007) Decoding in phrase-based models Machine Translation Part III (Thanks to Philipp Koehn for giving me the slides from his EACL 2006 tutorial) 1 3

394 views • 7 slides

INTRODUCTION TO MACHINE LEARNING Joseph C. Osborn CS 51A Spring 2020 Machine Learning is

INTRODUCTION TO MACHINE LEARNING Joseph C. Osborn CS 51A Spring 2020 Machine Learning is Machine learning is about predicting the future based on the past. -- Hal Daume III Machine Learning is Machine learning is about predicting

917 views • 59 slides

Part III: Machine Learning CS 188: Artificial Intelligence Up until now: how to reason in a

Part III: Machine Learning CS 188: Artificial Intelligence Up until now: how to reason in a model and how to make optimal decisions Machine learning: how to acquire a model Lecture 20: Dynamic Bayes Nets, Nave Bayes on the basis

821 views • 7 slides

Lipschitz Continuity in Model-based Reinforcement Learning Kavosh Asadi, Dipendra Misra,

Lipschitz Continuity in Model-based Reinforcement Learning Kavosh Asadi*, Dipendra Misra*, Michael L. Littman * denotes equal contribution 1 Model-based RL value/policy planning acting model experience model learning model learning: T

592 views • 13 slides