A Desktop Can Machines Learn? Pascal Poupart Associate Professor - PDF document

A Desktop Can Machines Learn? Pascal Poupart Associate Professor David R. Cheriton School of Computer Science University of Waterloo 1 2 A Computer Program Machine Learning • Arthur Samuel (1959): Machine learning is the field of study that gives computers the ability to learn without being explicitly programmed. • Tom Mitchell (1998): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. 3 4 Three categories Supervised Learning • Example: digit recognition (postal code) Supervised learning • Simplest approach: Reinforcement learning memorization Unsupervised learning 5 6 1

Supervised Learning More Formally • Nearest neighbour: • Inductive learning: – Given a training set of examples of the form (x,f(x)) • x is the input, f(x) is the output – Return a function h that approximates f • h is called the hypothesis 7 8 Prediction Prediction • Find function h that fits f at instances x • Find function h that fits f at instances x 9 10 Prediction Prediction • Find function h that fits f at instances x • Find function h that fits f at instances x 11 12 2

Generalization Prediction • Key: a good hypothesis will generalize • Find function h that fits f at instances x well (i.e. predict unseen examples correctly) • Ockham’s razor: prefer the simplest hypothesis consistent with data 13 14 Reinforcement Learning Animal Psychology • Differs from supervised learning • Negative reinforcements: – Pain and hunger • Positive reinforcements: Reinforcement learning Supervised learning Don’t – Pleasure and food touch. You will get • Reinforcements used to train animals burnt Ouch! • Let’s do the same with computers! 15 16 Backgammon Helicopter Control • TD-Gammon: – Gerald Tesauro (1995) • Difficult to control: – Computer program – Highly unstable – Best backgammon player! • Play many games in simulation against itself • Andrew Ng (Stanford, 2006): – +1 for each win – Autonomous control by reinforcement learning – -1 for each loss – Step 1: learn neural net simulator based on flight data with human pilot • Optimization problem: find strategy that maximizes – Step 2: optimize controller based on reinforcements for cumulative score following a predefined trajectory 17 18 3

Applications of Machine Learning Vision • Speech recognition • Meta-programming: program computers to learn by themselves – dictation software • Natural Language Processing • Lifelong machine learning: machines that continuously learn – Text categorization – Information Retrieval • Transfer learning: machines that generalize their experience to new situations • Data Mining – Customer profiling • Robotic Control • Challenges: – Mobile robots – Computational complexity – Soccer playing robots – Sample complexity 19 20 Thank You Questions? 21 4