A Gentle Introduction to Machine Learning First Lecture Olov - PDF document

10/7/2014 A Gentle Introduction to Machine Learning First Lecture Olov Andersson, AIICS Linköpings Universitet What is Machine Learning about? • To imbue the capacity to learn into machines • Our only frame of reference for learning is from biology  …but brains are hideously complex, the result of ages of evolution • Like much of AI, Machine Learning mainly takes an engineering approach 1 • Humanity didn’t first master flight by just imitating birds! 1. Although there is some occasional biological inspiration 2014-10-07 2 1

10/7/2014 Why Machine Learning • It may be impossible to manually program for every situation in advance • The world may change, if the agent cannot adapt it will fail • Many argue that learning is required for AI to scale up • We are still far from a general learning agent!  but the algorithms we have so far have shown themselves to be useful in a wide range of applications! 2014-10-07 3 Some Application Aspects • May not be as versatile as human learning, but domain specific problems can often be processed much faster than by a human • Computers work 24/7 and you can often scale performance by piling on more of them Data Mining  Companies can collect ever more data and processing power is cheap  Put it to use automatically analyzing the performance of products!  Machine Learning is almost ubiquitous on the web: Mail filters, search engines, product recommendations , customized content, ad serving…  “Big Data” – much hyped technology trend. Robotics, Computer Vision  Many capabilities that humans take for granted like locomotion, grasping and recognizing objects have turned out to be ridiculously difficult to manually construct rules for. 2014-10-07 4 2

10/7/2014 Demo – Stanford Helicopter Acrobatics …in narrow applications machine learning can even rival human performance 2014-10-07 5 To Define Machine Learning A machine learns with respect to a particular task T, performance metric P, and type of experience E, if the system reliably improves its performance P at task T, following experience E -Tom Mitchell From the agent perspective: Performance Metric Input (Sensors) (Environment) Task Agent Output (Actuators) 2014-10-07 6 3

10/7/2014 The Three Main Types of Machine Learning Machine learning is a young science that is still changing, but traditionally algorithms are usually divided into three types depending on their purpose. • Supervised Learning • Reinforcement Learning • Unsupervised Learning 2014-10-07 7 Supervised Learning at a glance A machine learns with respect to a particular task T, performance metric P, and type of experience E, if the system reliably improves its performance P at task T, following experience E -Tom Mitchell In Supervised Learning: • The correct output is given to the algorithm during a training phase • Experience E is thus tuples of training data consisting of (Input,Output) pairs • Performance metric P is some function of how well the predicted output matches the given correct output Mathematically, can be seen as trying to approximate an unknown function f (x) = y given examples of (x, y) 2014-10-07 8 4

10/7/2014 Supervised Learning at a glance II Representation from agent perspective: Performance Metric Input (Sensors) state Reactive (Environment) Agent Task f(input)=output or f(state) = action Output (Actuators) action …but it can also be used as a component in other architectures Supervised Learning is surprisingly powerful and ubiquitous Some real world examples • Spam Filter : f(mail) = spam? • Microsoft Kinect : f(pixels, distance) = body part 2014-10-07 9 Body Part Classification on the Microsoft Kinect left right hand neck shoulder right elbow Shotton et al @ MSR, CVPR 2011 2014-10-07 10 5

10/7/2014 Reinforcement Learning at a glance A machine learns with respect to a particular task T, performance metric P, and type of experience E, if the system reliably improves its performance P at task T, following experience E -Tom Mitchell In Reinforcement Learning: • A reward is given at each step instead of the correct input/output • Experience E consists of the history of inputs, the chosen outputs and the rewards • Performance metric P is some sum of how much reward the agent can accumulate Inspired by early work in psychology and how pets are trained The agent can learn on its own as long as the reward signal can be concisely defined. 2014-10-07 11 Reinforcement Learning at a glance II RL fits neatly into a utility (reward) maximizing agent framework • Rewards of actions in different states are learned • Agent plans ahead to maximize reward over time Performance Metric (reward) Input (Sensors) state (Environment) RL Agent f(state, action) = reward Task Maximize future reward Output (Actuators) action Real world examples – Robot Control, Game Playing (Checkers…) 2014-10-07 12 6

10/7/2014 Unsupervised Learning at a glance A machine learns with respect to a particular task T, performance metric P, and type of experience E, if the system reliably improves its performance P at task T, following experience E -Tom Mitchell In Unsupervised Learning: • The Task is to find a more concise representation of the data • Neither the correct answer, nor a reward is given • Experience E is just the given data • P depends on the task Examples: Clustering – When the data distribution is confined to lie in a small number of “clusters “ we can find these and use them instead of the original representation Dimensionality Reduction – Finding a suitable lower dimensional representation while preserving as much information as possible 2014-10-07 13 Clustering – Continuous Data (Bishop, 2006) Two-dimensional continuous input 2014-10-07 15 7

10/7/2014 Outline of Machine Learning Lectures First we will talk about Supervised Learning • Definition • Main Concepts • Some Approaches & Applications • Pitfalls & Limitations • In-depth: Decision Trees (a supervised learning approach) Then finish with a short introduction to Reinforcement Learning The idea is that you will be informed enough to find and try a learning algorithm if the need arises. 2014-10-07 17 Supervised Learning in more detail… Remember, in Supervised Learning: • Tuples of training data consisting of ( x ,y) pairs are given to the algorithm • The objective is to learn to predict the output y i for an input x i Can be seen as searching for an approximation to the unknown function y = f(x) given N examples of x and y: ( x 1 ,y 1 ), … ,( x n ,y n ) • A candidate approximation is sometimes called a hypothesis Two major classes of supervised learning • Classification – Output is a discrete category label  Example: Detecting cancer, y = “healthy” or “ill” • Regression – Output is a numeric value  Example: Predicting temperature, y = 15.3 degrees In either case input data x i can be vector valued and discrete , continuous or mixed . Example: x 1 = (12.5, “cloud free”, 1.35). 2014-10-07 18 8

10/7/2014 Supervised Learning in Practice Can be seen as searching for an approximation to the unknown function y = f(x) given N examples of x and y: ( x 1 ,y 1 ), … ,( x n ,y n ) The goal is to have the algorithm learn from training examples to successfully generalize to new examples • First construct an input vector x i of examples by encoding relevant problem data. This is often called the feature vector . • Examples of such x i , y i is the training set. • A model is selected and trained on the examples by searching for parameters (the hypothesis space) that yield a good , approximation to the unknown true function. , , • Evaluate performance, (carefully) tweak algorithm or features. 2014-10-07 19 Feature Vector Construction Want to learn y = f(x) given N examples of x and y: ( x 1 ,y 1 ), … ,( x n ,y n ) • Standard algorithms tend to work on variables defined as numbers • If the inputs x or outputs y contain categorical values like “book” or “car” we need to encode them, typically as integers.  With only two classes we get y in {0,1}, called binary classification  Classification into multiple classes can be reduced to a sequence of binary one-vs-all classifiers • The variables may also be structured like in text, graphs, audio, image or video data • Finding a suitable feature representation can be non-trivial, but there are standard approaches for the common domains 2014-10-07 20 9