Machine Learning 2007: Slides 1 Instructor: Tim van Erven (Tim.van.Erven@cwi.nl) Website: www.cwi.nl/˜erven/teaching/0708/ml/ September 6, 2007 1 / 43
Overview Course Organisation Course Organisation ● Tentative Course Tentative Course Outline ● Outline What is Machine Learning? ● What is Machine Learning? This Lecture versus Mitchell ● This Lecture versus Supervised versus Unsupervised Learning ● Mitchell The Most Important Supervised Learning Problems ● Supervised versus Unsupervised Learning ✦ Prediction Prediction ✦ Regression Regression ✦ Classification Classification Hypotheses and Hypothesis Spaces ● Hypotheses and Hypothesis Spaces Least Squares ● Least Squares 2 / 43
People Instructor: Tim van Erven Course Organisation Tentative Course Outline E-mail: Tim.van.Erven@cwi.nl ● What is Machine Bio: ● Learning? This Lecture versus ✦ Studied AI at the University of Amsterdam Mitchell ✦ Currently a PhD student at the Centrum voor Wiskunde Supervised versus Unsupervised en Informatica (CWI) in Amsterdam Learning ✦ Research focuses on the Minimum Description Length Prediction (MDL) principle for learning and prediction Regression Classification Teaching Assistent: Rogier van het Schip Hypotheses and Hypothesis Spaces Least Squares E-mail: rsp400@few.vu.nl ● Bio: ● ✦ 6th year AI student ✦ Intends to start graduation work this year 3 / 43
Course Materials Materials: Course Organisation Tentative Course Outline “Machine Learning” by Tom M. Mitchell, McGraw-Hill, 1997 ● What is Machine Learning? Extra materials (on course website) ● This Lecture versus Slides (on course website) ● Mitchell Supervised versus Course Website: Unsupervised Learning www.cwi.nl/˜erven/teaching/0708/ml/ Prediction Important Note: Regression Classification I will not always stick to the book. Don’t forget to study the slides Hypotheses and and extra materials! Hypothesis Spaces Least Squares 4 / 43
Grading Course Organisation Part Relative Weight Tentative Course Homework assignments 40% Outline Intermediate exam 20% What is Machine Learning? Final exam ( ≥ 5.5) 40% This Lecture versus Mitchell 5 ≤ average grade ≤ 6 ⇒ round to whole point ● Supervised versus Unsupervised Else ⇒ round to half point ● Learning To pass: rounded average grade ≥ 6 AND final exam ≥ 5 . 5 ● Prediction Regression Classification Hypotheses and Hypothesis Spaces Least Squares 5 / 43
Homework Assignments Course Organisation Should be submitted using Blackboard before the deadline ● Tentative Course (on the assignment) Outline Late submissions: ● What is Machine Learning? Solutions discussed in class ⇒ reject ✦ This Lecture versus Mitchell Else ⇒ minus half a point per day ✦ Supervised versus Unsupervised Exclude lowest grade ● Learning Average assignment grades, no rounding ● Prediction Unsubmitted ⇒ 1 ● Regression Classification Hypotheses and Hypothesis Spaces Least Squares 6 / 43
Homework Assignments Usually theoretical exercises (math or theory) Course Organisation ● Tentative Course One practical assignment using Weka ● Outline One essay assignment near the end of the course ● What is Machine Learning? This Lecture versus Mitchell Supervised versus Unsupervised Learning Prediction Regression Classification Hypotheses and Hypothesis Spaces Least Squares 7 / 43
Overview Course Organisation Course Organisation ● Tentative Course Tentative Course Outline ● Outline What is Machine Learning? ● What is Machine Learning? This Lecture versus Mitchell ● This Lecture versus Supervised versus Unsupervised Learning ● Mitchell The Most Important Supervised Learning Problems ● Supervised versus Unsupervised Learning ✦ Prediction Prediction ✦ Regression Regression ✦ Classification Classification Hypotheses and Hypothesis Spaces ● Hypotheses and Hypothesis Spaces Least Squares ● Least Squares 8 / 43
Tentative Course Outline Course Organisation Date Topic Tentative Course Sept. 6, 13 Basic concepts , list-then-eliminate algorithm, decision trees Outline Sept. 20 Neural networks What is Machine Sept. 27 Instance-based learning: k-nearest neighbour classifier Learning? Oct. 4 Naive Bayes This Lecture versus Mitchell Oct. 11 Bayesian learning Supervised versus Oct. 18 Minimum description length (MDL) learning Unsupervised Learning ? Intermediate Exam Prediction Oct. 31 Statistical estimation (don’t read Mitchell sect. 5.5.1!) Nov. 7 Support vector machines Regression Nov. 14 Computational learning theory: PAC learning, VC dimension Classification Nov. 21 Graphical models Hypotheses and Hypothesis Spaces Nov. 28 Unsupervised learning: clustering Least Squares Dec. 5 - Dec. 12 The grounding problem, discussion, questions ? Final exam 9 / 43
Overview Course Organisation Course Organisation ● Tentative Course Tentative Course Outline ● Outline What is Machine Learning? ● What is Machine Learning? This Lecture versus Mitchell ● This Lecture versus Supervised versus Unsupervised Learning ● Mitchell The Most Important Supervised Learning Problems ● Supervised versus Unsupervised Learning ✦ Prediction Prediction ✦ Regression Regression ✦ Classification Classification Hypotheses and Hypothesis Spaces ● Hypotheses and Hypothesis Spaces Least Squares ● Least Squares 10 / 43
Machine Learning “Machine Learning is the study of computer algorithms that Course Organisation Tentative Course improve automatically through experience.” – T. M. Mitchell Outline What is Machine For example: Learning? Handwritten digit recognition: examples from MNIST This Lecture versus ● Mitchell database (figure taken from [LeCun et al., 1998]) Supervised versus Unsupervised Learning Prediction Regression Classification Hypotheses and Hypothesis Spaces Least Squares 11 / 43
Machine Learning “Machine Learning is the study of computer algorithms that Course Organisation Tentative Course improve automatically through experience.” – T. M. Mitchell Outline What is Machine For example: Learning? Handwritten digit recognition: examples from MNIST This Lecture versus ● Mitchell database (figure taken from [LeCun et al., 1998]) Supervised versus Unsupervised Classifying genes by gene expression (figure taken from ● Learning [Molla et al.]) Prediction Regression Classification Hypotheses and Hypothesis Spaces Least Squares 11 / 43
Machine Learning “Machine Learning is the study of computer algorithms that Course Organisation Tentative Course improve automatically through experience.” – T. M. Mitchell Outline What is Machine For example: Learning? Handwritten digit recognition: examples from MNIST This Lecture versus ● Mitchell database (figure taken from [LeCun et al., 1998]) Supervised versus Unsupervised Classifying genes by gene expression (figure taken from ● Learning [Molla et al.]) Prediction Evaluating a board state in checkers based on a set of board ● Regression features. E.g. the number of black pieces on the board. (c.f. Classification Mitchell) Hypotheses and Hypothesis Spaces Least Squares 11 / 43
Deduction versus Induction We will (mostly) consider induction rather than deduction. Course Organisation Tentative Course Deduction: a particular case from general principles Outline What is Machine Learning? 1. You need at least a 6 to pass this course. ( A → B ) This Lecture versus 2. You have achieved at least a 6 . ( A ) Mitchell Hence, you pass this course. (Therefore B ) 3. Supervised versus Unsupervised Learning Induction: general laws from particular facts Prediction Regression Name Average Grade Pass? Classification Sanne 7.5 Yes Hypotheses and Sem 6 Yes Hypothesis Spaces Lotte 5 No Least Squares Ruben 9 Yes Sophie 7 Yes Daan 4 No Lieke 6 Yes Me 8 ? 12 / 43
Why Machine Learning? Too much data to analyse by humans (e.g. ranking websites, Course Organisation ● Tentative Course spam filtering, classifying genes by gene expression) Outline Too difficult data representations (e.g. 3D brain scans, angle ● What is Machine Learning? measurements on joints of an industrial robot) This Lecture versus Algorithms for machine learning keep improving ● Mitchell Computation is cheap; humans are expensive ● Supervised versus Unsupervised Some jobs are too boring for humans (e.g. spam filtering) ● Learning . . . ● Prediction Regression Classification Hypotheses and Hypothesis Spaces Least Squares 13 / 43
Overview Course Organisation Course Organisation ● Tentative Course Tentative Course Outline ● Outline What is Machine Learning? ● What is Machine Learning? This Lecture versus Mitchell ● This Lecture versus Supervised versus Unsupervised Learning ● Mitchell The Most Important Supervised Learning Problems ● Supervised versus Unsupervised Learning ✦ Prediction Prediction ✦ Regression Regression ✦ Classification Classification Hypotheses and Hypothesis Spaces ● Hypotheses and Hypothesis Spaces Least Squares ● Least Squares 14 / 43
Recommend
More recommend
