Mac hine L e ar ning Intr oduc tion Stanley Liang, PhD York - PDF document

6/2/2017 Mac hine L e ar ning Intr oduc tion Stanley Liang, PhD York University What is Mac hine L e ar ning? Definitio n • “The goal of machine learning is to program computers to use example data or past experience to solve a given problem. Many successful applications of machine learning exist already, including systems that analyze past sales data to predict customer behavior, optimize robot behavior so that a task can be completed using minimum resources, and extract knowledge from bioinformatics data.” – ACM ( Association for Computing Machinery ) • “ How do we create computer programs that improve with experience ” – Tom Mitchell • “Machine learning is the science of getting computers to act without being explicitly programed.” – Andrew Ng 1

6/2/2017 T ypic al Mac hine L e ar ning topic s Supervised Learning Unsupervised Learning Supervised learning is a type of machine learning algorithm that Unsupervised learning is a type of machine learning uses a known dataset (called the training dataset) to make algorithm used to draw inferences from datasets consisting predictions. The training dataset includes input data and of input data without labeled responses. response values. From it, the supervised learning algorithm seeks to build a model that can make predictions of the response values for a new dataset. Classification / Regression Clustering, semi ‐ supervised learning • Clustering: the method for exploratory data • Classification: for categorical response values, where the analysis to find hidden patterns or grouping in data can be separated into specific “classes”. data. The clusters are modeled using a measure of • Regression: for continuous-response values. similarity which is defined upon metrics such as Euclidean or probabilistic distance. • Semi ‐ supervised learning is a class of supervised learning tasks and techniques that also make use of unlabeled data for training – typically a small amount of labeled data with a large amount of unlabeled data. Supe r vise d L e ar ning Algor ithms • Supervised learning is to build a model that makes predictions based on evidence in the presence of uncertainty. • As adaptive algorithms identify patterns in data, a computer ʺ learns ʺ from the observations. When exposed to more observations, the computer improves its predictive performance. • Typical Algorithms – Classification: Decision Trees, Discriminant Analysis, Naive Bayes, k Nearest Neighbors (kNN), Support Vector Machines (SVM), Classification Ensembles ( a predictive model composed of a weighted combination of multiple classification models) – Regression: Linear Regression, Generalized Linear Models, Nonlinear Regression, Support Vector Machines (SVM), Gaussian Process Regression Models, Decision Trees, Regression Tree Ensembles 2

6/2/2017 Unsupe r vise d L e ar ning Algor ithms • Unsupervised learning is a type of machine learning algorithm used to draw inferences from datasets consisting of input data without labeled responses. • Typical Algorithms – Hierarchical clustering : builds a multilevel hierarchy of clusters by creating a cluster tree – k ‐ Means clustering : partitions data into k distinct clusters based on distance to the centroid of a cluster – Gaussian mixture models : models clusters as a mixture of multivariate normal density components – Self ‐ organizing maps : uses neural networks that learn the topology and distribution of the data – Hidden Markov models : uses observed data to recover the sequence of states • Unsupervised learning methods are used in bioinformatics for sequence analysis and genetic clustering; in data mining for sequence and pattern mining; in medical imaging for image segmentation; and in computer vision for object recognition Mac hine L e ar ning F lowc har t 3

6/2/2017 Mac hine L e ar ning Apps Classification Learner App Regression Learner App Unde r - fitting vs. Ove r - fitting • Underfitting occurs when a statistical • Overfitting occurs when a model is model or machine learning algorithm excessively complex, such as having cannot capture the underlying trend too many parameters relative to the of the data. number of observations. • Underfitting would occur when fitting • Overfitting model has poor predictive a linear model to non ‐ linear data. performance, as it overreacts to minor Such a model would have poor fluctuations in the training data. predictive performance. 4

6/2/2017 E valuate a Mac hine L e ar ning Mode l • Cross validation : a model validation technique for assessing how the results of a statistical analysis will generalize to an independent data set. It is mainly used to estimate how accurately a predictive model will perform in practice. • Confusion matrix : a specific table layout that allows visualization of the performance of an algorithm. Each column of the matrix represents the instances in a predicted class while each row represents the instances in an actual class. 5