Decision Trees TJ Machine Learning Club Classification vs. - PowerPoint PPT Presentation

Dec 28, 2023 •233 likes •505 views

Decision Trees TJ Machine Learning Club Classification vs. Regression Classification Classifying photos of fruits Determining whether tumor is benign or malignant Regression Predicting COVID-19 cases given demographic data

Decision Trees TJ Machine Learning Club
Classification vs. Regression Classification ○ Classifying photos of fruits ■ Determining whether tumor is benign or malignant ■ Regression ○ Predicting COVID-19 cases given demographic data ■ Predicting house prices given house features ■ Source: https://medium.com/datasoc/whats-the-problem-1ff8b338094b
Features Labels Features vs. Labels Features (like x): Characteristics of the input - In the picture, features are whether or not patient smokes (smoke), consumes alcohol (alco), and performs physical activity (active) Label (like y): The prediction or classification of the input - Whether or not patient has cardiovascular disease (cardio)
Training and Testing Datasets Training data has both features and labels Testing data only has the features Need to predict cardio
What is a Decision Tree? A decision tree is just a series of questions ● The key in creating a decision tree is asking the right questions ●
Gini Impurity - Measure of how “messy” some collection of data is i = some data k = class index c = total number of classes p(k|i) = probability of randomly selecting item of class k from data
Ex. Gini Impurity Let’s calculate the Gini Impurity for these groups of data, where the two possible classes are blue or red:
Ex. Gini Impurity 0.444 0.5 0
Ex. Gini Impurity 0.444 0.5 0 Minimum possible impurity Maximum possible impurity
Information Gain - D p , D left , D right are the parent node, left node dataset, and right node dataset respectively - I is a measure of impurity (like Gini Impurity) - N p , N left , and N right are the number of items in the parent, left, and right nodes respectively - f is the question you are asking to create the split
Let’s figure out which question is a better question T = Tennis Player to ask to split the athletes according to sport B = Basketball Player T B T B B T T B T B B T Age > 27? Height > 6’4’’ Y Y N N T B B T T B T T B B B T
T B T B B T Age > 27? Y N T B B B T T
1/2 T B T B B T Age > 27? Y N 4/9 4/9 T B B B T T
1/2 T B T B B T Age > 27? Y N 4/9 4/9 T B B B T T
T B T B B T Height > 6’4’’ Y N T T B B B T
1/2 T B T B B T Height > 6’4’’ Y N 0 3/8 T T B B B T
1/2 T B T B B T Height > 6’4’’ Y N 0 3/8 T T B B B T
Information Gain: 0.055556 Information Gain: 0.25 Since Information Gain is higher, this the better question to ask to classify our athletes T B T B B T T B T B B T Age > 27? Height > 6’4’’ Y Y N N T B B T T B T T B B B T
How to Come Up with Values for the Questions? - The most straightforward way: Try out different values from the items in your training dataset
Overfitting Techniques to prevent overfitting in decision trees: ● Continue recursively generating nodes only if information gain is larger than some threshold (e.g. ● 0.1) After creating the tree, prune all nodes that are at a depth greater than some threshold ●

Recommend

Decision Trees Lecture 23 To left or to right 1 Decision Trees 2 Decision Trees A different

Decision Trees Lecture 23 To left or to right 1 Decision Trees 2 Decision Trees A different complexity measure 2 Decision Trees A different complexity measure Number of bits of input read 2 Decision Trees A different complexity measure

1.3k views • 107 slides

Decision Trees Lecture 22 To left or to right 1 Decision Trees 2 Decision Trees A different

Decision Trees Lecture 22 To left or to right 1 Decision Trees 2 Decision Trees A different complexity measure 2 Decision Trees A different complexity measure Number of bits of input read 2 Decision Trees A different complexity measure

1.62k views • 107 slides

Lecture 23: Decision Trees Decision trees Prof. Julia Hockenmaier

CS440/ECE448: Intro to Artificial Intelligence Lecture 23: Decision Trees Decision trees Prof. Julia Hockenmaier juliahmr@illinois.edu http://cs.illinois.edu/fa11/cs440 Decision trees Decision tree

338 views • 7 slides

Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees

HTF: 9.2 B: 14.4 R N , C h a p t e r 1 8 1 8 . 3 Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees Algorithm for Learning Decision Trees Entropy, Inductive Bias (Occam's

1.43k views • 87 slides

1 Baysian networks for decision analysis The TV show problem as a BN Problems More than

Several names Decision graphs Decision trees Influence diagrams Decision graphs I A certain kind of strictly symmetric decision trees Decisions and utilities No forgetting assumption LIMIDs Limited Memory

596 views • 6 slides

1 Baysian networks for decision analysis The TV show problem as a BN Problems Choice 1

185 views • 6 slides

Learning Decision Trees Representation is a decision tree. Bias is towards simple decision

Learning Decision Trees Representation is a decision tree. Bias is towards simple decision trees. Search through the space of decision trees, from simple decision trees to more complex ones. Decision trees A decision tree is

484 views • 10 slides

Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like

Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible MSE 2400 EaLiCaRA consequences, including chance event Dr. Tom Way outcomes, resource costs, and

568 views • 8 slides

Tree Computation for Ranking and Classification CS240A, T. Yang, 2016 Outlines Decision Trees

Tree Computation for Ranking and Classification CS240A, T. Yang, 2016 Outlines Decision Trees Learning Assembles: Random forest, boosted trees Decision Trees Decision trees can express any function of the input attributes.

452 views • 31 slides

15-388/688 - Practical Data Science: Decision trees and interpretable models J. Zico Kolter

15-388/688 - Practical Data Science: Decision trees and interpretable models J. Zico Kolter Carnegie Mellon University Spring 2018 1 Outline Decision trees Training (classification) decision trees Interpreting predictions Boosting

381 views • 37 slides

Machine Learning III: Beyond Decision Trees Extensions to Decision Trees AI Class 15 (Ch.

Todays Class Machine Learning III: Beyond Decision Trees Extensions to Decision Trees AI Class 15 (Ch. 20.120.2) Sources of error Evaluating learned models B E Bayesian Learning E [1] B [1] A [1] C [1]

137 views • 3 slides

Medical Decision Making Learning: Decision Trees Artificial Intelligence CSPP 56553 February

Medical Decision Making Learning: Decision Trees Artificial Intelligence CSPP 56553 February 11, 2004 Agenda Decision Trees: Motivation: Medical Experts: Mycin Basic characteristics Sunburn example From trees to rules

578 views • 28 slides

Decision Trees 2-26-16 Reading Quiz Decision trees are an algorithm for which machine learning

Decision Trees 2-26-16 Reading Quiz Decision trees are an algorithm for which machine learning task? a) clustering b) dimensionality reduction c) classification d) regression Reading Quiz Which error metric is most appropriate for

483 views • 14 slides

Machine Learning and Data Mining Decision Trees Kalev Kask Decision trees Functional form

+ Machine Learning and Data Mining Decision Trees Kalev Kask Decision trees Functional form f(x; ): nested if -then- else statements Discrete features: fully expressive (any function) Structure: Internal nodes: check

303 views • 27 slides

Wrap Up! Lecture 25 Decision Trees & Branching Programs Many Topics Not Covered! Decision

Wrap Up! Lecture 25 Decision Trees & Branching Programs Many Topics Not Covered! Decision Trees Another model of non-uniform computation A full binary tree with each internal Q 0 node labelled with an elementary boolean function of

490 views • 19 slides

IAML: Decision Trees Chris Williams and Victor Lavrenko School of Informatics Semester 1 1 / 17

IAML: Decision Trees Chris Williams and Victor Lavrenko School of Informatics Semester 1 1 / 17 Outline Decision trees: the idea Examples Top-down induction of decision trees Entropy Information gain Overfitting and

205 views • 17 slides

CSC 411 Lecture 3: Decision Trees Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 3: Decision Trees Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 03-Decision Trees 1 / 33 Today Decision Trees Simple but powerful learning algorithm One of the most

1.09k views • 33 slides

Optimal Sparse Decision Trees Xiyang Hu Cynthia Rudin Margo Seltzer Carnegie Mellon Duke

Optimal Sparse Decision Trees Xiyang Hu Cynthia Rudin Margo Seltzer Carnegie Mellon Duke University University of British University Columbia Decision Trees Decision Trees Decision Trees Should I click on the link in this email? Do I

589 views • 35 slides

Decision Trees Thomas Schwarz, SJ Decision Trees One of many machine learning methods

Decision Trees Thomas Schwarz, SJ Decision Trees One of many machine learning methods Used to learn categories Example: The Iris Data Set Four measurements of flowers Learn how to predict species from measurement Iris Data

405 views • 40 slides

Decision trees PRISM - Nicolas Sutton-Charani 20/01/2020 N. Sutton-Charani Artificial

Decision trees PRISM - Nicolas Sutton-Charani 20/01/2020 N. Sutton-Charani Artificial intelligence Decision trees 1 / 47 1. Introduction 2. Use of decision trees 2.1 Prediction 2.2 Interpretability : Descriptive data analysis 3. Learning

478 views • 47 slides

Introduction to Machine Learning CMU-10701 23. Decision Trees Barnabs Pczos Contents

Introduction to Machine Learning CMU-10701 23. Decision Trees Barnabs Pczos Contents Decision Trees: Definition + Motivation Algorithm for Learning Decision Trees Entropy, Mutual Information, Information gain Generalizations

1.06k views • 68 slides

Learning Decision Trees Machine Learning 1 Some slides from Tom Mitchell, Dan Roth and others

Learning Decision Trees Machine Learning 1 Some slides from Tom Mitchell, Dan Roth and others This lecture: Learning Decision Trees 1. Representation : What are decision trees? 2. Algorithm : Learning decision trees The ID3 algorithm: A greedy

1.2k views • 67 slides

Decision Trees and Nave Bayes 3/29/17 Hypothesis Spaces Decision Trees and K-Nearest

Decision Trees and Nave Bayes 3/29/17 Hypothesis Spaces Decision Trees and K-Nearest Neighbors Continuous inputs Discrete outputs Nave Bayes Discrete inputs Discrete outputs Building a Decision Tree Greedy algorithm:

468 views • 17 slides

Implementation of Decision Trees using R Margaret Mir-Juli, Arnau Mir and Monica J.

Implementation of Decision Trees using R Margaret Mir-Juli, Arnau Mir and Monica J. Ruiz-Mir University of the Balearic Islands Palma de Mallorca, SPAIN Implementation of Decision Trees using R Data vs. Knowledge A large collection of

335 views • 21 slides