Decision Tree Mahdi Roozbahani Lecturer, Computational Science and - PowerPoint PPT Presentation

Mar 10, 2024 •175 likes •652 views

Class Website CX4242: Decision Tree Mahdi Roozbahani Lecturer, Computational Science and Engineering, Georgia Tech These slides are adopted from Polo, Andrew w. Moore, and Vivek Srikumar 2 1 Visual Introduction to Decision Tree

Class Website CX4242: Decision Tree Mahdi Roozbahani Lecturer, Computational Science and Engineering, Georgia Tech These slides are adopted from Polo, Andrew w. Moore, and Vivek Srikumar
𝑌 2 𝑌 1
Visual Introduction to Decision Tree Building a tree to distinguish homes in New York from homes in San Francisco 3
Decision Tree: Example (2) Will I play tennis today? 4
Decision trees (DT) Outlook? The classifier: f T (x) : majority class in the leaf in the tree T containing x Model parameters: The tree structure and size 5
Decision trees Pieces: 1. Find the best attribute to split on 2. Find the best split on the chosen attribute 3. Decide on when to stop splitting 6
Categorical or Discrete attributes Label
Attribute
Continuous attributes
Test data
Information Content Coin flip Entropy ~ Uncertainty Which coin will give us the purest information? Lower uncertainty, higher information gain
different
What will happen if a tree is too large? Overfitting High variance Instability in predicting test data
How to avoid overfitting? • Acquire more training data • Remove irrelevant attributes (manual process – not always possible) • Grow full tree, then post-prune • Ensemble learning
Reduced-Error Pruning Split data into training and validation sets Grow tree based on training set Do until further pruning is harmful: 1. Evaluate impact on validation set of pruning each possible node (plus those below it) 2. Greedily remove the node that most improves validation set accuracy
How to decide to remove it a node using pruning • Pruning of the decision tree is done by replacing a whole subtree by a leaf node. • The replacement takes place if a decision rule establishes that the expected error rate in the subtree is greater than in the single leaf.

Recommend

Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like

Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible MSE 2400 EaLiCaRA consequences, including chance event Dr. Tom Way outcomes, resource costs, and

568 views • 8 slides

1 Optimization in decision graphs Unfolding to decision tree Only option until Shachter

Decision graphs II Influence Diagrams Advanced Herd Management Anders Ringgaard Kristensen Slide 1 Outline Optimization methods Decision tree Strong junction tree Single Policy Updating Decision node ordering Advantages and

467 views • 10 slides

1 Optimization in decision graphs The repeated milk test problem Unfolding to decision tree The

Outline Decision trees Optimization methods Decision tree Decision graphs II Strong junction tree (Limited Memory) Influence Diagrams Single Policy Updating Markov property versus no forgetting Advanced Herd Management

512 views • 4 slides

A Brief History of Decision Tree Implementation MAX AUSTIN Overview Famous Decision Tree

A Brief History of Decision Tree Implementation MAX AUSTIN Overview Famous Decision Tree Algorithms Chi-squared Automatic Interaction Detector (CHAID) Classification and Regression Tree (CART) Iterative Dichotomiser 3 (ID3)

114 views • 9 slides

Decision Tree and Automata Learning Stefan Edelkamp 1 Overview - Decision tree representation

Decision Tree and Automata Learning Stefan Edelkamp 1 Overview - Decision tree representation - Top Down Induction, Attribute Selection - Entropy, Information gain - ID3 learning algorithm - Overfitting - Continous, multi-valued, costly and

718 views • 33 slides

Decision tree learning Aim: find a small tree consistent with the training examples Idea:

Decision tree learning Aim: find a small tree consistent with the training examples Idea: (recursively) choose most significant attribute as root of (sub)tree function DTL ( examples, attributes, default ) returns a decision tree if examples

261 views • 13 slides

Decision Tree Learning: Part 2 CS 760@UW-Madison Goals for the last lecture you should

Decision Tree Learning: Part 2 CS 760@UW-Madison Goals for the last lecture you should understand the following concepts the decision tree representation the standard top-down approach to learning a tree Occams razor

505 views • 38 slides

Decision-Tree for Classication MACH IN E LEARN IN G W ITH TREE-BAS ED MODELS IN P YTH ON

Decision-Tree for Classication MACH IN E LEARN IN G W ITH TREE-BAS ED MODELS IN P YTH ON Elie Kawerk Data Scientist Course Overview Chap 1 : Classication And Regression Tree (CART) Chap 2 : The Bias-Variance Tradeoff Chap 3 : Bagging

320 views • 29 slides

Assessing The Necessity Survey and Decision Tree Activities Conducted Decision tree created

Assessing The Necessity Survey and Decision Tree Activities Conducted Decision tree created December 2013. Survey closed March 5, 2014. 132 survey responses. 72 public health and 60 non-public health (NF, ALF, HCBS, other).

280 views • 17 slides

Learning Decision Trees Representation is a decision tree. Bias is towards simple decision

Learning Decision Trees Representation is a decision tree. Bias is towards simple decision trees. Search through the space of decision trees, from simple decision trees to more complex ones. Decision trees A decision tree is

484 views • 10 slides

Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees

HTF: 9.2 B: 14.4 R N , C h a p t e r 1 8 1 8 . 3 Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees Algorithm for Learning Decision Trees Entropy, Inductive Bias (Occam's

1.43k views • 87 slides

Decision Tree Algorithm Decision Tree Algorithm Week 4 1 Team Homework Assignment #5 Team

Decision Tree Algorithm Decision Tree Algorithm Week 4 1 Team Homework Assignment #5 Team Homework Assignment #5 Read pp. 105 117 of the text book. R d 105 117 f h b k Do Examples 3.1, 3.2, 3.3 and Exercise 3.4 (a). Prepare

385 views • 21 slides

Chapter 18 Learning from Observations Decision tree examples Additional source used in preparing

Chapter 18 Learning from Observations Decision tree examples Additional source used in preparing the slides: Jean-Claude Latombes CS121 slides: robotics.stanford.edu/~latombe/cs121 1 Decision Trees A decision tree allows a

608 views • 33 slides

Keep the Decision Tree and Estimate the Class Probabilities Using its Decision Boundary Isabelle

Keep the Decision Tree and Estimate the Class Probabilities Using its Decision Boundary Isabelle Alvarez (1 , 2) Stephan Bernard (2) Guillaume Deffuant (2) (1) LIP6, Paris VI University (2) Cemagref, LISC (2) Cemagref, LISC 4 place Jussieu

310 views • 6 slides

Decision Tree CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2019

Decision Tree CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2019 Decision tree } One of the most intuitive classifiers that is easy to understand and construct } However, it also works very (very) well }

453 views • 41 slides

Decision Tree Learning Mitchell, Chapter 3 CptS 570 Machine Learning School of EECS Washington

Decision Tree Learning Mitchell, Chapter 3 CptS 570 Machine Learning School of EECS Washington State University Outline Decision tree representation ID3 learning algorithm Entropy and information gain Overfitting

540 views • 30 slides

Lazy Associative Classification Decision Tree Classifier (Eager) Associative Classifier By

Contents: Classification Lazy Associative Classification Decision Tree Classifier (Eager) Associative Classifier By Adriano Veloso,Wagner Meira Jr. , Mohammad J. Zaki Comparison between Decision Tree and Associative Classifier

105 views • 7 slides

Introduction Decision Tree for PlayTennis (Mitchell) CSCE CSCE 478/878 478/878 Outlook

Introduction Decision Tree for PlayTennis (Mitchell) CSCE CSCE 478/878 478/878 Outlook Lecture 3: Lecture 3: Learning Learning Decision trees form a simple, easily-interpretable, Decision Decision Trees Trees hypothesis Stephen Scott

400 views • 4 slides

Decision Tree Based Learning of Program Invariants Deepak DSouza Department of Computer

Floyd-Hoare Style Verification Decision Tree Learning ICE Learning Proofs with Multiple Invariants Decision Tree Based Learning of Program Invariants Deepak DSouza Department of Computer Science and Automation Indian Institute of Science,

648 views • 37 slides

10-701 Fall 2017 Recitation 3 Agenda Q1 - Decision Tree to KNN A1 Q2.1 - KNN to Decision

10-701 Fall 2017 Recitation 3 Agenda Q1 - Decision Tree to KNN A1 Q2.1 - KNN to Decision Trees Q 2.1 A 2.1 Q2.2 A 2.2 A 2.2 A 2.2 HW 1 doubts ?

261 views • 12 slides

Tree Computation for Ranking and Classification CS240A, T. Yang, 2016 Outlines Decision Trees

Tree Computation for Ranking and Classification CS240A, T. Yang, 2016 Outlines Decision Trees Learning Assembles: Random forest, boosted trees Decision Trees Decision trees can express any function of the input attributes.

452 views • 31 slides

Decision tree learning Andrea Passerini passerini@disi.unitn.it Machine Learning Decision trees

Decision tree learning Andrea Passerini passerini@disi.unitn.it Machine Learning Decision trees Learning the concept Go to lesson OUTLOOK Rain Overcast Sunny TRANSPORTATION LESSON NO Uncovered Covered Theoretical Practical NO YES

393 views • 12 slides

Explaining the result of a Decision Tree to the End-User Isabelle Alvarez 1 2 Abstract. This

Explaining the result of a Decision Tree to the End-User Isabelle Alvarez 1 2 Abstract. This paper addresses the problem of the explanation of tests that are in the trace can have no consequences on the result. the result given by a decision

548 views • 6 slides

Learning from Data: Decision Trees Amos Storkey, School of Informatics University of Edinburgh

Learning from Data: Decision Trees Amos Storkey, School of Informatics University of Edinburgh Semester 1, 2004 LfD 2004 Decision Tree Learning - Overview Decision tree representation ID3 learning algorithm Entropy, Information gain

586 views • 23 slides