decision trees
play

Decision Trees Petr Po s k This lecture is largely based on the - PowerPoint PPT Presentation

Jan 10, 2024 •317 likes •800 views

CZECH TECHNICAL UNIVERSITY IN PRAGUE Faculty of Electrical Engineering Department of Cybernetics Decision Trees Petr Po s k This lecture is largely based on the book Artificial Intelligence: A Modern Approach, 3rd ed. by Stuart

  1. CZECH TECHNICAL UNIVERSITY IN PRAGUE Faculty of Electrical Engineering Department of Cybernetics Decision Trees Petr Poˇ s´ ık This lecture is largely based on the book Artificial Intelligence: A Modern Approach, 3rd ed. by Stuart Russell and Peter Norvig (Prentice Hall, 2010). P. Poˇ s´ ık c � 2015 Artificial Intelligence – 1 / 29

  2. Decision Trees P. Poˇ s´ ık c � 2015 Artificial Intelligence – 2 / 29

  3. What is a decision tree? Decision tree ■ is a function that Decision Trees ■ takes a vector of attribute values as its input, and • What is a decision tree? ■ returns a “decision” as its output. • Attribute description ■ Both input and output values can be measured on a nominal, ordinal, interval, • Expressiveness of and ratio scales, can be discrete or continuous. decision trees Learning a Decision ■ The decision is formed via a sequence of tests: Tree Generalization and ■ each internal node of the tree represents a test, Overfitting ■ the branches are labeled with possible outcomes of the test, and Broadening the Applicability of ■ each leaf node represents a decision to be returned by the tree. Desicion Trees Summary P. Poˇ s´ ık c � 2015 Artificial Intelligence – 3 / 29

  4. What is a decision tree? Decision tree ■ is a function that Decision Trees ■ takes a vector of attribute values as its input, and • What is a decision tree? ■ returns a “decision” as its output. • Attribute description ■ Both input and output values can be measured on a nominal, ordinal, interval, • Expressiveness of and ratio scales, can be discrete or continuous. decision trees Learning a Decision ■ The decision is formed via a sequence of tests: Tree Generalization and ■ each internal node of the tree represents a test, Overfitting ■ the branches are labeled with possible outcomes of the test, and Broadening the Applicability of ■ each leaf node represents a decision to be returned by the tree. Desicion Trees Summary Decision trees examples: ■ classification schemata in biology (urˇ covac´ ı kl´ ıˇ ce) ■ diagnostic sections in illness encyclopedias ■ online troubleshooting section on software web pages ■ . . . P. Poˇ s´ ık c � 2015 Artificial Intelligence – 3 / 29

  5. Attribute description Example: A computer game. The main character of the game meets various robots along his way. Some behave like allies, others like enemies. Decision Trees • What is a decision tree? ally head body smile neck holds class • Attribute description circle circle yes tie nothing ally • Expressiveness of decision trees circle square no tie sword enemy Learning a Decision . . . . . . . . . . . . . . . . . . Tree Generalization and Overfitting The game engine may use e.g. the following tree to assign the Broadening the ally or enemy attitude to the generated robots: Applicability of Desicion Trees enemy neck Summary other tie body smile triangle other s e no y ally enemy ally enemy P. Poˇ s´ ık c � 2015 Artificial Intelligence – 4 / 29

  6. Expressiveness of decision trees The tree on previous slide is a Boolean decision tree: ■ the decision is a binary variable (true, false), and Decision Trees ■ the attributes are discrete. • What is a decision tree? ■ It returns ally iff the input attributes satisfy one of the paths leading to an ally leaf: • Attribute description ally ⇔ ( neck = tie ∧ smile = yes ) ∨ ( neck = ¬ tie ∧ body = triangle ) , • Expressiveness of decision trees Learning a Decision i.e. in general Tree Goal ⇔ ( Path 1 ∨ Path 2 ∨ . . . ) , where Generalization and ■ Overfitting Path is a conjuction of attribute-value tests, i.e. ■ Broadening the Applicability of ■ the tree is equivalent to a DNF of a function. Desicion Trees Summary Any function in propositional logic can be expressed as a dec. tree. ■ Trees are a suitable representation for some functions and unsuitable for others. ■ What is the cardinality of the set of Boolean functions of n attributes? ■ It is equal to the number of truth tables that can be created with n attributes. ■ The truth table has 2 n rows, i.e. there is 2 2 n different functions. ■ The set of trees is even larger; several trees represent the same function. ■ We need a clever algorithm to find good hypotheses (trees) in such a large space. P. Poˇ s´ ık c � 2015 Artificial Intelligence – 5 / 29

  7. Learning a Decision Tree P. Poˇ s´ ık c � 2015 Artificial Intelligence – 6 / 29

  8. A computer game Example 1: Can you distinguish between allies and enemies after seeing a few of them? Decision Trees Enemies Allies Learning a Decision Tree • A computer game • A computer game • Alternative hypotheses • How to choose the best tree? • Learning a Decision Tree • Attribute importance • Choosing the test attribute • Choosing the test attribute (special case: binary classification) • Choosing the test attribute (example) • Choosing subsequent test attribute • Decision tree building procedure • Algorithm characteristics Generalization and Overfitting Broadening the Applicability of Desicion Trees Summary P. Poˇ s´ ık c � 2015 Artificial Intelligence – 7 / 29

  9. A computer game Example 1: Can you distinguish between allies and enemies after seeing a few of them? Decision Trees Enemies Allies Learning a Decision Tree • A computer game • A computer game • Alternative hypotheses • How to choose the best tree? • Learning a Decision Tree • Attribute importance • Choosing the test attribute • Choosing the test attribute (special case: binary classification) • Choosing the test attribute (example) • Choosing subsequent test attribute • Decision tree building procedure • Algorithm characteristics Hint: concentrate on the shapes of heads and bodies. Generalization and Overfitting Broadening the Applicability of Desicion Trees Summary P. Poˇ s´ ık c � 2015 Artificial Intelligence – 7 / 29

  10. A computer game Example 1: Can you distinguish between allies and enemies after seeing a few of them? Decision Trees Enemies Allies Learning a Decision Tree • A computer game • A computer game • Alternative hypotheses • How to choose the best tree? • Learning a Decision Tree • Attribute importance • Choosing the test attribute • Choosing the test attribute (special case: binary classification) • Choosing the test attribute (example) • Choosing subsequent test attribute • Decision tree building procedure • Algorithm characteristics Hint: concentrate on the shapes of heads and bodies. Generalization and Overfitting Answer: Seems like allies have the same shape of their head and body. Broadening the How would you represent this by a decision tree? (Relation among attributes.) Applicability of Desicion Trees Summary P. Poˇ s´ ık c � 2015 Artificial Intelligence – 7 / 29

  11. A computer game Example 1: Can you distinguish between allies and enemies after seeing a few of them? Decision Trees Enemies Allies Learning a Decision Tree • A computer game • A computer game • Alternative hypotheses • How to choose the best tree? • Learning a Decision Tree • Attribute importance • Choosing the test attribute • Choosing the test attribute (special case: binary classification) • Choosing the test attribute (example) • Choosing subsequent test attribute • Decision tree building procedure • Algorithm characteristics Hint: concentrate on the shapes of heads and bodies. Generalization and Overfitting Answer: Seems like allies have the same shape of their head and body. Broadening the How would you represent this by a decision tree? (Relation among attributes.) Applicability of How do you know that you are right? Desicion Trees Summary P. Poˇ s´ ık c � 2015 Artificial Intelligence – 7 / 29

  12. A computer game Example 2: Some robots changed their attitudes: Decision Trees Enemies Allies Learning a Decision Tree • A computer game • A computer game • Alternative hypotheses • How to choose the best tree? • Learning a Decision Tree • Attribute importance • Choosing the test attribute • Choosing the test attribute (special case: binary classification) • Choosing the test attribute (example) • Choosing subsequent test attribute • Decision tree building procedure • Algorithm characteristics Generalization and Overfitting Broadening the Applicability of Desicion Trees Summary P. Poˇ s´ ık c � 2015 Artificial Intelligence – 8 / 29

  13. A computer game Example 2: Some robots changed their attitudes: Decision Trees Enemies Allies Learning a Decision Tree • A computer game • A computer game • Alternative hypotheses • How to choose the best tree? • Learning a Decision Tree • Attribute importance • Choosing the test attribute • Choosing the test attribute (special case: binary classification) • Choosing the test attribute (example) • Choosing subsequent test attribute • Decision tree building procedure • Algorithm characteristics No obvious simple rule. Generalization and Overfitting How to build a decision tree discriminating the 2 robot classes? Broadening the Applicability of Desicion Trees Summary P. Poˇ s´ ık c � 2015 Artificial Intelligence – 8 / 29

Recommend

Decision Trees Lecture 23 To left or to right 1 Decision Trees 2 Decision Trees A different

Decision Trees Lecture 23 To left or to right 1 Decision Trees 2 Decision Trees A different

Decision Trees Lecture 23 To left or to right 1 Decision Trees 2 Decision Trees A different complexity measure 2 Decision Trees A different complexity measure Number of bits of input read 2 Decision Trees A different complexity measure

1.3k views • 107 slides
Decision Trees Lecture 22 To left or to right 1 Decision Trees 2 Decision Trees A different

Decision Trees Lecture 22 To left or to right 1 Decision Trees 2 Decision Trees A different

Decision Trees Lecture 22 To left or to right 1 Decision Trees 2 Decision Trees A different complexity measure 2 Decision Trees A different complexity measure Number of bits of input read 2 Decision Trees A different complexity measure

1.62k views • 107 slides
Lecture 23: Decision Trees Decision trees Prof. Julia Hockenmaier

Lecture 23: Decision Trees Decision trees Prof. Julia Hockenmaier

CS440/ECE448: Intro to Artificial Intelligence Lecture 23: Decision Trees Decision trees Prof. Julia Hockenmaier juliahmr@illinois.edu http://cs.illinois.edu/fa11/cs440 Decision trees Decision tree

338 views • 7 slides
Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees

Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees

HTF: 9.2 B: 14.4 R N , C h a p t e r 1 8 1 8 . 3 Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees Algorithm for Learning Decision Trees Entropy, Inductive Bias (Occam's

1.43k views • 87 slides
1 Baysian networks for decision analysis The TV show problem as a BN Problems More than

1 Baysian networks for decision analysis The TV show problem as a BN Problems More than

Several names Decision graphs Decision trees Influence diagrams Decision graphs I A certain kind of strictly symmetric decision trees Decisions and utilities No forgetting assumption LIMIDs Limited Memory

596 views • 6 slides
1 Baysian networks for decision analysis The TV show problem as a BN Problems Choice 1

1 Baysian networks for decision analysis The TV show problem as a BN Problems Choice 1

Several names Decision graphs Decision trees Influence diagrams Decision graphs I A certain kind of strictly symmetric decision trees Decisions and utilities No forgetting assumption LIMIDs Limited Memory

185 views • 6 slides
Learning Decision Trees Representation is a decision tree. Bias is towards simple decision

Learning Decision Trees Representation is a decision tree. Bias is towards simple decision

Learning Decision Trees Representation is a decision tree. Bias is towards simple decision trees. Search through the space of decision trees, from simple decision trees to more complex ones. Decision trees A decision tree is

484 views • 10 slides
Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like

Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like

Decision Tree Decision Trees A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible MSE 2400 EaLiCaRA consequences, including chance event Dr. Tom Way outcomes, resource costs, and

568 views • 8 slides
Tree Computation for Ranking and Classification CS240A, T. Yang, 2016 Outlines Decision Trees

Tree Computation for Ranking and Classification CS240A, T. Yang, 2016 Outlines Decision Trees

Tree Computation for Ranking and Classification CS240A, T. Yang, 2016 Outlines Decision Trees Learning Assembles: Random forest, boosted trees Decision Trees Decision trees can express any function of the input attributes.

452 views • 31 slides
15-388/688 - Practical Data Science: Decision trees and interpretable models J. Zico Kolter

15-388/688 - Practical Data Science: Decision trees and interpretable models J. Zico Kolter

15-388/688 - Practical Data Science: Decision trees and interpretable models J. Zico Kolter Carnegie Mellon University Spring 2018 1 Outline Decision trees Training (classification) decision trees Interpreting predictions Boosting

381 views • 37 slides
Machine Learning III: Beyond Decision Trees Extensions to Decision Trees AI Class 15 (Ch.

Machine Learning III: Beyond Decision Trees Extensions to Decision Trees AI Class 15 (Ch.

Todays Class Machine Learning III: Beyond Decision Trees Extensions to Decision Trees AI Class 15 (Ch. 20.120.2) Sources of error Evaluating learned models B E Bayesian Learning E [1] B [1] A [1] C [1]

137 views • 3 slides
Medical Decision Making Learning: Decision Trees Artificial Intelligence CSPP 56553 February

Medical Decision Making Learning: Decision Trees Artificial Intelligence CSPP 56553 February

Medical Decision Making Learning: Decision Trees Artificial Intelligence CSPP 56553 February 11, 2004 Agenda Decision Trees: Motivation: Medical Experts: Mycin Basic characteristics Sunburn example From trees to rules

578 views • 28 slides
Decision Trees 2-26-16 Reading Quiz Decision trees are an algorithm for which machine learning

Decision Trees 2-26-16 Reading Quiz Decision trees are an algorithm for which machine learning

Decision Trees 2-26-16 Reading Quiz Decision trees are an algorithm for which machine learning task? a) clustering b) dimensionality reduction c) classification d) regression Reading Quiz Which error metric is most appropriate for

483 views • 14 slides
Machine Learning and Data Mining Decision Trees Kalev Kask Decision trees Functional form

Machine Learning and Data Mining Decision Trees Kalev Kask Decision trees Functional form

+ Machine Learning and Data Mining Decision Trees Kalev Kask Decision trees Functional form f(x; ): nested if -then- else statements Discrete features: fully expressive (any function) Structure: Internal nodes: check

303 views • 27 slides
Wrap Up! Lecture 25 Decision Trees & Branching Programs Many Topics Not Covered! Decision

Wrap Up! Lecture 25 Decision Trees & Branching Programs Many Topics Not Covered! Decision

Wrap Up! Lecture 25 Decision Trees & Branching Programs Many Topics Not Covered! Decision Trees Another model of non-uniform computation A full binary tree with each internal Q 0 node labelled with an elementary boolean function of

490 views • 19 slides
IAML: Decision Trees Chris Williams and Victor Lavrenko School of Informatics Semester 1 1 / 17

IAML: Decision Trees Chris Williams and Victor Lavrenko School of Informatics Semester 1 1 / 17

IAML: Decision Trees Chris Williams and Victor Lavrenko School of Informatics Semester 1 1 / 17 Outline Decision trees: the idea Examples Top-down induction of decision trees Entropy Information gain Overfitting and

205 views • 17 slides
CSC 411 Lecture 3: Decision Trees Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 3: Decision Trees Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 3: Decision Trees Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 03-Decision Trees 1 / 33 Today Decision Trees Simple but powerful learning algorithm One of the most

1.09k views • 33 slides
Optimal Sparse Decision Trees Xiyang Hu Cynthia Rudin Margo Seltzer Carnegie Mellon Duke

Optimal Sparse Decision Trees Xiyang Hu Cynthia Rudin Margo Seltzer Carnegie Mellon Duke

Optimal Sparse Decision Trees Xiyang Hu Cynthia Rudin Margo Seltzer Carnegie Mellon Duke University University of British University Columbia Decision Trees Decision Trees Decision Trees Should I click on the link in this email? Do I

589 views • 35 slides
Decision Trees Thomas Schwarz, SJ Decision Trees One of many machine learning methods

Decision Trees Thomas Schwarz, SJ Decision Trees One of many machine learning methods

Decision Trees Thomas Schwarz, SJ Decision Trees One of many machine learning methods Used to learn categories Example: The Iris Data Set Four measurements of flowers Learn how to predict species from measurement Iris Data

405 views • 40 slides
Decision trees PRISM - Nicolas Sutton-Charani 20/01/2020 N. Sutton-Charani Artificial

Decision trees PRISM - Nicolas Sutton-Charani 20/01/2020 N. Sutton-Charani Artificial

Decision trees PRISM - Nicolas Sutton-Charani 20/01/2020 N. Sutton-Charani Artificial intelligence Decision trees 1 / 47 1. Introduction 2. Use of decision trees 2.1 Prediction 2.2 Interpretability : Descriptive data analysis 3. Learning

478 views • 47 slides
Introduction to Machine Learning CMU-10701 23. Decision Trees Barnabs Pczos Contents

Introduction to Machine Learning CMU-10701 23. Decision Trees Barnabs Pczos Contents

Introduction to Machine Learning CMU-10701 23. Decision Trees Barnabs Pczos Contents Decision Trees: Definition + Motivation Algorithm for Learning Decision Trees Entropy, Mutual Information, Information gain Generalizations

1.06k views • 68 slides
Learning Decision Trees Machine Learning 1 Some slides from Tom Mitchell, Dan Roth and others

Learning Decision Trees Machine Learning 1 Some slides from Tom Mitchell, Dan Roth and others

Learning Decision Trees Machine Learning 1 Some slides from Tom Mitchell, Dan Roth and others This lecture: Learning Decision Trees 1. Representation : What are decision trees? 2. Algorithm : Learning decision trees The ID3 algorithm: A greedy

1.2k views • 67 slides
Decision Trees and Nave Bayes 3/29/17 Hypothesis Spaces Decision Trees and K-Nearest

Decision Trees and Nave Bayes 3/29/17 Hypothesis Spaces Decision Trees and K-Nearest

Decision Trees and Nave Bayes 3/29/17 Hypothesis Spaces Decision Trees and K-Nearest Neighbors Continuous inputs Discrete outputs Nave Bayes Discrete inputs Discrete outputs Building a Decision Tree Greedy algorithm:

468 views • 17 slides
Implementation of Decision Trees using R Margaret Mir-Juli, Arnau Mir and Monica J.

Implementation of Decision Trees using R Margaret Mir-Juli, Arnau Mir and Monica J.

Implementation of Decision Trees using R Margaret Mir-Juli, Arnau Mir and Monica J. Ruiz-Mir University of the Balearic Islands Palma de Mallorca, SPAIN Implementation of Decision Trees using R Data vs. Knowledge A large collection of

335 views • 21 slides

More recommend