Learning Terminological Na ve Bayesian Classifiers Under Different - PowerPoint PPT Presentation

Learning Terminological Na¨ ıve Bayesian Classifiers Under Different Assumptions on Missing Knowledge Pasquale Minervini Claudia d’Amato Nicola Fanizzi Department of Computer Science University of Bari URSW 2011 ⋄ Bonn, October 23, 2011

Introduction & Motivation Background Learning a Terminological Na¨ ıve Bayesian Network Classifying individuals with a TBN Conclusions and Future Works Contents Introduction & Motivation 1 Background 2 Learning a Terminological Na¨ ıve Bayesian Network 3 Classifying individuals with a TBN 4 Conclusions and Future Works 5 C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Background Learning a Terminological Na¨ ıve Bayesian Network Classifying individuals with a TBN Conclusions and Future Works Introduction & Motivations Uncertainty is inherently present in real-world knowledge In the SW context difficulties arise modeling real-world domains using only purely logical formalisms Several approaches for coping with unceratin knowledge have been proposed (probabilistic, fuzzy,...) usually probabilistic information is assumed to be available the CWA is adopted ⇓ Exploiting an already populated ontology, a method capturing probabilistic information could be of help the OWA has to be taken into account C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Background Learning a Terminological Na¨ ıve Bayesian Network Classifying individuals with a TBN Conclusions and Future Works Paper Contributions Proposal of a Terminological na¨ ıve Bayesian classifier for predicting class-membership probabilistically it is a na¨ ıve Bayesian network modeling the conditional dependencies between a learned set of Description Logic (complex) concepts and a target concept it deals with the incomplete knowledge due the OWA by considering different ignorance models: Missing Completely at Random Missing at Random Informatively Missing C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Background The Reference Representation Language Learning a Terminological Na¨ ıve Bayesian Network Bayesian Networks Classifying individuals with a TBN Conclusions and Future Works Knowledge Base Representation Assumption: resources, concepts and relationships are defined in terms of a representation that can be mapped to some DL language (with the standard model-theoretic semantics) K = �T , A� T-box T is a set of definitions A-box A contains extensional assertions on concepts and roles e.g. C ( a ) and R ( a , b ) The set of the individuals (resources) occurring in A will be denoted Ind( A ) C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Background The Reference Representation Language Learning a Terminological Na¨ ıve Bayesian Network Bayesian Networks Classifying individuals with a TBN Conclusions and Future Works Basics of Bayesian Networks... A Bayesian network (BN) is a DAG G representing the conditional dependencies in a set of random variables Each vertex in G corresponds to a random variable X i Each edge in G indicates a direct influence relation between the two connected random variables A set of conditional probability distributions θ G is associated with each vertex Each vertex X i in G is conditionally independent of any subset S ⊆ Nd ( X i ) of vertices that are not descendants of X i C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Background The Reference Representation Language Learning a Terminological Na¨ ıve Bayesian Network Bayesian Networks Classifying individuals with a TBN Conclusions and Future Works ...Basics of Bayesian Networks The joint probability distribution Pr( X 1 , . . . , X n ) over a set of random variables { X 1 , . . . , X n } is computed as n � Pr( X 1 , . . . , X n ) = Pr( X i | parents ( X i )); i =1 Given a BN, it is possible to evaluate inference queries by marginalization To decrease the inference complexity the na¨ ıve Bayes network is often considered it is assumed that the presence (or absence) of a particular feature (random variable) of a class is unrelated to the presence (or absence) of any other feature, given the class variable (random variable) C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Defining a Terminological Na¨ ıve Bayesian Network Background Learning a TBN: Problem Definition Learning a Terminological Na¨ ıve Bayesian Network TBN: the Learning Algorithm Classifying individuals with a TBN The Ignorance Models Conclusions and Future Works Terminological Na¨ ıve Bayesian Network: Definition A Terminological Bayesian Network (TBN) N K , w.r.t. a DL KB K , is defined as a pair �G , Θ G � , where: G = �V , E� is a directed acyclic graph, in which: V = { F 1 , . . . , F n , C } is a set of vertices, each F i representing a DL (eventually complex) concepts defined over K and C representing a target concept E ⊆ V × V is a set of edges, modeling the dependence relations between the elements of V ; Θ G is a set of conditional probability distributions (CPD), one for each V ∈ V , representing the conditional probability of the feature concept given its parents in the graph In the case of a Terminological Na¨ ıve Bayesian Network , E = {� C , F i � | i ∈ { 1 , . . . , n }} , namely ∀ i , j ∈ { 1 , . . . , n } and i � = j F i is independent of F j given the value of the target concept C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Defining a Terminological Na¨ ıve Bayesian Network Background Learning a TBN: Problem Definition Learning a Terminological Na¨ ıve Bayesian Network TBN: the Learning Algorithm Classifying individuals with a TBN The Ignorance Models Conclusions and Future Works Terminological Na¨ ıve Bayesian Network: Example Given: a set of DL feature concepts F = { Female , HasChild := ∃ hasChild . Person } (variable names are used instead of complex feature concepts) a target concept Father the Terminological Na¨ ıve Bayesian Network is: Pr( Female | Father ) Female Pr( Female |¬ Father ) Father Pr( HasChild | Father ) Pr( HasChild |¬ Father ) HasChild C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Defining a Terminological Na¨ ıve Bayesian Network Background Learning a TBN: Problem Definition Learning a Terminological Na¨ ıve Bayesian Network TBN: the Learning Algorithm Classifying individuals with a TBN The Ignorance Models Conclusions and Future Works Learning a TBN: Problem Definition Given: a target concept C a DL KB K = �T , A� the sets of of positive, negative and neutral examples for C , denoted with Ind + C ( A ), Ind − C ( A ) and Ind 0 C ( A ), so that: ∀ a ∈ Ind + C ( A ) : K | = C ( a ), ∀ a ∈ Ind − C ( A ) : K | = ¬ C ( a ), ∀ a ∈ Ind 0 C ( A ) : K �| = C ( a ) ∧ K �| = ¬ C ( a ); an ignorance model a scoring function score for a TBN N K w.r.t. Ind C ( A ) Find: a network N ∗ K maximizing the scoring function N ∗ K ← arg max score ( N K , Ind C ( A ))) N K C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Defining a Terminological Na¨ ıve Bayesian Network Background Learning a TBN: Problem Definition Learning a Terminological Na¨ ıve Bayesian Network TBN: the Learning Algorithm Classifying individuals with a TBN The Ignorance Models Conclusions and Future Works TBN: the Learning Algorithm... function learn ( K , Ind C ( A )) { The TBN is initialized as containing only the target concept node } N ∗ K = �G , Θ G � ; G = �V ← { C } , E ← ∅� ; N K ← ∅ ; repeat N K ← N ∗ K ; { A new network is created, having one more node and different parameters than the previous one } Network = � c ′ , N ′ K , s ′ � ← extend ( N K , Ind C ( A )); N ∗ K ← N ′ K ; { Possible stopping conditions: a ) improvements in score below a threshold ; b ) reaching a maximum number of nodes } until stopping criterion on Network ; return N K ; C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers

Introduction & Motivation Defining a Terminological Na¨ ıve Bayesian Network Background Learning a TBN: Problem Definition Learning a Terminological Na¨ ıve Bayesian Network TBN: the Learning Algorithm Classifying individuals with a TBN The Ignorance Models Conclusions and Future Works ...TBN: the Learning Algorithm function extend ( N K , Ind C ( A )) Concept ← Start ; Best ← ∅ ; repeat Concepts ← ∅ ; for c ′ ∈ { c ′ ∈ ρ cl ↓ ( Concept ) | | c ′ | ≤ min ( | c | + d , maxLen ) } do V ′ ← V ∪ { c ′ } ; N ′ K ← optimalNetwork ( V ′ , Ind C ( A )); s ′ ← score ( N ′ K , Ind C ( A )); Concepts ← Concepts ∪ {� c ′ , N ′ K , s ′ �} ; end for K , s ′ �∈ Concepts ∪{ Best } s ′ ; Best ← arg max � c ′ , N ′ Concept ← c : � c , N K , s � = Best ; { Possible stopping conditions: a ) exceeding a maximum number of iterations ; b ) exceeding a maximum number of refinement steps } until Stopping criterion on Best ; return Best ; C. d’Amato Learning Terminological Na¨ ıve Bayesian Classifiers