markov logic networks
play

Markov Logic Networks Matt Richardson and Pedro Domingos (2006), - PDF document

Oct 22, 2023 •180 likes •344 views

Markov Logic Networks Matt Richardson and Pedro Domingos (2006), Markov Logic Networks, Machine Learning , 62, 107-136, 2006. CS 786 University of Waterloo Lecture 23: July 19, 2012 Outline Markov Logic Networks Alchemy 2 CS786

  1. Markov Logic Networks Matt Richardson and Pedro Domingos (2006), Markov Logic Networks, Machine Learning , 62, 107-136, 2006. CS 786 University of Waterloo Lecture 23: July 19, 2012 Outline • Markov Logic Networks • Alchemy 2 CS786 Lecture Slides (c) 2012 P. Poupart 1

  2. Markov Logic Networks • Bayesian networks and Markov networks: – Model uncertainty – But propositional representation (e.g., we need one variable per object in the world) • First-order logic: – First-order representation (e.g., quantifiers allow us to reason about several objects simultaneously) – But we can’t deal with uncertainty • Markov logic networks: combine Markov networks and first-order logic 3 CS786 Lecture Slides (c) 2012 P. Poupart Markov Logic • A logical KB is a set of hard constraints on the set of possible worlds • Let’s make them soft constraints : when a world violates a formula, it becomes less probable, not impossible • Give each formula a weight : (higher weight  stronger constraint) P(world)  e  weights of formulas it satisfies 4 CS786 Lecture Slides (c) 2012 P. Poupart 2

  3. Markov Logic: Definition • A Markov Logic Network (MLN) is a set of pairs (F, w) where – F is a formula in first-order logic – w is a real number • Together with a set of constants, it defines a Markov network with – One node for each grounding of each predicate in the MLN – One feature for each grounding of each formula F in the MLN, with the corresponding weight w 5 CS786 Lecture Slides (c) 2012 P. Poupart Example: Friends & Smokers Smoking causes cancer. Friends have similar smoking habits. 6 CS786 Lecture Slides (c) 2012 P. Poupart 3

  4. Example: Friends & Smokers   x Smokes ( x ) Cancer ( x )      x , y Friends ( x , y ) Smokes ( x ) Smokes ( y ) 7 CS786 Lecture Slides (c) 2012 P. Poupart Example: Friends & Smokers   1 . 5 x Smokes ( x ) Cancer ( x )      1 . 1 x , y Friends ( x , y ) Smokes ( x ) Smokes ( y ) 8 CS786 Lecture Slides (c) 2012 P. Poupart 4

  5. Example: Friends & Smokers   1 . 5 x Smokes ( x ) Cancer ( x )      1 . 1 x , y Friends ( x , y ) Smokes ( x ) Smokes ( y ) Two constants: Anna (A) and Bob (B) 9 CS786 Lecture Slides (c) 2012 P. Poupart Example: Friends & Smokers   1 . 5 x Smokes ( x ) Cancer ( x )      1 . 1 x , y Friends ( x , y ) Smokes ( x ) Smokes ( y ) Two constants: Anna (A) and Bob (B) Smokes(A) Smokes(B) Cancer(A) Cancer(B) 10 CS786 Lecture Slides (c) 2012 P. Poupart 5

  6. Example: Friends & Smokers   1 . 5 x Smokes ( x ) Cancer ( x )      1 . 1 x , y Friends ( x , y ) Smokes ( x ) Smokes ( y ) Two constants: Anna (A) and Bob (B) Friends(A,B) Friends(A,A) Smokes(A) Smokes(B) Friends(B,B) Cancer(A) Cancer(B) Friends(B,A) 11 CS786 Lecture Slides (c) 2012 P. Poupart Example: Friends & Smokers   1 . 5 x Smokes ( x ) Cancer ( x )      1 . 1 x , y Friends ( x , y ) Smokes ( x ) Smokes ( y ) Two constants: Anna (A) and Bob (B) Friends(A,B) Friends(A,A) Smokes(A) Smokes(B) Friends(B,B) Cancer(A) Cancer(B) Friends(B,A) 12 CS786 Lecture Slides (c) 2012 P. Poupart 6

  7. Example: Friends & Smokers   1 . 5 x Smokes ( x ) Cancer ( x )      1 . 1 x , y Friends ( x , y ) Smokes ( x ) Smokes ( y ) Two constants: Anna (A) and Bob (B) Friends(A,B) Friends(A,A) Smokes(A) Smokes(B) Friends(B,B) Cancer(A) Cancer(B) Friends(B,A) 13 CS786 Lecture Slides (c) 2012 P. Poupart Markov Logic Networks • MLN is template for ground Markov nets • Probability of a world x :   1     P ( x ) exp w n ( x ) i i   Z i Weight of formula i No. of true groundings of formula i in x • Typed variables and constants greatly reduce size of ground Markov net 14 CS786 Lecture Slides (c) 2012 P. Poupart 7

  8. Alchemy • Open Source AI package • http://alchemy.cs.washington.edu • Implementation of Markov logic networks • Problem specified in two files: – File1.mln (Markov logic network) – File2.db (database / data set) • Learn weights and structure of MLN • Inference queries 15 CS786 Lecture Slides (c) 2012 P. Poupart Markov Logic Encoding • File.mln • Two parts: – Declaration • Domain of each variable • Predicates – Formula • Pairs of weights with logical formula 16 CS786 Lecture Slides (c) 2012 P. Poupart 8

  9. Markov Logic Encoding • Example declaration – Domain of each variable • person = {Anna, Bob} – Predicates: • Friends(person,person) • Smokes(person) • Cancer(person) • Example formula – 8 Smokes(x) => Cancer(x) – 5 Friends(x,y) => (Smokes(x)<=>Smokes(y)) NB: by default, formulas are universally quantified in Alchemy 17 CS786 Lecture Slides (c) 2012 P. Poupart Dataset • File.db • List of facts (ground atoms) • Example: – Friends(Anna,Bob) – Smokes(Anna) – Cancer(Bob) 18 CS786 Lecture Slides (c) 2012 P. Poupart 9

  10. Syntax • Logical connective: – ! (not), ^ (and), v (or), => (implies), <=> (iff) • Quantifiers: – forall (  ), exist (  ) – By default unquantified variables are universally quantified in Alchemy • Operator precedence: – ! > ^ > v > => > <=> > forall = exist 19 CS786 Lecture Slides (c) 2012 P. Poupart Syntax • Short hand for predicates – ! operator: indicates that the preceding variable has exactly one true grounding – Ex: HasPosition(x,y!) : for each grounding of x , exactly one grounding of y satisfies HasPosition • Short hand for multiple weights – + operator: indicates that a different weight should be learned for each grounding of the following variable – Ex: outcome(throw,+face): a different weight is learned for each grounding of face 20 CS786 Lecture Slides (c) 2012 P. Poupart 10

  11. Multinomial Distribution Example: Throwing dice Types: throw = { 1, … , 20 } face = { 1, … , 6 } Predicate: Outcome(throw,face) Formulas: Outcome(t,f) ^ f!=f’ => !Outcome(t,f’). Exist f Outcome(t,f). Too cumbersome! 21 CS786 Lecture Slides (c) 2012 P. Poupart Multinomial Distrib.: ! Notation Example: Throwing dice Types: throw = { 1, … , 20 } face = { 1, … , 6 } Predicate: Outcome(throw,face!) Formulas: Semantics: Arguments without “!” determine args with “!”. Only one face possible for each throw. 22 CS786 Lecture Slides (c) 2012 P. Poupart 11

  12. Multinomial Distrib.: + Notation Example: Throwing biased dice Types: throw = { 1, … , 20 } face = { 1, … , 6 } Predicate: Outcome(throw,face!) Formulas: Outcome(t,+f) Semantics: Learn weight for each grounding of args with “+”. 23 CS786 Lecture Slides (c) 2012 P. Poupart Text Classification page = { 1, … , n } word = { … } topic = { … } Topic(page,topic!) HasWord(page,word) Links(page,page) HasWord(p,+w) => Topic(p,+t) Topic(p,t) ^ Links(p,p') => Topic(p',t) 24 CS786 Lecture Slides (c) 2012 P. Poupart 12

  13. Information Retrieval InQuery(word) HasWord(page,word) Relevant(page) Links(page,page) InQuery(+w) ^ HasWord(p,+w) => Relevant(p) Relevant(p) ^ Links(p,p’) => Relevant(p’) Cf. L. Page, S. Brin, R. Motwani & T. Winograd, “The PageRank Citation Ranking: Bringing Order to the Web,” Tech. Rept., Stanford University, 1998. 25 CS486/686 Lecture Slides (c) 2012 P. Poupart Record deduplication Problem: Given database, find duplicate records HasToken(token,field,record) SameField(field,record,record) SameRecord(record,record) HasToken(+t,+f,r) ^ HasToken(+t,+f,r’) => SameField(f,r,r’) SameField(+f,r,r’) => SameRecord(r,r’) SameRecord(r,r’) ^ SameRecord(r’,r”) => SameRecord(r,r”) Cf. A. McCallum & B. Wellner, “Conditional Models of Identity Uncertainty with Application to Noun Coreference,” in Adv. NIPS 17 , 2005. 26 CS486/686 Lecture Slides (c) 2012 P. Poupart 13

  14. Record resolution Can also resolve fields: HasToken(token,field,record) SameField(field,record,record) SameRecord(record,record) HasToken(+t,+f,r) ^ HasToken(+t,+f,r’) => SameField(f,r,r’) SameField(+f,r,r’) <=> SameRecord(r,r’) SameRecord(r,r’) ^ SameRecord(r’,r”) => SameRecord(r,r”) SameField(f,r,r’) ^ SameField(f,r’,r”) => SameField(f,r,r”) More: P. Singla & P. Domingos, “Entity Resolution with Markov Logic”, in Proc. ICDM-2006 . 27 CS486/686 Lecture Slides (c) 2012 P. Poupart Information Extraction • Problem: Extract database from text or semi-structured sources • Example: Extract database of publications from citation list(s) (the “CiteSeer problem”) • Two steps: – Segmentation: Use HMM to assign tokens to fields – Record resolution: Use logistic regression and transitivity 28 CS486/686 Lecture Slides (c) 2012 P. Poupart 14

Recommend

Markov Logic Networks Matt Richardson and Pedro Domingos (2006), Markov Logic Networks, Machine

Markov Logic Networks Matt Richardson and Pedro Domingos (2006), Markov Logic Networks, Machine

Markov Logic Networks Matt Richardson and Pedro Domingos (2006), Markov Logic Networks, Machine Learning , 62, 107-136, 2006. CS 486/686 University of Waterloo Lecture 21: Nov 20, 2012 Outline Markov Logic Networks Alchemy 2

618 views • 13 slides
Markov Logic Networks November 18, 2008 CS 486/686 University of Waterloo Outline Markov

Markov Logic Networks November 18, 2008 CS 486/686 University of Waterloo Outline Markov

Markov Logic Networks November 18, 2008 CS 486/686 University of Waterloo Outline Markov Logic Networks Alchemy Readings: Matt Richardson and Pedro Domingos (2006), Markov Logic Networks, Machine Learning , 62, 107-136, 2006.

351 views • 21 slides
Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic

Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic

Putting the Pieces Together Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic Definition A Markov Logic Network (MLN) is a set of A logical KB is a set of hard constraints pairs (F, w) where on

350 views • 6 slides
Markov Logic Networks Andrea Passerini passerini@disi.unitn.it Statistical relational learning

Markov Logic Networks Andrea Passerini passerini@disi.unitn.it Statistical relational learning

Markov Logic Networks Andrea Passerini passerini@disi.unitn.it Statistical relational learning Markov Logic Networks Combining logic with probability Motivation First-order logic is a powerful language to represent complex relational

721 views • 54 slides
Learning and Inference in Markov Logic Networks CS 486/686 University of Waterloo Lecture 23:

Learning and Inference in Markov Logic Networks CS 486/686 University of Waterloo Lecture 23:

Learning and Inference in Markov Logic Networks CS 486/686 University of Waterloo Lecture 23: November 27, 2012 Outline Markov Logic Networks Parameter learning Lifted inference 2 CS486/686 Lecture Slides (c) 2012 P. Poupart 1

223 views • 10 slides
Outline Markov networks (a.k.a. Markov random fields) Markov Networks Reading: Michael

Outline Markov networks (a.k.a. Markov random fields) Markov Networks Reading: Michael

Outline Markov networks (a.k.a. Markov random fields) Markov Networks Reading: Michael Jordan, Graphical Models , Statistical Science (Special November 12, 2009 Issue on Bayesian Statistics), 19, 140- CS 486/686 155, 2004. University

325 views • 4 slides
Learning and Inference in Markov Logic Networks CS 786 University of Waterloo Lecture 24: July

Learning and Inference in Markov Logic Networks CS 786 University of Waterloo Lecture 24: July

Learning and Inference in Markov Logic Networks CS 786 University of Waterloo Lecture 24: July 24, 2012 Outline Markov Logic Networks Parameter learning Lifted inference 2 CS786 Lecture Slides (c) 2012 P. Poupart 1 Parameter

98 views • 6 slides
General Formulations for Structures: Markov Logic CS 6355: Structured Prediction 1 This lecture

General Formulations for Structures: Markov Logic CS 6355: Structured Prediction 1 This lecture

General Formulations for Structures: Markov Logic CS 6355: Structured Prediction 1 This lecture Graphical models Bayesian Networks Markov Random Fields (MRFs) Formulations of structured output Joint models Markov Logic

767 views • 43 slides
Probabilistic Event Calculus based on Markov Logic Networks Anastasios Skarlatidis 1 , 2 ,

Probabilistic Event Calculus based on Markov Logic Networks Anastasios Skarlatidis 1 , 2 ,

Probabilistic Event Calculus based on Markov Logic Networks Anastasios Skarlatidis 1 , 2 , Georgios Paliouras 1 , George Vouros 2 and Alexander Artikis 1 1 Institute of Informatics and Telecommunications NCSR Demokritos, Athens 15310, Greece

504 views • 29 slides
Markov Networks March 2, 2010 CS 886 University of Waterloo Outline Markov networks

Markov Networks March 2, 2010 CS 886 University of Waterloo Outline Markov networks

Markov Networks March 2, 2010 CS 886 University of Waterloo Outline Markov networks (a.k.a. Markov random fields) Reading: Michael Jordan, Graphical Models , Statistical Science (Special Issue on Bayesian Statistics), 19, 140- 155,

460 views • 21 slides
An introduction to Markov logic networks and their use in visual relational learning Willie Brink

An introduction to Markov logic networks and their use in visual relational learning Willie Brink

An introduction to Markov logic networks and their use in visual relational learning Willie Brink Applied Mathematics, Stellenbosch University wbrink@sun.ac.za Thanks to Luc De Raedt and the DTAI research group at KU Leuven 1/20 Elephants are

217 views • 20 slides
Efficient Weight Learning for Markov Logic Networks Speaker Manuel Noll Advisor Maximilian

Efficient Weight Learning for Markov Logic Networks Speaker Manuel Noll Advisor Maximilian

Efficient Weight Learning for Markov Logic Networks Speaker Manuel Noll Advisor Maximilian Dylla What is this talk about? Well, of course about weight learning. A weight represents the relative strenght or importance of that rule,

669 views • 24 slides
Markov Networks Alan Ri2er Markov Networks Undirected graphical models

Markov Networks Alan Ri2er Markov Networks Undirected graphical models

Markov Networks Alan Ri2er Markov Networks Undirected graphical models Smoking Cancer Asthma Cough l Poten;al func;ons defined over cliques Smoking Cancer (S,C) 1 P ( x

476 views • 22 slides
Markov Networks [ Michael Jordan, Graphical Models, Statistical Science (Special Issue on

Markov Networks [ Michael Jordan, Graphical Models, Statistical Science (Special Issue on

Markov Networks [ Michael Jordan, Graphical Models, Statistical Science (Special Issue on Bayesian Statistics), 19, 140-155, 2004.] CS 486/686 University of Waterloo Lecture 18: Nov 8, 2012 Outline Markov networks (a.k.a. Markov random

469 views • 11 slides
Integrating Logical Representations with Probabilistic Information using Markov Logic Dan

Integrating Logical Representations with Probabilistic Information using Markov Logic Dan

Integrating Logical Representations with Probabilistic Information using Markov Logic Dan Garrette, Katrin Erk, and Raymond Mooney The University of Texas at Austin 1 Overview Some phenomena best modeled through logic , others statistically

711 views • 51 slides
Markov Networks [KF] Chapter 4 CS 786 University of Waterloo Lecture 7: May 24, 2012 Outline

Markov Networks [KF] Chapter 4 CS 786 University of Waterloo Lecture 7: May 24, 2012 Outline

Markov Networks [KF] Chapter 4 CS 786 University of Waterloo Lecture 7: May 24, 2012 Outline Markov networks a.k.a. Markov random fields Conditional random fields 2 CS786 Lecture Slides (c) 2012 P. Poupart 1 Recall Bayesian

432 views • 15 slides
Modeling Data Correlations in Private Data Mining with Markov Model and Markov Networks Yang Cao

Modeling Data Correlations in Private Data Mining with Markov Model and Markov Networks Yang Cao

Modeling Data Correlations in Private Data Mining with Markov Model and Markov Networks Yang Cao Emory University 2017.11.15 Outline Data Mining with Di ff erential Privacy (DP) Scenario: Spatiotemporal Data Mining using DP Markov

801 views • 37 slides
Markov Models Yanbing Xue Outline Introduction Markov chains Dynamic belief networks

Markov Models Yanbing Xue Outline Introduction Markov chains Dynamic belief networks

2020 2 25 Markov Models Yanbing Xue Outline Introduction Markov chains Dynamic belief networks Hidden Markov models (HMMs) 1 2020 2 25 Outline Introduction Time series Probabilistic graphical

467 views • 29 slides
Lecture 6: Examples of Bayesian Networks and Markov Networks Zhenke Wu Department of

Lecture 6: Examples of Bayesian Networks and Markov Networks Zhenke Wu Department of

Lecture 6: Examples of Bayesian Networks and Markov Networks Zhenke Wu Department of Biostatistics, University of Michigan September 22, 2016 Lecture 5 Main Points Once Again Bayesian network ( ) , P - Directed acyclic graph (DAG):

476 views • 14 slides
Scalable Uncertainty Management 06 Markov Logic Rainer Gemulla July 13, 2012 Overview In

Scalable Uncertainty Management 06 Markov Logic Rainer Gemulla July 13, 2012 Overview In

Scalable Uncertainty Management 06 Markov Logic Rainer Gemulla July 13, 2012 Overview In this lecture Statistical relational learning (SRL) Introduction to probabilistic graphical models (PGM) Basics of undirected models (called Markov

1.14k views • 78 slides
Checkpointing for the RESTART Problem in Markov Networks Lester Lipsky Derek Doran Swapna

Checkpointing for the RESTART Problem in Markov Networks Lester Lipsky Derek Doran Swapna

Overview of ME distributions 2 Failure Recover Scenarios 7 A Taboo Process - Two Absorbing States 14 RESTART and Checkpoints for Markov Models 18 Example 31 Checkpointing for the RESTART Problem in Markov Networks Lester Lipsky Derek

773 views • 43 slides
Bayesian networks Independence Bayesian networks Markov conditions Inference by

Bayesian networks Independence Bayesian networks Markov conditions Inference by

Bayesian networks Independence Bayesian networks Markov conditions Inference by enumeration rejection sampling Gibbs sampler Independence if P(A=a,B=a) = P(A=a)P(B=b) for all a and b, then we call A and B

733 views • 19 slides
Outline Introduction Delay in a Logic Gate Multistage Logic Networks Choosing the

Outline Introduction Delay in a Logic Gate Multistage Logic Networks Choosing the

Introduction to CMOS VLSI Design Lecture 5: Logical Effort David Harris Harvey Mudd College Spring 2004 Outline Introduction Delay in a Logic Gate Multistage Logic Networks Choosing the Best Number of Stages Example

719 views • 24 slides
Markov Networks Asthma Cough Potential functions defined over cliques Smoking Cancer

Markov Networks Asthma Cough Potential functions defined over cliques Smoking Cancer

Markov Networks Undirected graphical models Smoking Cancer Markov Networks Asthma Cough Potential functions defined over cliques Smoking Cancer (S,C) 1 P ( x ) c x ( ) = c False False 4.5 Z c False True 4.5

62 views • 3 slides

More recommend