Contents Introduction 1 Graphical Models and the PC Algorithm - PDF document

Contents Introduction 1 Graphical Models and the PC Algorithm Conditional Independence Graphical Models Directed Acyclic Graphs Ewan Donnachie Estimating DAG Structures 2 General Approach 14 July 2006 The PC Algorithm Example Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 1 / 34 Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 2 / 34 The Problem with Causality Outline Introduction 1 “Causality is the centerpiece of the universe” 1 Conditional Independence Graphical Models “The central aim of many studies . . . is the elucidation of Directed Acyclic Graphs cause-effect relationships between variables or events” 2 Criticism of statistical science: focus on probabilistic and statistical 2 Estimating DAG Structures inference at the expense of causational enquiry General Approach The PC Algorithm Example 1 Causality - Wikipedia, the free encyclopedia 2 Preface to Pearl (2000) Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 3 / 34 Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 4 / 34 Conditional Independence Yule-Simpson Paradox Let n ij , N ij , i ∈ { 1 , 2 } and j ∈ { A , B } , be integers. Then it is possible that: Definition (Conditional Independence) n 1 A < n 1 B The random variables X and Y are said to be conditionally N 1 A N 1 B independent given the value of a third random variable Z , if and f ( X | Y , Z ) = f ( X | Z ) . n 2 A < n 2 B N 2 A N 2 B Write: X � Y | Z but n 1 A + n 2 A > n 1 B + n 2 B Intuitively, if Z is known, Y adds no information about the value of N 1 A + N 2 A N 1 B N 2 B X . Applying this to the calculation of conditional probabilities leads to the The difference between independence and conditional Yule-Simpson paradox, credited to George Udny Yule (1903) and independence is demonstrated by the Yule-Simpson Paradox. popularised by E.H. Simpson (1951). Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 5 / 34 Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 6 / 34 Example: The Berkeley sex-bias case Outline The University of California, Berkeley, were sued for bias against Introduction 1 women applying to grad school: Conditional Independence In the university as a whole, men were more likely to be admitted Graphical Models to a course than women Directed Acyclic Graphs Examining individual departments (conditioning on the departments), there was no significant bias against women—in 2 Estimating DAG Structures fact, most departments showed a slight bias against men General Approach Explanation: The PC Algorithm ◮ women tended to apply for courses with low admission rates ◮ men tended to apply for courses with high admission rates Example Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 7 / 34 Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 8 / 34

Graphical Models Why Graphical Models? Nodes: The vertices ( i ∈ V ) of the graph (Nodes and vertices used interchangeably) The role of graphs in probabilistic and statistical modeling is threefold: Edges: Connections ( ( i , j ) ∈ E ) between vertices to provide convenient means of expressing substantive 1 Path: A route along (directed) edges from one node to another assumptions; (e.g. i → j → k → l ) to facilitate economical representation of joint probability 2 functions; and Definition (Graphical Model) to facilitate efficient inferences from observations. 3 A graphical model G is a system of nodes and connecting edges: G = ( V , E ) Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 9 / 34 Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 10 / 34 Conditional Independence Graph The Pairwise Markov Property Definition (Conditional Independence Graph) A graph has the pairwise Markov property if, for all non-adjacent (not directly connected) vertices i and j , The conditional independence graph of X is the undirected graph G = ( V , E ) where V = { 1 , 2 , . . . v } and ( i , j ) is not in the edge set E iff X i � X j | X V � { i , j } X i � X j | X V � { i , j } . Undirected conditional independence graphs are formed using More informally: this definition Start with the complete graph, where each node is connected to all other nodes Therefore, if X i and X j are non-adjacent vertices: Remove the edge between X i and X j if they are independent conditional on the remaining nodes X j is irrelevant for the prediction of X i , and vice-versa X i � X j | rest Separation Theorem : X i � X j | rest ⇒ X i � X j | X a , where X a are N.B.: The conditional dependencies do not represent causal or the vertices separating X i and X j . directed relationships between variables. Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 11 / 34 Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 12 / 34 The Local Markov Property The Global Markov Property A graph has the local Markov property if, for every vertix i , with boundary a = bd ( i ) and b the set of remaining verties, Let a , b and c be disjoint subsets of V . Then, a graph has the global Markov property if, whenever b and c are separated by a in the graph, X i � X b | X a then: X b � X c | X a More informally, if: X i � rest | boundary Global in the sense that the subsets are potentially arbitrary Closely related to prediction—conditioned only on adjacent variables Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 13 / 34 Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 14 / 34 Equivalence of Markov Properties Outline Introduction 1 The three Markov properties: pairwise Markov, local Markov and Conditional Independence global Markov, are equivalent. Graphical Models Directed Acyclic Graphs As the boundary set is always a separating set, global Markov = ⇒ local Markov Local Markov = ⇒ pairwise Markov 2 Estimating DAG Structures By separation theorem, pairwise Markov = ⇒ global Markov General Approach The PC Algorithm Example Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 15 / 34 Ewan Donnachie () Graphical Models and the PC Algorithm 14 July 2006 16 / 34