Course Overview and Introduction Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Some slides have been adopted from Eric Zing, CMU
Course info Instructor: Mahdieh Soleymani Email: soleymani@sharif.edu Teacher assistants: Fatemeh Seyyedsalehi Ehsan Montahaei Amirshayan Haghipour Seyed Mohammad Chavosian Sajad Behfar 2
Text book D. Koller and N. Friedman, “ Probabilistic Graphical Models: Principles and Techniques ” , MIT Press, 2009. M.I. Jordan, “ An Introduction to Probabilistic Graphical Models ” , Preprint . Other C.M. Bishop, “ Pattern Recognition and Machine Learning ” , Springer, 2006. Chapters 8-11,13. K.P. Murphy, “ Machine Learning:A Probabilistic Perspective ” , MIT Press, 2012. 3
Evaluation policy Mid-term: 20% Final: 30% Mini-exams: 10% Home works & project: 40% 4
Why using probabilistic models? Partial knowledge of the state of the world Noisy or incomplete observations We may not know or cover all the involved phenomena in our model Partial knowledge can cause the world seems to be stochastic To deal with partial knowledge and/or stochastic worlds we need reasoning under uncertainty 5
Reasoning under uncertainty 6
Representation, inference, and learning We will cover three aspects of the probabilistic models: Representation of probabilistic knowledge Inference algorithms on these models Learning: Using the data to acquire the distribution 7
Representation, inference, and learning Representation: When variables tends to interact directly with few other variables (local structure) Inference: answering queries using the model algorithms for answering questions/queries according to the model and/or based given observation. Learning of both the parameters and the structure of the graphical models 8
Main problems Representation : what is the joint probability dist. on multiple variables? 𝑄(𝑌 1 , … , 𝑌 𝑒 ) How many state configurations in total? Are they all needed to be represented? Do we get any scientific/medical insight? Inference : If not all variables are observable, how to compute the conditional distribution of latent variables given evidence? Learning : where do we get all this probabilities? Maximal-likelihood estimation? but how many data do we need? Are there other est. principles? Where do we put domain knowledge in terms of plausible relationships between variables, and plausible values of the probabilities? 9
Probability review Marginal probabilities 𝑄 𝑌 = 𝑧 𝑄(𝑌, 𝑍 = 𝑧) Conditional probabilities 𝑄(𝑌,𝑍) 𝑄 𝑌|𝑍 = 𝑄(𝑍) Bayes rule: 𝑄 𝑍|𝑌 𝑄(𝑌) 𝑄 𝑌|𝑍 = 𝑄(𝑍) Chain rule: 𝑜 𝑄 𝑌 1 , … , 𝑌 𝑜 = 𝑗=1 𝑄(𝑌 𝑗 |𝑌 1 , … , 𝑌 𝑗−1 ) 10
Medical diagnosis example Representation 𝑒 1 𝑒 2 𝑒 3 𝑒 4 diseases Findings 𝑔 𝑔 𝑔 𝑔 𝑔 5 1 2 3 4 (symptoms & tests) 𝑄(𝑔 1 |𝑒 1 ) 𝑄(𝑔 2 |𝑒 1 , 𝑒 2 , 𝑒 3 ) 𝑄(𝑔 3 |𝑒 3 ) … 11
Example Graphical Model 12
Representation: summary of advantages Representing large multivariate distributions directly and exhaustively is hopeless: The number of parameters is exponential in the number of random variables Inference can be exponential in the number of variables PGM representation Compact representation of the joint distribution Transparent We can combine expert knowledge and accumulated data to learn the model Effective for inference and learning 13
Why using a graph for representation? Intuitively appealing interface by which we can models highly interacting sets of variables It allows us to design efficient general purpose inference algorithms 14
Graph structure Denotes conditional dependence structure between random variables One view: Graph represents a set of independencies Another view: Graph shows a skeleton for factorizing a joint distribution 15
PGMs as a framework General-purpose framework for representing uncertain knowledge and learning and inference in uncertain conditions . A graph-based representation as the basis of encoding a complex distribution compactly allows declarative representation (with clear semantics) of the probabilistic knowledge 16
PGMs as a framework Intuitive & compact data structure for representation Efficient reasoning using general-purpose algorithms Sparse parameterization (enables us to elicit or learn from data) 17
PGM: declarative representation Separation of knowledge and reasoning We need to specify our model for a specific application that represents our probabilistic knowledge There is a general suite of reasoning algorithms that can be used. 18
Data Integration Due to the local structure of the graph, we can handle different source of information Modular combination of heterogeneous parts – data fusion Combining different data modality Example:T ext + Image + Network => Holistic Social Media 19
History Wright 1921, 1934 and before Bayesian networks are independently developed by Spiegelhalter and Lauritzen in statistics and Pearl in computer science in the late 1980 ’ s First applications (1990 ’ s): expert systems and information retrieval 20
PGMs: some application areas Machine Learning and computational statistics Computer vision: e.g., segmenting and denoising images Robotics: e.g., robot localization and mapping Natural Language Processing Speech recognition Information Retrieval AI: game playing, planning Computational Biology Networks: decoding messages (sent over a noisy channel) Medical diagnosis and prognosis … 21
Graphical models: directed & undirected Two kinds of graphical models: Directed: Bayesian Networks (BNs) Undirected: Markov Random Fields (MRFs) B A A B C C D D Causality relations Correlation of variables 22
Graphical models: directed & undirected [Pathfinder Project, 1992] 23
Medical diagnosis example Representation 𝑒 1 𝑒 2 𝑒 3 𝑒 4 diseases Findings 𝑔 𝑔 𝑔 𝑔 𝑔 5 1 2 3 4 (symptoms & tests) Inference: Given symptoms, what disease is likely? Eliciting or learning the required probabilities from the data 24
Image denoising example 25 [Bishop]
Genetic pedigree example A0 B0 A B Bg Ag A1 B1 D C C0 D0 Cg Dg C1 D1 E E0 Eg E1 26
Plan in our course Fundamentals of Graphical Models: Representation Bayesian Network Markov Random Fields Exact inference Basics of learning Case studies: Popular graphical models Multivariate Gaussian Models FA, PPCA HMM, CRF, Kalman filter Approximate inference Variational methods Monte Carlo algorithms 27
Recommend
More recommend
