belief propagation for spatial network embeddings
play

Belief Propagation for Spatial Network Embeddings Andrew Frank - PowerPoint PPT Presentation

May 16, 2023 •232 likes •1.04k views

Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth Department of Computer Science UC Irvine August 25, 2009 Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem

  1. Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth Department of Computer Science UC Irvine August 25, 2009

  2. Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem Description Model Formulation Experimental Results Latent Space Embeddings of Social Networks 3 Problem Description Model Formulation Preliminary Results

  3. Graphical Models Markov Random Fields Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem Description Model Formulation Experimental Results Latent Space Embeddings of Social Networks 3 Problem Description Model Formulation Preliminary Results

  4. Graphical Models Markov Random Fields What Are Graphical Models? Concise representations of probabilistic models Several types: Bayesian networks (DAGs) Markov random fields (undirected graphs) Factor graphs (bipartite graphs) . . . and others!

  5. Graphical Models Markov Random Fields What Are Graphical Models? Concise representations of probabilistic models Several types: Bayesian networks (DAGs) Markov random fields (undirected graphs) Factor graphs (bipartite graphs) . . . and others! A Nodes = random variables B C Edges = dependencies E D between variables

  6. Graphical Models Markov Random Fields What Are Graphical Models? Concise representations of probabilistic models Several types: Bayesian networks (DAGs) Markov random fields (undirected graphs) Factor graphs (bipartite graphs) . . . and others! A suspects Nodes = random variables B C Edges = dependencies E D between variables {innocent,guilty}

  7. Graphical Models Markov Random Fields What Are Graphical Models? Concise representations of probabilistic models Several types: Bayesian networks (DAGs) Markov random fields (undirected graphs) Factor graphs (bipartite graphs) . . . and others! friends A suspects Nodes = random variables B C Edges = dependencies E D between variables {innocent,guilty}

  8. Graphical Models Markov Random Fields Representing Conditional Independencies Interpreting a Markov Random Field If all paths from X to Y pass through Z, then we can say X and Y are conditionally independent given Z. Graphically, with a Textually, through enumeration: Markov Random Field (MRF): A ⊥ D , E | C A B ⊥ C , D , E | A C ⊥ B | A B C D ⊥ A , B , E | C E D E ⊥ A , B , D | C . . .

  9. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D )

  10. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D )

  11. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A )

  12. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A )

  13. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A )

  14. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A )

  15. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A )

  16. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A )

  17. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A ) p ( D | C )

  18. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A ) p ( D | C )

  19. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A ) p ( D | C ) p ( E | C )

  20. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A ) p ( D | C ) p ( E | C ) Largest factor involves 2 variables!

  21. Graphical Models Markov Random Fields Hammersley-Clifford Theorem General factorization property of all MRFs: Hammersley-Clifford Theorem Every MRF factors as the product of potential functions defined over cliques of the graph. Potential functions are. . . Strictly positive Unnormalized

  22. Graphical Models Markov Random Fields Hammersley-Clifford Theorem General factorization property of all MRFs: Hammersley-Clifford Theorem Every MRF factors as the product of potential functions defined over cliques of the graph. A Potential functions are. . . B C Strictly positive E D Unnormalized p ( · ) ∝ f A ( A ) f B ( B ) f C ( C ) f D ( D ) f E ( E ) f AB ( A , B ) f AC ( A , C ) f CD ( C , D ) f CE ( C , E )

  23. Graphical Models Markov Random Fields Specifying a Markov Random Field Model Define the potential functions, e.g.: Let our domain be 0=innocent, 1=guilty. � . 4 B = 0 f B ( B ) = . 6 B = 1 � A = B 2 f AB ( A , B ) = 1 A � = B

  24. Graphical Models Markov Random Fields Specifying a Markov Random Field Model Define the potential functions, e.g.: Let our domain be 0=innocent, 1=guilty. � . 4 B = 0 Suspect B is acting suspicious. f B ( B ) = . 6 B = 1 � A = B 2 f AB ( A , B ) = 1 A � = B

  25. Graphical Models Markov Random Fields Specifying a Markov Random Field Model Define the potential functions, e.g.: Let our domain be 0=innocent, 1=guilty. � . 4 B = 0 Suspect B is acting suspicious. f B ( B ) = . 6 B = 1 � A = B 2 Suspects A and B are friends. f AB ( A , B ) = 1 A � = B

  26. Graphical Models Inference Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem Description Model Formulation Experimental Results Latent Space Embeddings of Social Networks 3 Problem Description Model Formulation Preliminary Results

  27. Graphical Models Inference Marginalization with MRFs Query p(A): � O ( d n ) p ( A ) = p ( A , B , C , D , E ) B , C , D , E

  28. Graphical Models Inference Marginalization with MRFs Query p(A): � O ( d n ) p ( A ) = p ( A , B , C , D , E ) B , C , D , E A Use graph structure to compute p(A) B C in O ( dn 2 ) . E D

  29. Graphical Models Inference Belief Propagation (Sum-Product Algorithm) View marginalization as a “message-passing” algorithm Variables are computational nodes. Intermediate results are “messages” between nodes. A B C E D � f ( A ) f ( B ) f ( C ) f ( D ) f ( E ) f ( A , B ) f ( A , C ) f ( C , D ) f ( C , E ) B , C , D , E

  30. Graphical Models Inference Belief Propagation (Sum-Product Algorithm) View marginalization as a “message-passing” algorithm Variables are computational nodes. Intermediate results are “messages” between nodes. A B C E D � f ( A ) f ( B ) f ( C ) f ( D ) f ( E ) f ( A , B ) f ( A , C ) f ( C , D ) f ( C , E ) B , C , D , E

Recommend

Shuffled Belief Propagation Decoding Juntan Zhang and Marc Fossorier Department of Electrical

Shuffled Belief Propagation Decoding Juntan Zhang and Marc Fossorier Department of Electrical

Shuffled Belief Propagation Decoding Juntan Zhang and Marc Fossorier Department of Electrical Engineering University of Hawaii at Manoa Honolulu, HI 96816 Outline Review of LDPC Codes Standard Belief Propagation Algorithm Shuffled

467 views • 31 slides
LEARNING AFFINITY VIA SPATIAL PROPAGATION NETWORK Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu

LEARNING AFFINITY VIA SPATIAL PROPAGATION NETWORK Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu

LEARNING AFFINITY VIA SPATIAL PROPAGATION NETWORK Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu Zhong, Ming-Hsuan Yang, Jan Kautz NVIDIA Research, March 27, 2018 WHAT IS AFFINITY? The relations between two pixels/regions A typical

945 views • 40 slides
An empirical study of Gaussian belief propagation and application in the detection of F-formations

An empirical study of Gaussian belief propagation and application in the detection of F-formations

Regularized Gaussian belief propagation Empirical Study F-formation detection Concluding Remarks An empirical study of Gaussian belief propagation and application in the detection of F-formations Francois Kamper Stellenbosch University

578 views • 20 slides
Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani

Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani

Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn 2016 Recall: Belief

1.07k views • 61 slides
Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani

Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani

Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn 2014 Recall: Belief

674 views • 50 slides
Approximation-aware Dependency Parsing by Belief Propagation

Approximation-aware Dependency Parsing by Belief Propagation

Approximation-aware Dependency Parsing by Belief Propagation Matt Gormley Mark Dredze Jason Eisner September 19, 2015 TACL at EMNLP 1

833 views • 44 slides
From Elimination to Belief Propagation Recall that Induced dependency during marginalization

From Elimination to Belief Propagation Recall that Induced dependency during marginalization

School of Computer Science The Belief Propagation (Sum-Product) Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 5, Sep 31, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B

455 views • 17 slides
Belief network inference Four main approaches to determine posterior distributions in belief

Belief network inference Four main approaches to determine posterior distributions in belief

Belief network inference Four main approaches to determine posterior distributions in belief networks: Variable Elimination: exploit the structure of the network to eliminate (sum out) the non-observed, non-query variables one at a time.

330 views • 29 slides
Belief Propagation Matt Gormley Lecture 9 Sep. 25, 2019 1 Q&A Q: What if I already

Belief Propagation Matt Gormley Lecture 9 Sep. 25, 2019 1 Q&A Q: What if I already

10-418 / 10-618 Machine Learning for Structured Data Machine Learning Department School of Computer Science Carnegie Mellon University Belief Propagation Matt Gormley Lecture 9 Sep. 25, 2019 1 Q&A Q: What if I already answered a

1.18k views • 72 slides
Probabilistic Graphical Models Lecture 13 Loopy Belief Propagation CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 13 Loopy Belief Propagation CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 13 Loopy Belief Propagation CS/CNS/EE 155 Andreas Krause Announcements Homework 3 out Lighter problem set to allow more time for project Next Monday: Guest lecture by Dr. Baback Moghaddam from the

1.35k views • 40 slides
Belief Networks Some Belief Network references E. Charniak Bayesian Networks without

Belief Networks Some Belief Network references E. Charniak Bayesian Networks without

Belief Networks Some Belief Network references E. Charniak Bayesian Networks without Tears, AI Magazine Winter 1991, pp 50-63 Chris Williams, School of Informatics University of Edinburgh D. Heckerman, A Tutorial on Learning

486 views • 6 slides
Problem Session 2 Problem 1: from belief propagation to Bayes AMP state evolution Below I have

Problem Session 2 Problem 1: from belief propagation to Bayes AMP state evolution Below I have

OOPS 2020 Mean field methods in high-dimensional statistics and nonconvex optimization Lecturer: Andrea Montanari Problem session leader: Michael Celentano July 6, 2020 Problem Session 2 Problem 1: from belief propagation to Bayes AMP state

321 views • 4 slides
19 : Baham Sabbir Scribes : , Belief Propagation : Problem The With Loops R far Graph

19 : Baham Sabbir Scribes : , Belief Propagation : Problem The With Loops R far Graph

The Algorithm Lecture Junction Tree 19 : Baham Sabbir Scribes : , Belief Propagation : Problem The With Loops R far Graph Directed b ; a , b. d pca ) c clb d ) p ( b ( Pla ) dla l a) p( ) = p , A C p fz fu a. Fi

480 views • 25 slides
Parallel Splash Belief Propagation Joseph E. Gonzalez Yucheng Low Carlos Guestrin David

Parallel Splash Belief Propagation Joseph E. Gonzalez Yucheng Low Carlos Guestrin David

Parallel Splash Belief Propagation Joseph E. Gonzalez Yucheng Low Carlos Guestrin David OHallaron Computers which worked on this project: BigBro1, BigBro2, BigBro3, BigBro4, BigBro5, BigBro6, BiggerBro, BigBroFS Tashish01, Tashi02,

817 views • 64 slides
A Multi-Round Side Channel Attack on AES using Belief Propagation Hlne Le Bouder 1 Ronan

A Multi-Round Side Channel Attack on AES using Belief Propagation Hlne Le Bouder 1 Ronan

A Multi-Round Side Channel Attack on AES using Belief Propagation Hlne Le Bouder 1 Ronan Lashermes 1 Yanis Linge 2 Gal Thomas 3 Jean Yves Zie 1 INRIA Rennes, LHS/PEC 2 STMicroelectronics 3 Orange Labs Issy Les Moulineaux October 25th, 2016

315 views • 17 slides
Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1

Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1

Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1 Hilbert J.Kappen 1 M. Chertkov 2 1 Department of Biophysics Radboud University, Nijmegen, The Netherlands 2 Theoretical Division and Center of

711 views • 28 slides
Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical

Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical

Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical inference is the determination of an underlying probability distribution from observed data. Probabilistic Graphical Models 1 1 =

522 views • 17 slides
Fast Methods and Nonparametric Belief Propagation Alexander Ihler Massachusetts Institute of

Fast Methods and Nonparametric Belief Propagation Alexander Ihler Massachusetts Institute of

Fast Methods and Nonparametric Belief Propagation Alexander Ihler Massachusetts Institute of Technology ihler@mit.edu Joint work with Erik Sudderth William Freeman Alan Willsky Introduction Nonparametric BP Perform inference on

632 views • 50 slides
Non-Binary Belief Propagation in Large-Dimension Communication Receivers A. Chockalingam

Non-Binary Belief Propagation in Large-Dimension Communication Receivers A. Chockalingam

Non-Binary Belief Propagation in Large-Dimension Communication Receivers A. Chockalingam Department of Electrical Communication Engineering Indian Institute of Science Bangalore CEFIPRA Workshop 2014, IISc, Bangalore 14 January 2014 Outline

426 views • 30 slides
Pruning Neural Belief Propagation Decoders Andreas Buchberger, 1 Christian H ager, 1 Henry D.

Pruning Neural Belief Propagation Decoders Andreas Buchberger, 1 Christian H ager, 1 Henry D.

Pruning Neural Belief Propagation Decoders Andreas Buchberger, 1 Christian H ager, 1 Henry D. Pfister, 2 Laurent Schmalen 3 and, Alexandre Graell i Amat, 1 1 Chalmers University of Technology, G oteborg, Sweden 2 Duke University, Durham, North

427 views • 23 slides
On the Maximal Number of Real Embeddings of Spatial Minimally Rigid Graphs Vangelis Bartzos,

On the Maximal Number of Real Embeddings of Spatial Minimally Rigid Graphs Vangelis Bartzos,

On the Maximal Number of Real Embeddings of Spatial Minimally Rigid Graphs Vangelis Bartzos, Ioannis Z. Emiris, Jan Legersk, Elias Tsigaridas July 2018 The International Symposium on Symbolic and Algebraic Computation, New York A R C A D E S

537 views • 32 slides
Graphical Models Graphical Models Clique trees & Belief Propagation Siamak Ravanbakhsh Fall

Graphical Models Graphical Models Clique trees & Belief Propagation Siamak Ravanbakhsh Fall

Graphical Models Graphical Models Clique trees & Belief Propagation Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives message passing on clique trees its relation to variable elimination two different forms of belief

649 views • 38 slides
Simplicity and Complexity of Belief-Propagation Elchanan Mossel 1 1 MIT Dec 2020 Elchanan Mossel

Simplicity and Complexity of Belief-Propagation Elchanan Mossel 1 1 MIT Dec 2020 Elchanan Mossel

Simplicity and Complexity of Belief-Propagation Elchanan Mossel 1 1 MIT Dec 2020 Elchanan Mossel Simplicity & Complexity of BP Markov Random Fields and Information Flow on Trees Consider the following process on a tree. Color the root

454 views • 10 slides
Belief Propagation Probabilistic Graphical Models Sharif University of Technology Spring 2017

Belief Propagation Probabilistic Graphical Models Sharif University of Technology Spring 2017

Sum-Product: Message Passing Belief Propagation Probabilistic Graphical Models Sharif University of Technology Spring 2017 Soleymani All single-node marginals If we need the full set of marginals, repeating elimination algorithm for each

517 views • 24 slides

More recommend