graphical models
play

Graphical Models 10-715 Fall 2015 Alexander Smola alex@smola.org - PowerPoint PPT Presentation

Aug 31, 2023 •203 likes •1.98k views

Graphical Models 10-715 Fall 2015 Alexander Smola alex@smola.org Office hours - after class in my office Marianas Labs Directed Graphical Models Brain & Brawn p (brain) = 0 . 1 p (sports) = 0 . 2 smart strong 0 1 0 0.1 0.8 1

  1. Example - PCA/ICA Latent Factors Observed Effects • Observed effects 
 Click behavior, queries, watched news, emails d ! d X Y y i v i , σ 2 1 x ∼ N and p ( y ) = p ( y i ) i =1 i =1

  2. Example - PCA/ICA Latent Factors Observed Effects • Observed effects 
 Click behavior, queries, watched news, emails d ! d X Y y i v i , σ 2 1 x ∼ N and p ( y ) = p ( y i ) i =1 i =1

  3. Example - PCA/ICA Latent Factors Observed Effects • Observed effects 
 Click behavior, queries, watched news, emails d ! d X Y y i v i , σ 2 1 x ∼ N and p ( y ) = p ( y i ) i =1 i =1

  4. Example - PCA/ICA Latent Factors Observed Effects • Observed effects 
 Click behavior, queries, watched news, emails d ! d X Y y i v i , σ 2 1 x ∼ N and p ( y ) = p ( y i ) i =1 i =1 • p(y) is Gaussian for PCA. General for ICA

  5. Cocktail party problem

  6. Recommender Systems u m r

  7. Recommender Systems u m r • Users u • Movies m • Ratings r (but only for a subset of users)

  8. Recommender Systems u m r ... intersecting plates ... (like nested FOR loops) • Users u • Movies m • Ratings r (but only for a subset of users)

  9. Recommender Systems news, SearchMonkey u m answers social ranking r ... intersecting plates ... OMG personals (like nested FOR loops) • Users u • Movies m • Ratings r (but only for a subset of users)

  10. Challenges engineering machine learning

  11. Challenges • How to design models engineering • Common (engineering) sense • Computational tractability machine learning

  12. Challenges • How to design models engineering • Common (engineering) sense • Computational tractability • Dependency analysis machine learning

  13. Challenges • How to design models engineering • Common (engineering) sense • Computational tractability • Dependency analysis machine learning • Inference • Easy for fully observed situations • Many algorithms if not fully observed • Dynamic programming / message passing

  14. Summary • Repeated structure - encode with plate • Chains, bipartite graphs, etc (more later) • Plates can intersect • Not all variables are observed Y Θ p ( X, θ ) = p ( θ ) p ( x i | θ ) Θ i x1 x2 x3 x4 xi

  16. Markov Chains n − 1 Y x 0 x 1 x 2 x 3 p ( x ; θ ) = p ( x 0 ; θ ) p ( x i +1 | x i ; θ ) i =1 x 0 x 1 x 2 Transition Matrices 0 1 0 1 0 1 x 0 x 2 x 3 0 0.4 x1 0 0.2 0.1 0 0.8 0.5 0 0 1 1 0.6 1 0.8 0.9 1 0.2 0.5 1 1 0 Unraveling the chain X p ( x 1 ) = p ( x 1 | x 0 ) p ( x 0 ) ⇐ ⇒ π 1 = Π 0 → 1 π 0 x 0 X p ( x 2 ) = p ( x 2 | x 1 ) p ( x 1 ) ⇐ ⇒ π 2 = Π 1 → 2 π 1 = Π 1 → 2 Π 0 → 1 π 0 x 1

  17. Markov Chains n − 1 Y x 0 x 1 x 2 x 3 p ( x ; θ ) = p ( x 0 ; θ ) p ( x i +1 | x i ; θ ) i =1 • From the start - sum sequentially i − 1 X Y p ( x i | x 1 ) = p ( x l +1 | x l ) · p ( x 2 | x 1 ) | {z } x j :1 <j<i l =2 =: l 2 ( x 2 ) i − 1 X Y X = p ( x l +1 | x l ) · p ( x 3 | x 2 ) l 2 ( x 2 ) x j :2 <j<i x 2 l =3 | {z } =: l 3 ( x 3 ) i − 1 X Y X = p ( x l +1 | x l ) · p ( x 4 | x 3 ) l 3 ( x 3 ) x j :3 <j<i l =4 x 3 | {z } =: l 4 ( x 4 )

  18. 
 Markov Chains n − 1 Y x 0 x 1 x 2 x 3 p ( x ; θ ) = p ( x 0 ; θ ) p ( x i +1 | x i ; θ ) i =1 x 0 x 1 x 2 Transition Matrices 0 1 0 1 0 1 x 0 x 2 x 3 0 0.4 x1 0 0.2 0.1 0 0.8 0.5 0 0 1 1 0.6 1 0.8 0.9 1 0.2 0.5 1 1 0 Unraveling the chain only need matrix-vector x0 = [0.4; 0.6]; 
 Pi1 = [0.2 0.1; 0.8 0.9]; 
 Pi2 = [0.8 0.5; 0.2 0.5]; 
 Pi3 = [0 1; 1 0]; 
 x3 = Pi3 * Pi2 * Pi1 * x0 = [0.45800; 0.54200]

  19. Markov Chains n − 1 Y x 0 x 1 x 2 x 3 p ( x ; θ ) = p ( x 0 ; θ ) p ( x i +1 | x i ; θ ) i =1 • From the end - sum sequentially n − 1 normalize in X Y p ( x 1 | x n ) ∝ p ( x l +1 | x l ) · 1 |{z} the end x j :1 <j<n l =1 =: r n ( x n ) n − 2 X Y X = p ( x l +1 | x l ) · p ( x n | x n − 1 ) r n ( x n ) x j :1 <j<n − 1 l =1 x n | {z } =: r n − 1 ( x n − 1 ) n − 3 X Y X = p ( x l +1 | x l ) · p ( x n − 1 | x n − 2 ) r n − 1 ( x n − 1 ) x j :1 <j<n − 2 l =1 x n − 1 | {z } =: r n − 2 ( x n − 2 )

  20. Example - inferring lunch current • Initial probability 
 p(x 0 =t)=p(x 0 =b) = 0.5 • Stationary transition matrix • On fifth day observed at Tazza d’oro p(x 5 =t)=1 0.9 0.2 • Distribution on day 3 • Left messages to 3 0.1 0.8 • Right messages to 3 • Renormalize

  21. Example - inferring lunch current > Pi = [0.9, 0.2; 0.1 0.8] Pi = 0.90000 0.20000 0.10000 0.80000 > l1 = [0.5; 0.5]; > l3 = Pi * Pi * l1 l3 = 0.58500 0.9 0.2 0.41500 > r5 = [1; 0]; > r3 = Pi' * Pi' * r5 r3 = 0.83000 0.1 0.8 0.34000 > (l3 .* r3) / sum(l3 .* r3) ans = 0.77483 0.22517

  22. 
 Message Passing l i = Π i l i � 1 x 0 x 1 x 2 x 3 x 4 x 5 r i = Π > i r i +1 • Send forward messages starting from left node 
 X m i − 1 → i ( x i ) = m i − 2 → i − 1 ( x i − 1 ) f ( x i − 1 , x i ) x i − 1 • Send backward messages starting from right node X m i +1 → i ( x i ) = m i +2 → i +1 ( x i +1 ) f ( x i , x i +1 ) x i +1

  23. Higher Order Markov Chains • First order chain x 0 x 1 x 2 x 3 Y p ( X ) = p ( x 0 ) p ( x i +1 | x i ) i • Second order x 0 x 1 x 2 x 3 Y p ( X ) = p ( x 0 , x 1 ) p ( x i +1 | x i , x i − 1 ) i

  24. Higher Order Markov Chains • First order chain x 0 x 1 x 2 x 3 Y p ( X ) = p ( x 0 ) p ( x i +1 | x i ) i • Second order x 0 x 1 x 2 x 3 Y p ( X ) = p ( x 0 , x 1 ) p ( x i +1 | x i , x i − 1 ) i

  25. Trees x3 x4 x5 x0 x1 x2 x7 x8 x6 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ...

  26. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 7 x 8 x 6 X X l 1 ( x 1 ) = p ( x 0 ) p ( x 1 | x 0 ) r 7 ( x 7 ) = p ( x 8 | x 7 ) x 0 x 8 X X l 2 ( x 2 ) = l 1 ( x 1 ) p ( x 2 | x 1 ) r 6 ( x 6 ) = r 7 ( x 7 ) p ( x 7 | x 6 ) x 1 x 7 X r 2 ( x 2 ) = r 6 ( x 6 ) p ( x 6 | x 2 ) x 6 X l 3 ( x 3 ) = l 2 ( x 2 ) p ( x 3 | x 2 ) r 2 ( x 2 ) x 2 . . .

  27. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 7 x 8 x 6 X X l 1 ( x 1 ) = p ( x 0 ) p ( x 1 | x 0 ) r 7 ( x 7 ) = p ( x 8 | x 7 ) x 0 x 8 X X l 2 ( x 2 ) = l 1 ( x 1 ) p ( x 2 | x 1 ) r 6 ( x 6 ) = r 7 ( x 7 ) p ( x 7 | x 6 ) x 1 x 7 X r 2 ( x 2 ) = r 6 ( x 6 ) p ( x 6 | x 2 ) x 6 X l 3 ( x 3 ) = l 2 ( x 2 ) p ( x 3 | x 2 ) r 2 ( x 2 ) x 2 . . .

  28. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 7 x 8 x 6 X X l 1 ( x 1 ) = p ( x 0 ) p ( x 1 | x 0 ) r 7 ( x 7 ) = p ( x 8 | x 7 ) x 0 x 8 X X l 2 ( x 2 ) = l 1 ( x 1 ) p ( x 2 | x 1 ) r 6 ( x 6 ) = r 7 ( x 7 ) p ( x 7 | x 6 ) x 1 x 7 X r 2 ( x 2 ) = r 6 ( x 6 ) p ( x 6 | x 2 ) x 6 X l 3 ( x 3 ) = l 2 ( x 2 ) p ( x 3 | x 2 ) r 2 ( x 2 ) x 2 . . .

  29. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 7 x 8 x 6 X X l 1 ( x 1 ) = p ( x 0 ) p ( x 1 | x 0 ) r 7 ( x 7 ) = p ( x 8 | x 7 ) x 0 x 8 X X l 2 ( x 2 ) = l 1 ( x 1 ) p ( x 2 | x 1 ) r 6 ( x 6 ) = r 7 ( x 7 ) p ( x 7 | x 6 ) x 1 x 7 X r 2 ( x 2 ) = r 6 ( x 6 ) p ( x 6 | x 2 ) x 6 X l 3 ( x 3 ) = l 2 ( x 2 ) p ( x 3 | x 2 ) r 2 ( x 2 ) x 2 . . .

  30. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 7 x 8 x 6 X X l 1 ( x 1 ) = p ( x 0 ) p ( x 1 | x 0 ) r 7 ( x 7 ) = p ( x 8 | x 7 ) x 0 x 8 X X l 2 ( x 2 ) = l 1 ( x 1 ) p ( x 2 | x 1 ) r 6 ( x 6 ) = r 7 ( x 7 ) p ( x 7 | x 6 ) x 1 x 7 X r 2 ( x 2 ) = r 6 ( x 6 ) p ( x 6 | x 2 ) x 6 X l 3 ( x 3 ) = l 2 ( x 2 ) p ( x 3 | x 2 ) r 2 ( x 2 ) x 2 . . .

  31. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 7 x 8 x 6 X X l 1 ( x 1 ) = p ( x 0 ) p ( x 1 | x 0 ) r 7 ( x 7 ) = p ( x 8 | x 7 ) x 0 x 8 X X l 2 ( x 2 ) = l 1 ( x 1 ) p ( x 2 | x 1 ) r 6 ( x 6 ) = r 7 ( x 7 ) p ( x 7 | x 6 ) x 1 x 7 X r 2 ( x 2 ) = r 6 ( x 6 ) p ( x 6 | x 2 ) x 6 X l 3 ( x 3 ) = l 2 ( x 2 ) p ( x 3 | x 2 ) r 2 ( x 2 ) x 2 . . .

  32. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 7 x 8 x 6 X X l 1 ( x 1 ) = p ( x 0 ) p ( x 1 | x 0 ) r 7 ( x 7 ) = p ( x 8 | x 7 ) x 0 x 8 X X l 2 ( x 2 ) = l 1 ( x 1 ) p ( x 2 | x 1 ) r 6 ( x 6 ) = r 7 ( x 7 ) p ( x 7 | x 6 ) x 1 x 7 X r 2 ( x 2 ) = r 6 ( x 6 ) p ( x 6 | x 2 ) x 6 X l 3 ( x 3 ) = l 2 ( x 2 ) p ( x 3 | x 2 ) r 2 ( x 2 ) x 2 . . .

  33. Junction Template • Order of computation • Dependence does not matter 
 (only matters for parametrization) X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 4 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 3 out in 1 2 i n 4

  34. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  35. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  36. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  37. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  38. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  39. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  40. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  41. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  42. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  43. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  44. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  45. Trees x 3 x 4 x 5 x 0 x 1 x 2 x 6 x 7 x 8 • Forward/Backward messages as normal for chain • When we have more edges for a vertex use ... X m 2 → 3 ( x 3 ) = m 1 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 2 , x 3 ) x 2 X m 2 → 6 ( x 6 ) = m 1 → 2 ( x 2 ) m 3 → 2 ( x 2 ) f ( x 2 , x 6 ) x 2 X m 2 → 1 ( x 1 ) = m 3 → 2 ( x 2 ) m 6 → 2 ( x 2 ) f ( x 1 , x 2 ) x 2

  46. Summary • Markov chains • Present only depends on recent past • Higher order - longer history. • Dynamic programming • Exponential if brute force. • Linear in chain if we iterate. • For junctions treat like chains but 
 3 integrate signals from all sources. out in • Exponential in the history size. 1 2 i n 4

  47. Hidden Markov Models

  48. Clustering and 
 Hidden Markov Models x 1 x 2 x 3 x 4 x m x i x i+1 ... y 1 y 2 y 3 y 4 y m y i i=1..m x 1 x 2 x 3 x 4 x m x i x i+1 ... y 1 y 2 y 3 y 4 y m y i i=1..m • Clustering - no dependence between observations • Hidden Markov Model - dependence between states

  49. Applications x 1 x 2 x 3 x 4 x m x i x i+1 ... y 1 y 2 y 3 y 4 y m y i i=1..m • Speech recognition (sound|text) • Optical character recognition (writing|text) • Gene finding (DNA sequence|genes) • Activity recognition (accelerometer|activity)

Recommend

Transforming Graphical System Models to Graphical Attack Models ! Joint work with Marieta

Transforming Graphical System Models to Graphical Attack Models ! Joint work with Marieta

From Graphical System Models... ... to Graphical Attack Models ... ... to Risk Assessment Transforming Graphical System Models to Graphical Attack Models ! Joint work with Marieta Georgieva Ivanova, ! ! Ren e Rydhof Hansen, and Florian

650 views • 27 slides
Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on

Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on

Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on Previously on Probabilistic Graphical Models Probabilistic Graphical Models Probability distribution and density functions Random variable

1.39k views • 104 slides
Undirected Graphical Models Aaron Courville, Universit de Montral 2 (UNDIRECTED) GRAPHICAL

Undirected Graphical Models Aaron Courville, Universit de Montral 2 (UNDIRECTED) GRAPHICAL

Undirected Graphical Models Aaron Courville, Universit de Montral 2 (UNDIRECTED) GRAPHICAL MODELS Overview : Directed versus undirected graphical models Conditional independence Energy function formalism Maximum likelihood

1.06k views • 61 slides
Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov networks: how it represents a prob. dist. independence assumptions

941 views • 57 slides
Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of

Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of

Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Basics Multivariate distributions with large number of variables Independency assumptions are useful

503 views • 36 slides
Graphical Models Independence &amp; Factorization Including structure Complexity of

Graphical Models Independence &amp; Factorization Including structure Complexity of

Graphical Models slides adopted from Sandro Schnborn 1 Graphical Models Independence &amp; Factorization Including structure Complexity of multivariate problems Independence assumptions Graphical Models Graphs to depict

385 views • 23 slides
Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak Ravanbakhsh Fall 2019 Learning objective Learning objective an intuition for inference in graphical models why is it difficult? exact inference by

1.49k views • 62 slides
Probabilistic Graphical Models CMSC 691 UMBC Two Problems for Graphical Models 1 ,

Probabilistic Graphical Models CMSC 691 UMBC Two Problems for Graphical Models 1 ,

Probabilistic Graphical Models CMSC 691 UMBC Two Problems for Graphical Models 1 , 2 , 3 , , = 1 Finding the normalizer Computing the marginals Two Problems for Graphical

778 views • 61 slides
Undirected Graphical Models: Markov Random Fields Probabilistic Graphical Models Sharif

Undirected Graphical Models: Markov Random Fields Probabilistic Graphical Models Sharif

Undirected Graphical Models: Markov Random Fields Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Markov Network Structure: undirected graph Undirected edges show correlations (non-causal

870 views • 53 slides
Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian networks Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives likelihood function and MLE role of the sufficient statistics MLE for

644 views • 60 slides
Outline 1 What are Graphical Models? Infering Graphical Models from Time Series Graph Theory 2

Outline 1 What are Graphical Models? Infering Graphical Models from Time Series Graph Theory 2

Outline Outline Outline 1 What are Graphical Models? Infering Graphical Models from Time Series Graph Theory 2 Modelling, Simulating, and Inference of Complex Biological Graphs and Digraphs Systems Dags 3 Probability Theory Speaker:

359 views • 7 slides
Graphical Models Graphical Models Relationship between the directed &amp; undirected models

Graphical Models Graphical Models Relationship between the directed &amp; undirected models

Graphical Models Graphical Models Relationship between the directed &amp; undirected models Siamak Ravanbakhsh Winter 2018 Two directions Two directions Markov network Bayes-net Markov network Bayes-net

311 views • 19 slides
Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected

Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected

Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected models Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives the form of likelihood for undirected models why is it difficult to

826 views • 49 slides
Two types of GMs Directed edges give causality relationships ( Bayesian Network or Directed

Two types of GMs Directed edges give causality relationships ( Bayesian Network or Directed

School of Computer Science Undirected Graphical Models Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 2, Sep 17, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X 2 X 2 X 2 Eric

692 views • 20 slides
Two types of GMs Directed edges give causality relationships ( Bayesian Network or Directed

Two types of GMs Directed edges give causality relationships ( Bayesian Network or Directed

School of Computer Science Undirected Graphical Models Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 2, Sep 17, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X 2 X 2 X 2 Eric

485 views • 21 slides
Directed Graphical Models + Undirected Graphical Models Matt Gormley Lecture 7 Sep. 18, 2019

Directed Graphical Models + Undirected Graphical Models Matt Gormley Lecture 7 Sep. 18, 2019

10-418 / 10-618 Machine Learning for Structured Data Machine Learning Department School of Computer Science Carnegie Mellon University Directed Graphical Models + Undirected Graphical Models Matt Gormley Lecture 7 Sep. 18, 2019 1 Q&amp;A

590 views • 39 slides
Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &amp;

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &amp;

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &amp; Variational Inference I Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives entropy exponential family distribution duality in

1.38k views • 85 slides
Graphical models Review Graphical models (Bayes nets, Markov random fields, factor graphs) !

Graphical models Review Graphical models (Bayes nets, Markov random fields, factor graphs) !

Graphical models Review Graphical models (Bayes nets, Markov random fields, factor graphs) ! graphical tests for conditional independence (e.g., d- separation for Bayes nets; Markov blanket) ! format conversions: always possible, may lose

363 views • 23 slides
The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models

The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models

School of Computer Science The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 4, Sep 26, 2007 Receptor A Receptor A Receptor B Receptor B X 1 X 1 X 1 X 2 X 2 X 2 Eric Xing

457 views • 17 slides
Graphical models Review P [( x y z ) ( y u ) ( z w ) ( z

Graphical models Review P [( x y z ) ( y u ) ( z w ) ( z

Graphical models Review P [( x y z ) ( y u ) ( z w ) ( z u v )] Dynamic programming on graphs ! variable elimination example ! Graphical model = graph + model ! e.g., Bayes net: DAG + CPTs !

469 views • 33 slides
Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives multivariate Gaussian density: different parametrizations marginalization and

540 views • 39 slides
Graphical Models Graphical Models Review of probability theory Review of probability theory

Graphical Models Graphical Models Review of probability theory Review of probability theory

Graphical Models Graphical Models Review of probability theory Review of probability theory Siamak Ravanbakhsh Winter 2018 Learning objectives Learning objectives Probability distribution and density functions Random variable Bayes' rule

988 views • 65 slides
Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that

Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that

Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that represents a probability distribution over random variables 1 , , Probabilistic Graphical Models A graph G that represents a

1.74k views • 129 slides
Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed

Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed

Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed &amp; undirected models Siamak Ravanbakhsh Fall 2019 Learning Objective Learning Objective understand the relationship between CIs in directed

694 views • 23 slides

More recommend