bayesian network parameter learning
play

Bayesian Network Parameter Learning from Incomplete Data Guy Van - PowerPoint PPT Presentation

Efficient Algorithms for Bayesian Network Parameter Learning from Incomplete Data Guy Van den Broeck, Karthika Mohan, Arthur Choi, Adnan Darwiche, and Judea Pearl UCLA UAI 2015 Learning from Incomplete Data Input: data and BN structure


  1. Efficient Algorithms for Bayesian Network Parameter Learning from Incomplete Data Guy Van den Broeck, Karthika Mohan, Arthur Choi, Adnan Darwiche, and Judea Pearl UCLA UAI 2015

  2. Learning from Incomplete Data • Input: data and BN structure E.g., Gender wage gap study Gender Qualification X1 X2 X3 X4 ( X 1 ) ( X 3 ) (Gender) (Experience) (Qualification) (Income) 0 1 0 1 1 1 ? 1 0 1 0 1 1 ? 1 0 1 0 ? ? ( X 2 ) ( X 4 ) 0 0 ? ? Experience Income 0 1 0 1 • Output: BN parameters E.g., θ Gender , θ Experience|Gender , θ Qualification|Gender, , etc.

  3. Current Approaches: Properties Likelihood Optimization ✘ Inference-Free ✔ Consistent for MCAR ✔ Consistent for MAR ✘ Consistent for MNAR ✔ Maximum Likelihood

  4. Current Approaches: Properties Likelihood Expectation Optimization Maximization ✘ ✘ Inference-Free ✔ ✔ / ✘ Consistent for MCAR ✔ ✔ / ✘ Consistent for MAR ✘ ✘ Consistent for MNAR ✔ ✔ / ✘ Maximum Likelihood ✘ Closed Form n/a Passes over the data n/a ?

  5. Problem Statement Likelihood Expectation Optimization Maximization ✘ ✘ Inference-Free ✔ ✔ / ✘ Consistent for MCAR ✔ ✔ / ✘ Consistent for MAR ✘ ✘ Consistent for MNAR ✔ ✔ / ✘ Maximum Likelihood ✘ Closed Form n/a Passes over the data n/a ? Conventional wisdom: this is inevitable!

  6. Contribution Likelihood Expectation Deletion [this paper] Optimization Maximization ✘ ✘ ✔ Inference-Free ✔ ✔ / ✘ ✔ Consistent for MCAR ✔ ✔ / ✘ ✔ Consistent for MAR ✘ ✘ ✔ / ✘ Consistent for MNAR ✔ ✔ / ✘ ✘ Maximum Likelihood ✘ ✔ Closed Form n/a Passes over the data n/a ? 1

  7. Missingness Graphs Gender Qualification Gender Qualification ( X 1 ) ( X 3 ) ( X 1 ) ( X 3 ) R X4 R X2 R X3 * X 2 ( X 2 ) ( X 4 ) ( X 2 ) ( X 4 ) Experience Income Experience Income R X1 X 1 + Fully observed variables X o = {X 1 } * X 1 Partially observed variables * = X 1 if R X1 = ob X m = {X 2 , X 3 , X 4 } X 1 m if R X1 = unob

  8. Missingness Dataset X 1 X* 2 X* 3 R X2 R X3 P* • Encoding of the data 0 0 0 ob ob 0.200 0 0 1 ob ob 0.100 – Fully observed vars X o 0 1 0 ob ob 0.050 – Causal mechanisms R 0 1 1 ob ob 0.050 1 0 0 ob ob 0.060 – Proxies for X m 1 0 1 ob ob 0.040 1 1 0 ob ob 0.070 * = X 1 if R X1 = ob 1 1 1 ob ob 0.030 X 1 m if R X1 = unob 0 0 m ob unob 0.100 0 1 m ob unob 0.020 • Fully observed 1 0 m ob unob 0.080 1 1 m ob unob 0.180 • Data distribution Pr D (.) 0 m 0 unob ob 0.100 0 m 1 unob ob 0.020 … … … … … …

  9. Algorithms • Missingness categories (classes of graphs) – Missing Completely At Random (MCAR) – Missing At Random (MAR) – Missing Not At Random (MNAR) • Deletion techniques – Direct Deletion – Factored Deletion – Informed Deletion

  10. Missing Completely at Random (MCAR) ( X 1 ) ( X 3 ) ( X m X o ) R R X4 R X2 R X3 ( X 2 ) ( X 4 ) Experience Income

  11. Missing Completely at Random (MCAR) ( X 1 ) ( X 3 ) ( X m X o ) R R X4 R X2 R X3 ( X 2 ) ( X 4 ) Experience Income (X 1 X 2 X 3 X 4 ) (R X2 R X3 R X4 )

  12. Direct Deletion (MCAR) ( X m X o ) R Independencies: • (X 1 X 2 ) ⫫ R • (X 1 X 2 ) ⫫ R X2 Estimand: 𝑄𝑠 𝑌 1 , 𝑌 2

  13. Direct Deletion (MCAR) ( X m X o ) R Independencies: • (X 1 X 2 ) ⫫ R • (X 1 X 2 ) ⫫ R X2 Estimand: 𝑄𝑠 𝑌 1 , 𝑌 2 = 𝑄𝑠 𝑌 1 𝑌 2 𝑆 𝑌 2 = 𝑝𝑐

  14. Direct Deletion (MCAR) ( X m X o ) R Independencies: • (X 1 X 2 ) ⫫ R • (X 1 X 2 ) ⫫ R X2 Estimand: 𝑄𝑠 𝑌 1 , 𝑌 2 = 𝑄𝑠 𝑌 1 𝑌 2 𝑆 𝑌 2 = 𝑝𝑐 ∗ 𝑆 𝑌 2 = 𝑝𝑐 = 𝑄𝑠 𝑌 1 𝑌 2

  15. Direct Deletion (MCAR) ( X m X o ) R Independencies: • (X 1 X 2 ) ⫫ R • (X 1 X 2 ) ⫫ R X2 Estimand: 𝑄𝑠 𝑌 1 , 𝑌 2 = 𝑄𝑠 𝑌 1 𝑌 2 𝑆 𝑌 2 = 𝑝𝑐 ∗ 𝑆 𝑌 2 = 𝑝𝑐 = 𝑄𝑠 𝑌 1 𝑌 2 ∗ |𝑆 𝑌 2 = 𝑝𝑐) = 𝑄𝑠 𝐸 (𝑌 1 𝑌 2

  16. Direct Deletion (MCAR) ( X m X o ) R Independencies: X 1 X* 2 X* 3 R X2 R X3 P* • (X 1 X 2 ) ⫫ R 0 0 0 ob ob 0.200 • (X 1 X 2 ) ⫫ R X2 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 Estimand: … … … … … … 𝑄𝑠 𝑌 1 , 𝑌 2 0 1 m ob unob 0.020 = 𝑄𝑠 𝑌 1 𝑌 2 𝑆 𝑌 2 = 𝑝𝑐 1 0 m ob unob 0.080 1 1 m ob unob 0.180 ∗ 𝑆 𝑌 2 = 𝑝𝑐 = 𝑄𝑠 𝑌 1 𝑌 2 0 m 0 unob ob 0.100 ∗ |𝑆 𝑌 2 = 𝑝𝑐) 0 m 1 unob ob 0.020 = 𝑄𝑠 𝐸 (𝑌 1 𝑌 2 … … … … … …

  17. Direct Deletion (MCAR) ( X m X o ) R Independencies: X 1 X* 2 X* 3 R X2 R X3 P* • (X 1 X 2 ) ⫫ R 0 0 0 ob ob 0.200 • (X 1 X 2 ) ⫫ R X2 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 Estimand: … … … … … … 𝑄𝑠 𝑌 1 , 𝑌 2 0 1 m ob unob 0.020 = 𝑄𝑠 𝑌 1 𝑌 2 𝑆 𝑌 2 = 𝑝𝑐 1 0 m ob unob 0.080 1 1 m ob unob 0.180 ∗ 𝑆 𝑌 2 = 𝑝𝑐 = 𝑄𝑠 𝑌 1 𝑌 2 0 m 0 unob ob 0.100 ∗ |𝑆 𝑌 2 = 𝑝𝑐) 0 m 1 unob ob 0.020 = 𝑄𝑠 𝐸 (𝑌 1 𝑌 2 … … … … … … Cf. listwise and pairwise deletion in statistics

  18. Factored Deletion (MCAR) Many ways of factorizing the estimand! X 1 X* 2 X* 3 R X2 R X3 P* 0 0 0 ob ob 0.200 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 1 0 0 ob ob 0.060 1 0 1 ob ob 0.040 1 1 0 ob ob 0.070 1 1 1 ob ob 0.030 0 0 m ob unob 0.100 0 1 m ob unob 0.020 1 0 m ob unob 0.080 1 1 m ob unob 0.180 1 m m unob unob 0.020

  19. Factored Deletion (MCAR) Many ways of factorizing the estimand! X 1 X* 2 X* 3 R X2 R X3 P* 0 0 0 ob ob 0.200 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 1 0 0 ob ob 0.060 1 0 1 ob ob 0.040 1 1 0 ob ob 0.070 1 1 1 ob ob 0.030 0 0 m ob unob 0.100 𝑄(𝑌 1 ) 0 1 m ob unob 0.020 1 0 m ob unob 0.080 1 1 m ob unob 0.180 1 m m unob unob 0.020 1

  20. Factored Deletion (MCAR) Many ways of factorizing the estimand! X 1 X* 2 X* 3 R X2 R X3 P* 0 0 0 ob ob 0.200 𝑄 𝑌 2 = 𝑄(𝑌 2 |𝑆 𝑌 2 = 𝑝𝑐) 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 1 0 0 ob ob 0.060 1 0 1 ob ob 0.040 1 1 0 ob ob 0.070 1 1 1 ob ob 0.030 0 0 m ob unob 0.100 𝑄(𝑌 2 ) 𝑄(𝑌 1 ) 0 1 m ob unob 0.020 𝑄(𝑌 2 ) 1 0 m ob unob 0.080 1 1 m ob unob 0.180 1 m m unob unob 0.020 1

  21. Factored Deletion (MCAR) Many ways of factorizing the estimand! X 1 X* 2 X* 3 R X2 R X3 P* 0 0 0 ob ob 0.200 𝑄 𝑌 3 = 𝑄(𝑌 3 |𝑆 𝑌 3 = 𝑝𝑐) 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 1 0 0 ob ob 0.060 1 0 1 ob ob 0.040 1 1 0 ob ob 0.070 1 1 1 ob ob 0.030 0 0 m ob unob 0.100 𝑄(𝑌 2 ) 𝑄(𝑌 3 ) 𝑄(𝑌 1 ) 0 1 m ob unob 0.020 𝑄(𝑌 2 ) 1 0 m ob unob 0.080 1 1 m ob unob 0.180 1 m m unob unob 0.020 1

  22. Factored Deletion (MCAR) Many ways of factorizing the estimand! X 1 X* 2 X* 3 R X2 R X3 P* 0 0 0 ob ob 0.200 𝑄 𝑌 1 , 𝑌 2 = 𝑄 𝑌 2 𝑌 1 , 𝑆 𝑌 2 = 𝑝𝑐 𝑄(𝑌 1 ) 𝑄 𝑌 1 , 𝑌 2 = 𝑄(𝑌 1 |𝑌 2 , 𝑆 𝑌 2 = 𝑝𝑐) 𝑄(𝑌 2 |𝑆 𝑌 2 = 𝑝𝑐) 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 𝑄(𝑌 1 , 𝑌 2 ) 1 0 0 ob ob 0.060 𝑄(𝑌 2 |𝑌 1 ) 1 0 1 ob ob 0.040 1 1 0 ob ob 0.070 1 1 1 ob ob 0.030 0 0 m ob unob 0.100 𝑄(𝑌 2 ) 𝑄(𝑌 3 ) 𝑄(𝑌 1 ) 0 1 m ob unob 0.020 𝑄(𝑌 2 ) 1 0 m ob unob 0.080 1 1 m ob unob 0.180 1 m m unob unob 0.020 1

  23. Factored Deletion (MCAR) Many ways of factorizing the estimand! X 1 X* 2 X* 3 R X2 R X3 P* 0 0 0 ob ob 0.200 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 𝑄(𝑌 1 , 𝑌 2 ) 𝑄(𝑌 1 , 𝑌 3 ) 1 0 0 ob ob 0.060 𝑄(𝑌 2 |𝑌 1 ) 1 0 1 ob ob 0.040 1 1 0 ob ob 0.070 1 1 1 ob ob 0.030 0 0 m ob unob 0.100 𝑄(𝑌 2 ) 𝑄(𝑌 3 ) 𝑄(𝑌 1 ) 0 1 m ob unob 0.020 𝑄(𝑌 2 ) 1 0 m ob unob 0.080 1 1 m ob unob 0.180 1 m m unob unob 0.020 1

  24. Factored Deletion (MCAR) Many ways of factorizing the estimand! X 1 X* 2 X* 3 R X2 R X3 P* 0 0 0 ob ob 0.200 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 𝑄(𝑌 1 , 𝑌 2 ) 𝑄(𝑌 2 , 𝑌 3 ) 𝑄(𝑌 1 , 𝑌 3 ) 1 0 0 ob ob 0.060 𝑄(𝑌 2 |𝑌 1 ) 1 0 1 ob ob 0.040 𝑄(𝑌 2 |𝑌 3 ) 1 1 0 ob ob 0.070 1 1 1 ob ob 0.030 0 0 m ob unob 0.100 𝑄(𝑌 2 ) 𝑄(𝑌 3 ) 𝑄(𝑌 1 ) 0 1 m ob unob 0.020 𝑄(𝑌 2 ) 1 0 m ob unob 0.080 1 1 m ob unob 0.180 1 m m unob unob 0.020 1

  25. Factored Deletion (MCAR) Many ways of factorizing the estimand! X 1 X* 2 X* 3 R X2 R X3 P* 𝑄(𝑌 1 , 𝑌 2 , 𝑌 3 ) 0 0 0 ob ob 0.200 0 0 1 ob ob 0.100 0 1 0 ob ob 0.050 0 1 1 ob ob 0.050 𝑄(𝑌 1 , 𝑌 2 ) 𝑄(𝑌 2 , 𝑌 3 ) 𝑄(𝑌 1 , 𝑌 3 ) 1 0 0 ob ob 0.060 𝑄(𝑌 2 |𝑌 1 ) 1 0 1 ob ob 0.040 𝑄(𝑌 2 |𝑌 3 ) 1 1 0 ob ob 0.070 1 1 1 ob ob 0.030 0 0 m ob unob 0.100 𝑄(𝑌 2 ) 𝑄(𝑌 3 ) 𝑄(𝑌 1 ) 0 1 m ob unob 0.020 𝑄(𝑌 2 ) 1 0 m ob unob 0.080 1 1 m ob unob 0.180 1 m m unob unob 0.020 1

Recommend


More recommend