open world probabilistic databases
play

Open-World Probabilistic Databases Guy Van den Broeck GCAI Oct - PowerPoint PPT Presentation

Sep 17, 2022 •265 likes •2.03k views

Open-World Probabilistic Databases Guy Van den Broeck GCAI Oct 21, 2017 Overview 1. Why probabilistic databases? 2. How probabilistic query evaluation? 3. Why open world? 4. How open-world query evaluation? 5. What is the broader picture?

  1. Background: Positive Partitioned 2CNF A PP2CNF is: F = ∧ (i,j) ∈ E (x i  y j ) where E = the edge set of a bipartite graph x y 1 F = (x 1  y 1 ) ∧ (x 2  y 1 ) ∧ (x 2  y 3 ) 1 ∧ (x 1  y 3 ) ∧ (x 2  y 2 ) 2 2 3 Theorem: #PP2CNF is #P -hard [Provan’83]

  2. Our Problematic Clause H 0 = ∀ x ∀ y Smoker(x) ∨ Friend(x,y) ∨ Jogger(y)

  3. Our Problematic Clause H 0 = ∀ x ∀ y Smoker(x) ∨ Friend(x,y) ∨ Jogger(y) Theorem. Computing P(H 0 ) is #P -hard in the size of the database. [Dalvi&Suciu’04]

  4. Our Problematic Clause H 0 = ∀ x ∀ y Smoker(x) ∨ Friend(x,y) ∨ Jogger(y) Theorem. Computing P(H 0 ) is #P -hard in the size of the database. [Dalvi&Suciu’04] Proof: PP2CNF: F = (X i1 ∨ Y j1 ) ∧ (X i2 ∨ Y j2 ) ∧ … reduce #F to computing P (H 0 ) By example:

  5. Our Problematic Clause H 0 = ∀ x ∀ y Smoker(x) ∨ Friend(x,y) ∨ Jogger(y) Theorem. Computing P(H 0 ) is #P -hard in the size of the database. [Dalvi&Suciu’04] Proof: PP2CNF: F = (X i1 ∨ Y j1 ) ∧ (X i2 ∨ Y j2 ) ∧ … reduce #F to computing P (H 0 ) By example: F = (X 1 ∨ Y 1 ) ∧ (X 1 ∨ Y 2 ) ∧ (X 2 ∨ Y 2 )

  6. Our Problematic Clause H 0 = ∀ x ∀ y Smoker(x) ∨ Friend(x,y) ∨ Jogger(y) Theorem. Computing P(H 0 ) is #P -hard in the size of the database. [Dalvi&Suciu’04] Proof: PP2CNF: F = (X i1 ∨ Y j1 ) ∧ (X i2 ∨ Y j2 ) ∧ … reduce #F to computing P (H 0 ) By example: Probabilities (tuples not shown have P=1) F = (X 1 ∨ Y 1 ) ∧ (X 1 ∨ Y 2 ) ∧ (X 2 ∨ Y 2 ) Smoker Friend Jogger X P X Y P Y P x 1 0.5 x 1 y 1 0 y 1 0.5 x 2 0.5 x 1 y 2 0 y 2 0.5 x 2 y 2 0

  7. Our Problematic Clause H 0 = ∀ x ∀ y Smoker(x) ∨ Friend(x,y) ∨ Jogger(y) Theorem. Computing P(H 0 ) is #P -hard in the size of the database. [Dalvi&Suciu’04] Proof: PP2CNF: F = (X i1 ∨ Y j1 ) ∧ (X i2 ∨ Y j2 ) ∧ … reduce #F to computing P (H 0 ) By example: Probabilities (tuples not shown have P=1) F = (X 1 ∨ Y 1 ) ∧ (X 1 ∨ Y 2 ) ∧ (X 2 ∨ Y 2 ) Smoker Friend Jogger X P X Y P Y P P(H 0 ) = P(F); hence P (H 0 ) is #P-hard x 1 0.5 x 1 y 1 0 y 1 0.5 x 2 0.5 x 1 y 2 0 y 2 0.5 x 2 y 2 0

  8. Are the Lifted Rules Complete? You already know: • Inference rules: PTIME data complexity • Some queries: #P-hard data complexity [Dalvi and Suciu;JACM’11]

  9. Are the Lifted Rules Complete? You already know: • Inference rules: PTIME data complexity • Some queries: #P-hard data complexity Dichotomy Theorem for UCQ / Mon. CNF • If lifted rules succeed, then PTIME query • If lifted rules fail, then query is #P-hard [Dalvi and Suciu;JACM’11]

  10. Are the Lifted Rules Complete? You already know: • Inference rules: PTIME data complexity • Some queries: #P-hard data complexity Dichotomy Theorem for UCQ / Mon. CNF • If lifted rules succeed, then PTIME query • If lifted rules fail, then query is #P-hard Lifted rules are complete for UCQ! [Dalvi and Suciu;JACM’11]

  11. Commercial Break • Survey book (2017) http://www.nowpublishers.com/article/Details/DBS-052 • IJCAI 2016 tutorial http://web.cs.ucla.edu/~guyvdb/talks/IJCAI16-tutorial/

  12. Why open world?

  13. Knowledge Base Completion Coauthor Given: x y P Einstein Straus 0.7 Erdos Straus 0.6 Einstein Pauli 0.9 … … … 0.8::Coauthor(x,y) :- Coauthor(z,x) ∧ Coauthor(z,y). Learn: Complete: x y P Straus Pauli 0.504 … … …

  14. Bayesian Learning Loop Bayesian view on learning: 1. Prior belief: P( Coauthor(Straus,Pauli) ) = 0.01 2. Observe page P( Coauthor(Straus,Pauli| ) = 0.2 3. Observe page P( Coauthor(Straus,Pauli)| , ) = 0.3 Principled and sound reasoning!

  15. Problem: Broken Learning Loop Bayesian view on learning: 1. Prior belief: P( Coauthor(Straus,Pauli) ) = 0 2. Observe page P( Coauthor(Straus,Pauli| ) = 0.2 3. Observe page P( Coauthor(Straus,Pauli)| , ) = 0.3 [Ceylan , Darwiche, Van den Broeck; KR’16]

  16. Problem: Broken Learning Loop Bayesian view on learning: 1. Prior belief: P( Coauthor(Straus,Pauli) ) = 0 2. Observe page P( Coauthor(Straus,Pauli| ) = 0.2 3. Observe page P( Coauthor(Straus,Pauli)| , ) = 0.3 [Ceylan , Darwiche, Van den Broeck; KR’16]

  17. Problem: Broken Learning Loop Bayesian view on learning: 1. Prior belief: P( Coauthor(Straus,Pauli) ) = 0 2. Observe page P( Coauthor(Straus,Pauli| ) = 0.2 3. Observe page P( Coauthor(Straus,Pauli)| , ) = 0.3 This is mathematical nonsense! [Ceylan , Darwiche, Van den Broeck; KR’16]

  18. What we’d like to do… ∃ x Coauthor(Einstein,x) ∧ Coauthor(Erdos,x) Ernst Straus Kristian Kersting , … Justin Bieber , …

  19. Coauthor Open World DB X Y P Einstein Straus 0.7 Erdos Straus 0.6 • What if fact missing? Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 • Probability 0 for: … … … Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x )

  20. Coauthor Open World DB X Y P Einstein Straus 0.7 Erdos Straus 0.6 • What if fact missing? Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 • Probability 0 for: … … … Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) Q2 = ∃ x Coauthor(Bieber, x ) ∧ Coauthor(Erdos, x )

  21. Coauthor Open World DB X Y P Einstein Straus 0.7 Erdos Straus 0.6 • What if fact missing? Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 • Probability 0 for: … … … Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) Q2 = ∃ x Coauthor(Bieber, x ) ∧ Coauthor(Erdos, x ) Q3 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus )

  22. Coauthor Open World DB X Y P Einstein Straus 0.7 Erdos Straus 0.6 • What if fact missing? Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 • Probability 0 for: … … … Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) Q2 = ∃ x Coauthor(Bieber, x ) ∧ Coauthor(Erdos, x ) Q3 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) Q4 = Coauthor(Einstein, Bieber ) ∧ Coauthor(Erdos, Bieber )

  23. Coauthor Open World DB X Y P Einstein Straus 0.7 Erdos Straus 0.6 • What if fact missing? Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 • Probability 0 for: … … … Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) Q2 = ∃ x Coauthor(Bieber, x ) ∧ Coauthor(Erdos, x ) Q3 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) Q4 = Coauthor(Einstein, Bieber ) ∧ Coauthor(Erdos, Bieber ) Q5 = Coauthor(Einstein, Bieber ) ∧ ¬ Coauthor( Einstein , Bieber )

  24. X Y P Einstein Straus 0.7 Intuition Erdos Straus 0.6 Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) … … … Q3 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) Q4 = Coauthor(Einstein, Bieber ) ∧ Coauthor(Erdos, Bieber ) [Ceylan , Darwiche, Van den Broeck; KR’16]

  25. X Y P Einstein Straus 0.7 Intuition Erdos Straus 0.6 Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) … … … Q3 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) Q4 = Coauthor(Einstein, Bieber ) ∧ Coauthor(Erdos, Bieber ) We know for sure that P(Q1 ) ≥ P(Q3), P(Q1 ) ≥ P(Q4) [Ceylan , Darwiche, Van den Broeck; KR’16]

  26. X Y P Einstein Straus 0.7 Intuition Erdos Straus 0.6 Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) … … … Q3 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) Q4 = Coauthor(Einstein, Bieber ) ∧ Coauthor(Erdos, Bieber ) Q5 = Coauthor(Einstein, Bieber ) ∧ ¬ Coauthor( Einstein , Bieber ) We know for sure that P(Q1 ) ≥ P(Q3), P(Q1 ) ≥ P(Q4) and P(Q3) ≥ P(Q5), P(Q4) ≥ P(Q5) [Ceylan , Darwiche, Van den Broeck; KR’16]

  27. X Y P Einstein Straus 0.7 Intuition Erdos Straus 0.6 Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) … … … Q3 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) Q4 = Coauthor(Einstein, Bieber ) ∧ Coauthor(Erdos, Bieber ) Q5 = Coauthor(Einstein, Bieber ) ∧ ¬ Coauthor( Einstein , Bieber ) We know for sure that P(Q1 ) ≥ P(Q3), P(Q1 ) ≥ P(Q4) and P(Q3) ≥ P(Q5), P(Q4) ≥ P(Q5) because P(Q5) = 0. [Ceylan , Darwiche, Van den Broeck; KR’16]

  28. X Y P Einstein Straus 0.7 Intuition Erdos Straus 0.6 Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 Q1 = ∃ x Coauthor(Einstein, x ) ∧ Coauthor(Erdos, x ) … … … Q2 = ∃ x Coauthor(Bieber, x ) ∧ Coauthor(Erdos, x ) Q3 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) Q4 = Coauthor(Einstein, Bieber ) ∧ Coauthor(Erdos, Bieber ) Q5 = Coauthor(Einstein, Bieber ) ∧ ¬ Coauthor( Einstein , Bieber ) We know for sure that P(Q1 ) ≥ P(Q3), P(Q1 ) ≥ P(Q4) and P(Q3) ≥ P(Q5), P(Q4) ≥ P(Q5) because P(Q5) = 0. We have strong evidence that P(Q1) ≥ P(Q2). [Ceylan , Darwiche, Van den Broeck; KR’16]

  29. Problem: Curse of Superlinearity Reality is worse: tuples intentionally missing! [Ceylan , Darwiche, Van den Broeck; KR’16]

  30. Problem: Curse of Superlinearity Reality is worse: tuples intentionally missing! Sibling x y P … … … Facebook scale [Ceylan , Darwiche, Van den Broeck; KR’16]

  31. Problem: Curse of Superlinearity Reality is worse: tuples intentionally missing! Sibling x y P … … … ⇒ 200 Exabytes of data Facebook scale [Ceylan , Darwiche, Van den Broeck; KR’16]

  32. Problem: Curse of Superlinearity Reality is worse: tuples intentionally missing! Sibling x y P … … … ⇒ 200 Exabytes of data Facebook scale All Google storage is 2 exabytes … [Ceylan , Darwiche, Van den Broeck; KR’16]

  33. Problem: Curse of Superlinearity Reality is worse: tuples intentionally missing! Sibling x y P … … … ⇒ 200 Exabytes of data Facebook scale All Google storage is 2 exabytes … [Ceylan , Darwiche, Van den Broeck; KR’16]

  34. Problem: Model Evaluation Coauthor Given: x y P Einstein Straus 0.7 Erdos Straus 0.6 Einstein Pauli 0.9 … … … 0.8::Coauthor(x,y) :- Coauthor(z,x) ∧ Coauthor(z,y). Learn: OR 0.6::Coauthor(x,y) :- Affiliation(x,z) ∧ Affiliation(y,z). [De Raedt et al; IJCAI’15]

  35. Problem: Model Evaluation Coauthor Given: x y P Einstein Straus 0.7 Erdos Straus 0.6 Einstein Pauli 0.9 … … … 0.8::Coauthor(x,y) :- Coauthor(z,x) ∧ Coauthor(z,y). Learn: OR 0.6::Coauthor(x,y) :- Affiliation(x,z) ∧ Affiliation(y,z). What is the likelihood, precision, accuracy, …? [De Raedt et al; IJCAI’15]

  36. Open-World Prob. Databases Intuition: tuples can be added with P < λ Q2 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) P(Q2) ≥ 0 Coauthor X Y P Einstein Straus 0.7 Einstein Pauli 0.9 Erdos Renyi 0.7 Kersting Natarajan 0.8 Luc Paol 0.1 … … …

  37. Open-World Prob. Databases Intuition: tuples can be added with P < λ Q2 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) P(Q2) ≥ 0 Coauthor Coauthor X Y P X Y P Einstein Straus 0.7 Einstein Straus 0.7 Einstein Pauli 0.9 Einstein Pauli 0.9 Erdos Renyi 0.7 Erdos Renyi 0.7 Kersting Natarajan 0.8 Kersting Natarajan 0.8 Luc Paol 0.1 Luc Paol 0.1 … … … … … … λ Erdos Straus

  38. Open-World Prob. Databases Intuition: tuples can be added with P < λ Q2 = Coauthor(Einstein, Straus ) ∧ Coauthor(Erdos, Straus ) 0.7 * λ ≥ P(Q2) ≥ 0 Coauthor Coauthor X Y P X Y P Einstein Straus 0.7 Einstein Straus 0.7 Einstein Pauli 0.9 Einstein Pauli 0.9 Erdos Renyi 0.7 Erdos Renyi 0.7 Kersting Natarajan 0.8 Kersting Natarajan 0.8 Luc Paol 0.1 Luc Paol 0.1 … … … … … … λ Erdos Straus

  39. How open-world query evaluation?

  40. UCQ / Monotone CNF • Lower bound = closed-world probability • Upper bound = probability after adding all tuples with probability λ

  41. UCQ / Monotone CNF • Lower bound = closed-world probability • Upper bound = probability after adding all tuples with probability λ • Polynomial time ☺

  42. UCQ / Monotone CNF • Lower bound = closed-world probability • Upper bound = probability after adding all tuples with probability λ • Polynomial time ☺ • Quadratic blow-up  • 200 exabytes … again 

  43. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y))

  44. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) Decomposable ∃ -Rule P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y))

  45. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) Decomposable ∃ -Rule P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) …

  46. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) Decomposable ∃ -Rule P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) … Complexity PTIME

  47. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) Decomposable ∃ -Rule P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) Check independence: Scientist(A) ∧ ∃ y Coauthor(A,y) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) Scientist(B) ∧ ∃ y Coauthor(B,y) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) … Complexity PTIME

  48. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) …

  49. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) No supporting facts x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) in database! x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) …

  50. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) No supporting facts x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) in database! x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) … Probability 0 in closed world

  51. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) No supporting facts x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) in database! x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) … Probability 0 in closed world Ignore these sub-queries!

  52. Closed-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) No supporting facts x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) in database! x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) … Probability 0 in closed world Ignore these sub-queries! Complexity linear time!

  53. Open-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) No supporting facts x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) in database! x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) …

  54. Open-World Lifted Query Eval Q = ∃ x ∃ y Scientist(x) ∧ Coauthor(x,y) P(Q) = 1 - Π A ∈ Domain (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) = 1 - (1 - P(Scientist(A) ∧ ∃ y Coauthor(A,y)) x (1 - P(Scientist(B) ∧ ∃ y Coauthor(B,y)) x (1 - P(Scientist(C) ∧ ∃ y Coauthor(C,y)) x (1 - P(Scientist(D) ∧ ∃ y Coauthor(D,y)) No supporting facts x (1 - P(Scientist(E) ∧ ∃ y Coauthor(E,y)) in database! x (1 - P(Scientist(F) ∧ ∃ y Coauthor(F,y)) … Probability λ in open world

Recommend

Open-World Probabilistic Databases Guy Van den Broeck FLAIRS May 23, 2017 Overview 1. Why

Open-World Probabilistic Databases Guy Van den Broeck FLAIRS May 23, 2017 Overview 1. Why

Open-World Probabilistic Databases Guy Van den Broeck FLAIRS May 23, 2017 Overview 1. Why probabilistic databases? 2. How probabilistic query evaluation? 3. Why open world? 4. How open-world query evaluation? 5. What is the broader picture? Why

866 views • 51 slides
Probabilistic Databases Guy Van den Broeck Scalable Uncertainty Management (SUM) Sep 21, 2016

Probabilistic Databases Guy Van den Broeck Scalable Uncertainty Management (SUM) Sep 21, 2016

Open-World Probabilistic Databases Guy Van den Broeck Scalable Uncertainty Management (SUM) Sep 21, 2016 Overview 1. Why probabilistic databases? 2. How probabilistic query evaluation? 3. Why open world? 4. How open-world query evaluation? 5.

1.73k views • 156 slides
Open-World Probabilistic Databases Guy Van den Broeck On joint work with Ismail Ilkan Ceylan,

Open-World Probabilistic Databases Guy Van den Broeck On joint work with Ismail Ilkan Ceylan,

Open-World Probabilistic Databases Guy Van den Broeck On joint work with Ismail Ilkan Ceylan, Adnan Darwiche Feb 3, 2016, SML Outline? or What we can do already &gt; 570 million entities &gt; 18 billion tuples What I want to do

642 views • 52 slides
Databases Peter Zaitsev Percona You Might Know I am passionate about Open Source Databases

Databases Peter Zaitsev Percona You Might Know I am passionate about Open Source Databases

Maximizing the Power and Value of Open Source Databases Peter Zaitsev Percona You Might Know I am passionate about Open Source Databases 2 Open Source Databases Leading in Growth Source: https://db-engines.com/en/ranking 3 Long Term

708 views • 42 slides
Scalable Uncertainty Management 04 Probabilistic Databases Rainer Gemulla Jun 1, 2012

Scalable Uncertainty Management 04 Probabilistic Databases Rainer Gemulla Jun 1, 2012

Scalable Uncertainty Management 04 Probabilistic Databases Rainer Gemulla Jun 1, 2012 Overview In this lecture Refresher: Finite probability (not presented) What is a probabilistic database? How can probabilistic information be

469 views • 46 slides
Open Source Databases Peter Zaitsev, CEO Percona What a Year! Huge changes for Open Source and

Open Source Databases Peter Zaitsev, CEO Percona What a Year! Huge changes for Open Source and

The State of Open Source Databases Peter Zaitsev, CEO Percona What a Year! Huge changes for Open Source and Open Source databases RedHat Acquired by IBM Image Source: https://techcrunch.com/story/ibm-acquires-red-hat/ Open Source

663 views • 29 slides
Querying Probabilistic XML Databases Sept. 21 st 2012 Asma Souihli Network and Computer Science

Querying Probabilistic XML Databases Sept. 21 st 2012 Asma Souihli Network and Computer Science

Querying Probabilistic XML Databases Sept. 21 st 2012 Asma Souihli Network and Computer Science Department XML for semi-structured data ( tree-like structure) 2 Probabilistic Data - PrXML Jung-Hee Yun and Chin-Wan Chung, 2012. 3 Context

633 views • 41 slides
Aggregation in Probabilistic Databases via Knowledge Compilation Robert Fink, Larisa Han, Dan

Aggregation in Probabilistic Databases via Knowledge Compilation Robert Fink, Larisa Han, Dan

Aggregation in Probabilistic Databases via Knowledge Compilation Robert Fink, Larisa Han, Dan Olteanu University of Oxford VLDB 2012, Istanbul 1 / 30 Outline Motivation Algebraic Foundations Representation System Query Evaluation 2 / 30

867 views • 49 slides
A Novel Probabilistic Pruning Approach to Speed Up Similarity Queries in Uncertain Databases

A Novel Probabilistic Pruning Approach to Speed Up Similarity Queries in Uncertain Databases

LUDWIG- MAXIMILIANS- DEPARTMENT DATABASE UNIVERSITY INSTITUTE FOR SYSTEMS MUNICH INFORMATICS GROUP A Novel Probabilistic Pruning Approach to Speed Up Similarity Queries in Uncertain Databases Thomas Bernecker*, Tobias Emrich*,

266 views • 25 slides
Dissociation-based Optimization in Probabilistic Databases Maarten Van den Heuvel 1 , Floris

Dissociation-based Optimization in Probabilistic Databases Maarten Van den Heuvel 1 , Floris

Dissociation-based Optimization in Probabilistic Databases Maarten Van den Heuvel 1 , Floris Geerts 1 , Martin Theobald 2 1 Universiteit Antwerpen, Belgium 2 Ulm University, Germany Contents Introduction Issues with safety

1.09k views • 25 slides
Open Source Databases for Modern Business Santa Clara, California | April 23th 25th, 2018

Open Source Databases for Modern Business Santa Clara, California | April 23th 25th, 2018

Open Source Databases for Modern Business Santa Clara, California | April 23th 25th, 2018 Peter Zaitsev, CEO Percona My Favorite Topic Open Source Databases 2 Open Source Databases Leading in Growth Source:

671 views • 49 slides
Techniques for managing probabilistic data Dan Suciu University of Washington 1 Databases Are

Techniques for managing probabilistic data Dan Suciu University of Washington 1 Databases Are

Techniques for managing probabilistic data Dan Suciu University of Washington 1 Databases Are Deterministic Applications since 1970s required precise semantics Accounting, inventory Database tools are deterministic A

1.66k views • 144 slides
A Dichotomy for Non-Repeating Queries with Negation in Probabilistic Databases Robert Fink and Dan

A Dichotomy for Non-Repeating Queries with Negation in Probabilistic Databases Robert Fink and Dan

A Dichotomy for Non-Repeating Queries with Negation in Probabilistic Databases Robert Fink and Dan Olteanu PODS June 24, 2014 1 / 20 Outline The Dichotomy The Interesting but Hard Queries The Easy Queries Leftovers 2 / 20 Problem Setting

505 views • 24 slides
Scalable Uncertainty Management 05 Query Evaluation in Probabilistic Databases Rainer Gemulla

Scalable Uncertainty Management 05 Query Evaluation in Probabilistic Databases Rainer Gemulla

Scalable Uncertainty Management 05 Query Evaluation in Probabilistic Databases Rainer Gemulla Jun 1, 2012 Overview In this lecture Primer: relational calculus Understand complexity of query evaluation How to determine whether a query is

1.54k views • 119 slides
The State of Open Source Databases Peter Zaitsev CEO, Percona October 1 st , 2019 Open Source

The State of Open Source Databases Peter Zaitsev CEO, Percona October 1 st , 2019 Open Source

The State of Open Source Databases Peter Zaitsev CEO, Percona October 1 st , 2019 Open Source Software Amazing times for Open Source Software! Commercial Success Fantastic Success of Open Source Based Businesses! RedHat Acquired by IBM

1.19k views • 52 slides
3D, Open World, Puzzle Game Team Parallel Design Concept Third Person 3D Open World We

3D, Open World, Puzzle Game Team Parallel Design Concept Third Person 3D Open World We

3D, Open World, Puzzle Game Team Parallel Design Concept Third Person 3D Open World We prefer a third person view. The players movement itself is normal control like arrow keys. In the open world there will be lots of

108 views • 8 slides
Counting Problems over Incomplete Databases Marcelo Arenas, Pablo Barcel, Mikal Monet June

Counting Problems over Incomplete Databases Marcelo Arenas, Pablo Barcel, Mikal Monet June

Counting Problems over Incomplete Databases Marcelo Arenas, Pablo Barcel, Mikal Monet June 15th, 2020 Incomplete databases Probabilistic databases: one way of dealing with uncertain data But this is not what is used in practice most

431 views • 41 slides
A Storm is Coming! A New Probabilistic Model Checker Joost-Pieter Katoen IFIP WG 1.8 Open

A Storm is Coming! A New Probabilistic Model Checker Joost-Pieter Katoen IFIP WG 1.8 Open

A Storm is Coming! A New Probabilistic Model Checker Joost-Pieter Katoen IFIP WG 1.8 Open Problems in Concurrency Theory, June 26, 2017 joint with: Christian Dehnert, Sebastian Junges and Matthias Volk Probabilistic Model Checking Theory in

468 views • 28 slides
GIS Databases What is GIS? &quot;...the merging of cartography, statistical analysis, and

GIS Databases What is GIS? &quot;...the merging of cartography, statistical analysis, and

GIS Databases What is GIS? &quot;...the merging of cartography, statistical analysis, and database technology.&quot; Open GIS: Background &amp; History OGC (Open Geospatial Network Consortium) Unix, NASA, Corps of Engineers, Ag.

135 views • 11 slides
Creating Databases and Tables Introduction to Databases in Python Creating Databases

Creating Databases and Tables Introduction to Databases in Python Creating Databases

INTRODUCTION TO DATABASES IN PYTHON Creating Databases and Tables Introduction to Databases in Python Creating Databases Varies by the database type Databases like PostgreSQL and MySQL have command line tools to initialize

878 views • 31 slides
Databases and PHP Accessing databases from PHP PHP &amp; Databases l PHP can connect to

Databases and PHP Accessing databases from PHP PHP &amp; Databases l PHP can connect to

Databases and PHP Accessing databases from PHP PHP &amp; Databases l PHP can connect to virtually any database l There are specific functions built-into PHP to connect with some DB l There is also generic ODBC functions that will work with many

868 views • 28 slides
Open-access Crystallographic Databases ( for the teaching of geometric structural crystallography

Open-access Crystallographic Databases ( for the teaching of geometric structural crystallography

Open-access Crystallographic Databases ( for the teaching of geometric structural crystallography and much more ) Peter Moeck , Physics, Portland State University , &amp; all other Members of International Advisory Board of the Crystallography

431 views • 24 slides
Recent activities around the crystallography open databases COD, PCOD and P2D2 Armel Le Bail and

Recent activities around the crystallography open databases COD, PCOD and P2D2 Armel Le Bail and

Recent activities around the crystallography open databases COD, PCOD and P2D2 Armel Le Bail and the COD Advisory Board Universit du Maine, Laboratoire des oxydes et Fluorures, CNRS UMR 6010, Avenue O. Messiaen, 72085 Le Mans Cedex 9, France.

639 views • 45 slides
Social Impact of Open Data by Sandra Moscoso, World Bank World Bank Group Open Finances,

Social Impact of Open Data by Sandra Moscoso, World Bank World Bank Group Open Finances,

Photo Credit: Andrea, Flickr Social Impact of Open Data by Sandra Moscoso, World Bank World Bank Group Open Finances, https://finances.worldbank.org World Bank Groups Media Development Program, Social Impact The net effect of an activity

373 views • 19 slides

More recommend