Scaling up Hybrid Probabilistic Inference with Logical and - PowerPoint PPT Presentation

Scaling up Hybrid Probabilistic Inference with Logical and Arithmetic Constraints via Message Passing Zhe Zeng* Paolo Morettin* Fanqi Yan* University of Trento, Italy University of California, Los Angeles AMSS, Chinese Academy of Sciences Antonio Vergari Guy Van den Broeck University of California, Los Angeles University of California, Los Angeles June 7th, 2020 - ICML 2020 - Virtual Vienna

Scaling up Hybrid Probabilistic Inference with Logical and Arithmetic Constraints via Message Passing Zhe Zeng* Paolo Morettin* Fanqi Yan* University of Trento, Italy University of California, Los Angeles AMSS, Chinese Academy of Sciences Antonio Vergari Guy Van den Broeck University of California, Los Angeles University of California, Los Angeles June 7th, 2020 - ICML 2020 - Virtual Vienna

Scaling up Hybrid Probabilistic Inference with Logical and Arithmetic Constraints via Message Passing Zhe Zeng* Paolo Morettin* Fanqi Yan* University of Trento, Italy University of California, Los Angeles AMSS, Chinese Academy of Sciences Antonio Vergari Guy Van den Broeck University of California, Los Angeles University of California, Los Angeles June 7th, 2020 - ICML 2020 - Virtual Vienna

Scaling up Hybrid Probabilistic Inference with Logical and Arithmetic Constraints via Message Passing Zhe Zeng* Paolo Morettin* Fanqi Yan* University of Trento, Italy University of California, Los Angeles AMSS, Chinese Academy of Sciences Antonio Vergari Guy Van den Broeck University of California, Los Angeles University of California, Los Angeles June 7th, 2020 - ICML 2020 - Virtual Vienna

Skill matching system 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system Each player has a certain skill 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system Each player has a certain skill ⇒ continuous variables 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system Each player has a certain skill Players can form teams 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system Each player has a certain skill Players can form teams intricate dependencies ⇒ 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system Each player has a certain skill Players can form teams Each team’s skill is bounded by its players’ skills 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system Each player has a certain skill Players can form teams Each team’s skill is bounded by its players’ skills complex constraints! ⇒ 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system Each player has a certain skill Players can form teams Each team’s skill is bounded by its players’ skills Good teams form a squad 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system Each player has a certain skill Players can form teams Each team’s skill is bounded by its players’ skills Good teams form a squad ⇒ discrete variables 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Skill matching system “What is the probability of team T 1 to outperform team T 2 , if T 1 is a squad but T 2 is not?” 5 /20 Minka et al., “Trueskill 2: An improved bayesian skill rating system”, 2018

Continuous + discrete + constraints = ? 6 /20

Continuous + discrete + constraints = ? Generative adversarial networks (GANs) [Goodfellow et al. 2014] Variational Autoencoders (VAEs) [Kingma et al. 2013] 6 /20

Continuous + discrete + constraints = ? Generative adversarial networks (GANs) [Goodfellow et al. 2014] Variational Autoencoders (VAEs) [Kingma et al. 2013] ⇒ limited inference capabilities, no constraints 6 /20

Continuous + discrete + constraints = ? Generative adversarial networks (GANs) [Goodfellow et al. 2014] Variational Autoencoders (VAEs) [Kingma et al. 2013] Hybrid Bayesian Netowrks (HBNs) [Heckerman et al. 1995; Shenoy et al. 2011] Mixed Probabilistic Graphical Models (MPGMs) [Yang et al. 2014] 6 /20

Continuous + discrete + constraints = ? Generative adversarial networks (GANs) [Goodfellow et al. 2014] Variational Autoencoders (VAEs) [Kingma et al. 2013] Hybrid Bayesian Netowrks (HBNs) [Heckerman et al. 1995; Shenoy et al. 2011] Mixed Probabilistic Graphical Models (MPGMs) [Yang et al. 2014] strong distributional assumptions ⇒ 6 /20

Continuous + discrete + constraints = ? Generative adversarial networks (GANs) [Goodfellow et al. 2014] Variational Autoencoders (VAEs) [Kingma et al. 2013] Hybrid Bayesian Netowrks (HBNs) [Heckerman et al. 1995; Shenoy et al. 2011] Mixed Probabilistic Graphical Models (MPGMs) [Yang et al. 2014] Tractable Probabilistic Circuits (PCs) [Molina et al. 2018; Vergari et al. 2019] 6 /20

Continuous + discrete + constraints = ? Generative adversarial networks (GANs) [Goodfellow et al. 2014] Variational Autoencoders (VAEs) [Kingma et al. 2013] Hybrid Bayesian Netowrks (HBNs) [Heckerman et al. 1995; Shenoy et al. 2011] Mixed Probabilistic Graphical Models (MPGMs) [Yang et al. 2014] Tractable Probabilistic Circuits (PCs) [Molina et al. 2018; Vergari et al. 2019] cannot deal with complex constraints ⇒ 6 /20

Continuous + discrete + constraints = SMT Satisfiability Modulo Theories of the linear arithmetic over the reals (SMT( LRA )) delivers all these ingredients by design! Widely used as a representation language for robotics , verification and planning [Barrett et al. 2010] 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

Continuous + discrete + constraints = SMT Each player has a certain skill 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

Continuous + discrete + constraints = SMT 0 ≤ X P i ≤ 10 for i = 1 , . . . , N 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

Continuous + discrete + constraints = SMT 0 ≤ X P i ≤ 10 for i = 1 , . . . , N Each team’s skill is bounded by its players’ skills 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

Continuous + discrete + constraints = SMT 0 ≤ X P i ≤ 10 for i = 1 , . . . , N | X T j − X P i | < 1 for j = 1 , . . . , M, i = 1 , . . . , | T j | 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

Continuous + discrete + constraints = SMT 0 ≤ X P i ≤ 10 for i = 1 , . . . , N | X T j − X P i | < 1 for j = 1 , . . . , M, i = 1 , . . . , | T j | Good teams form a squad 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

Continuous + discrete + constraints = SMT 0 ≤ X P i ≤ 10 for i = 1 , . . . , N | X T j − X P i | < 1 for j = 1 , . . . , M, i = 1 , . . . , | T j | B S j ⇒ X T j > 2 for j = 1 , . . . , M, i = 1 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

Continuous + discrete + constraints = SMT ∧ ∧ ∧ ∧ ∆ = 0 ≤ X P i ≤ 10 | X T j − X P i | < 1 ( B S j ⇒ X T j > 2) i ∈ T j i j j a single CNF SMT( LRA ) formula ∆ … 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

Continuous + discrete + constraints = SMT B S 1 X T 1 X T 2 B S 2 X P 1 X P 2 X P 3 X P 4 X P 5 X P 6 a single CNF SMT( LRA ) formula ∆ …and its primal graph 7 /20 Barrett et al., “Satisfiability modulo theories”, 2018

SMT + weights w ( X P i ) ,  ∧ 0 ≤ X P i ≤ 10  if 0 ≤ X P i ≤ 10    i   +   ∧ ∧  | X T j − X P i | < 1  w ( X T j , X P i ) ,  if | X T j − X P i | < 1 j i ∈ T j    ∧ ( B S j ⇒ X T j > 2)    w ( B S j , X T j ) ,   j   if B S j ⇒ X T j > 2  SMT formula ∆ weight functions W 8 /20 Belle et al., “Probabilistic inference in hybrid domains by weighted model integration”, 2015