Probabilistic Open Games Neil Ghani, Clemens Kupke, Alasdair - PowerPoint PPT Presentation

Probabilistic Open Games Neil Ghani, Clemens Kupke, Alasdair Lambert, Fredrik Nordvall Forsberg University Of Strathclyde SYCO3, Oxford, 28 March 2019

Game Theory

What is Game Theory? ◮ The mathematical study of strategic interaction between rational agents 2 / 25

What is Game Theory? ◮ The mathematical study of strategic interaction between rational agents ◮ Agents pick a strategy to play 2 / 25

What is Game Theory? ◮ The mathematical study of strategic interaction between rational agents ◮ Agents pick a strategy to play ◮ The outcome is determined by collective action of all agents 2 / 25

What is Game Theory? ◮ The mathematical study of strategic interaction between rational agents ◮ Agents pick a strategy to play ◮ The outcome is determined by collective action of all agents ◮ The outcome determines the utility each agent receives 2 / 25

What is Game Theory? ◮ The mathematical study of strategic interaction between rational agents ◮ Agents pick a strategy to play ◮ The outcome is determined by collective action of all agents ◮ The outcome determines the utility each agent receives ◮ Analyse these games via equilibrium 2 / 25

What is an equilibrium? ( σ 0 , σ 1 ) Nash Equilibrium if ◮ σ 0 ∈ arg max { u 0 ( σ ′ , σ 1 ) } ; and σ ′ ∈ Σ 0 ◮ σ 1 ∈ arg max { u 1 ( σ 0 , σ ′′ ) } σ ′′ ∈ Σ 1 3 / 25

Prisoner’s dilemma Player 2 C D 3 , 3 0 , 4 C Player 1 4 , 0 1 , 1 D 4 / 25

Prisoner’s dilemma Player 2 C D 3 , 3 0 , 4 C Player 1 4 , 0 1 , 1 D Only equilibrium: ( D , D ) . 4 / 25

Matching Pennies Bob H T H − 1 , 1 1 , − 1 Alice T 1 , − 1 − 1 , 1 5 / 25

Matching Pennies Bob H T H − 1 , 1 1 , − 1 Alice T 1 , − 1 − 1 , 1 No equilibrium. 5 / 25

Matching Pennies Bob H T H − 1 , 1 1 , − 1 Alice T 1 , − 1 − 1 , 1 No pure equilibrium. 5 / 25

Matching Pennies Bob H T H − 1 , 1 1 , − 1 Alice T 1 , − 1 − 1 , 1 No pure equilibrium. Only mixed equilibrium: both play 1 2 H + 1 2 T . 5 / 25

Problems with Game Theory ◮ Complexity issues ◮ Finding equilibria is computationally hard ◮ Games do not compose 6 / 25

Pure Open Games

Pure Open Games [Hedges 2016] Neil Ghani, Jules Hedges, Viktor Winschel, Philipp Zahn Compositional game theory. LICS 2018. ◮ A framework for building games compositionally ◮ Applying Category Theory to Game Theory 7 / 25

Pure open games: definition S X Σ R Y Let X , Y , R and S be sets. A pure open game G : ( X , S ) → ( Y , R ) consists of: 8 / 25

Pure open games: definition S X Σ R Y Let X , Y , R and S be sets. A pure open game G : ( X , S ) → ( Y , R ) consists of: ◮ a set Σ of strategy profiles for G 8 / 25

Pure open games: definition S X Σ R Y Let X , Y , R and S be sets. A pure open game G : ( X , S ) → ( Y , R ) consists of: ◮ a set Σ of strategy profiles for G ◮ a play function P : Σ × X → Y 8 / 25

Pure open games: definition S X Σ R Y Let X , Y , R and S be sets. A pure open game G : ( X , S ) → ( Y , R ) consists of: ◮ a set Σ of strategy profiles for G ◮ a play function P : Σ × X → Y ◮ a coutility function C : Σ × X × R → S 8 / 25

Pure open games: definition S X Σ R Y Let X , Y , R and S be sets. A pure open game G : ( X , S ) → ( Y , R ) consists of: ◮ a set Σ of strategy profiles for G ◮ a play function P : Σ × X → Y ◮ a coutility function C : Σ × X × R → S ◮ an equilibrium function E : X × ( Y → R ) → P (Σ) . 8 / 25

Pure open games: parallel composition S X S’ X’ ⊗ Σ Σ ′ R Y R’ Y’ 9 / 25

Pure open games: parallel composition S × S’ X × X’ Σ × Σ ′ R × R’ Y × Y’ 9 / 25

Pure open games: sequential composition S X Σ R Y ◦ R Y Σ ′ T Z 10 / 25

Pure open games: sequential composition S X Σ × Σ ′ T Z 10 / 25

Incorporating mixed strategies ◮ Want to also capture mixed strategies. ◮ Solution: use the distributions monad for categorical probability theory [Perrone 2018]. 11 / 25

Commutative Monads

Monads and strength ◮ A strong monad on a monoidal category C is a monad ( T , η, µ ) with a left strength s l : A ⊗ TB → T ( A ⊗ B ) . 12 / 25

Monads and strength ◮ A strong monad on a monoidal category C is a monad ( T , η, µ ) with a left strength s l : A ⊗ TB → T ( A ⊗ B ) . ◮ If C is symmetric monoidal, we can define a right strength s r : TA ⊗ B → T ( A ⊗ B ) by T γ � T ( A ⊗ B ) γ s l � B ⊗ TA � T ( B ⊗ A ) TA ⊗ B 12 / 25

� � � Commutative monads A strong monad on a symmetric monoidal category is commutative if s l Ts r � TT ( A ⊗ B ) TA ⊗ TB T ( TA ⊗ B ) s r µ Ts l � TT ( A ⊗ B ) µ � T ( A ⊗ B ) T ( A ⊗ TB ) We call this map ℓ : TA ⊗ TB → T ( A ⊗ B ) . 13 / 25

The finite distribution monad D : Set → Set Probability distribution on X : ◮ function ω : X → [ 0 , 1 ] ◮ � ω ( x ) = 1 x ◮ finite support. 14 / 25

The finite distribution monad D : Set → Set Probability distribution on X : ◮ function ω : X → [ 0 , 1 ] ◮ � ω ( x ) = 1 x ◮ finite support. D ( X ) collection of distributions on X . ◮ η : X → D X point distribution. ◮ µ : D 2 X → D X flattens distributions of distributions. ◮ ℓ : D X × D Y → D ( X × Y ) independent joint distribution. ◮ D -algebras: convex sets R , with “expectation” E : D R → R . 14 / 25

Probabilistic Open Games

Probabilistic Open Games Let X and Y be sets, and R and S D -algebras. A probabilistic open game G : ( X , S ) → ( Y , R ) consists of 15 / 25

Probabilistic Open Games Let X and Y be sets, and R and S D -algebras. A probabilistic open game G : ( X , S ) → ( Y , R ) consists of ◮ a set Σ of strategies 15 / 25

Probabilistic Open Games Let X and Y be sets, and R and S D -algebras. A probabilistic open game G : ( X , S ) → ( Y , R ) consists of ◮ a set Σ of strategies ◮ a play function P : Σ × X → Y 15 / 25

Probabilistic Open Games Let X and Y be sets, and R and S D -algebras. A probabilistic open game G : ( X , S ) → ( Y , R ) consists of ◮ a set Σ of strategies ◮ a play function P : Σ × X → Y ◮ a coutility function C : Σ × X × R → S 15 / 25

Probabilistic Open Games Let X and Y be sets, and R and S D -algebras. A probabilistic open game G : ( X , S ) → ( Y , R ) consists of ◮ a set Σ of strategies ◮ a play function P : Σ × X → Y ◮ a coutility function C : Σ × X × R → S ◮ an equilibrium function E : X × ( Y → R ) → P ( D Σ) 15 / 25

Parallel composition Play, coplay same as in pure case. For games G : ( X , S ) → ( Y , R ) and H : ( X ′ , S ′ ) → ( Y ′ , R ′ ) we need to define the equilibrium E G ⊗ H : X × X ′ × ( Y × Y ′ → R × R ′ ) → P ( D (Σ × Σ ′ )) 16 / 25

Parallel composition Play, coplay same as in pure case. For games G : ( X , S ) → ( Y , R ) and H : ( X ′ , S ′ ) → ( Y ′ , R ′ ) we need to define the equilibrium E G ⊗ H : X × X ′ × ( Y × Y ′ → R × R ′ ) → P ( D (Σ × Σ ′ )) Φ ∈ E G ⊗ H ( x 1 , x 2 ) k iff 16 / 25

Parallel composition Play, coplay same as in pure case. For games G : ( X , S ) → ( Y , R ) and H : ( X ′ , S ′ ) → ( Y ′ , R ′ ) we need to define the equilibrium E G ⊗ H : X × X ′ × ( Y × Y ′ → R × R ′ ) → P ( D (Σ × Σ ′ )) Φ ∈ E G ⊗ H ( x 1 , x 2 ) k iff Φ = ℓ ( φ 1 , φ 2 ) and 16 / 25

Parallel composition Play, coplay same as in pure case. For games G : ( X , S ) → ( Y , R ) and H : ( X ′ , S ′ ) → ( Y ′ , R ′ ) we need to define the equilibrium E G ⊗ H : X × X ′ × ( Y × Y ′ → R × R ′ ) → P ( D (Σ × Σ ′ )) Φ ∈ E G ⊗ H ( x 1 , x 2 ) k iff Φ = ℓ ( φ 1 , φ 2 ) and φ 1 ∈ E G 16 / 25

Parallel composition Play, coplay same as in pure case. For games G : ( X , S ) → ( Y , R ) and H : ( X ′ , S ′ ) → ( Y ′ , R ′ ) we need to define the equilibrium E G ⊗ H : X × X ′ × ( Y × Y ′ → R × R ′ ) → P ( D (Σ × Σ ′ )) Φ ∈ E G ⊗ H ( x 1 , x 2 ) k iff Φ = ℓ ( φ 1 , φ 2 ) and φ 1 ∈ E G x 1 16 / 25

Parallel composition Play, coplay same as in pure case. For games G : ( X , S ) → ( Y , R ) and H : ( X ′ , S ′ ) → ( Y ′ , R ′ ) we need to define the equilibrium E G ⊗ H : X × X ′ × ( Y × Y ′ → R × R ′ ) → P ( D (Σ × Σ ′ )) Φ ∈ E G ⊗ H ( x 1 , x 2 ) k iff Φ = ℓ ( φ 1 , φ 2 ) and φ 1 ∈ E G x 1 E [ D ( π 0 ) ◦ D ( k ) ◦ ℓ ( η _ , D ( P H ( _ , x 2 )) φ 2 )] and 16 / 25

Parallel composition Play, coplay same as in pure case. For games G : ( X , S ) → ( Y , R ) and H : ( X ′ , S ′ ) → ( Y ′ , R ′ ) we need to define the equilibrium E G ⊗ H : X × X ′ × ( Y × Y ′ → R × R ′ ) → P ( D (Σ × Σ ′ )) Φ ∈ E G ⊗ H ( x 1 , x 2 ) k iff Φ = ℓ ( φ 1 , φ 2 ) and φ 1 ∈ E G x 1 E [ D ( π 0 ) ◦ D ( k ) ◦ ℓ ( η _ , D ( P H ( _ , x 2 )) φ 2 )] and φ 2 ∈ E H x 2 E [ D ( π 1 ) ◦ D ( k ) ◦ ℓ ( D ( P G ( _ , x 1 )) φ 1 , η _ )] 16 / 25