Strategy Synthesis for Quantitative Objectives Krishnendu Chatterjee - PowerPoint PPT Presentation

Strategy Synthesis for Quantitative Objectives Krishnendu Chatterjee 1 Mickael Randour 2 cois Raskin 3 Jean-Fran¸ 1 IST Austria 2 UMONS 3 ULB 30.11.2011

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Aim of this work Study games with (multi-dimensional) quantitative objectives: energy and mean-payoff. Address questions that revolve around strategies : ⊲ bounds on memory, ⊲ synthesis algorithm, ? ⊲ randomness ∼ memory. Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 1 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Results Overview Strategy synthesis MEGs MMPGs optimal finite-memory optimal optimal Memory exp. exp. infinite [CDHR10] Synthesis / EXPTIME EXPTIME Randomness as a substitute for finite-memory MEGs EPGs MMPGs MPBGs MPPGs √ √ √ (conj.) 1-player × × √ √ (conj.) 2-player × × × Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 2 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion 1 Classical energy and mean-payoff games 2 Extensions to multi-dimensions and parity 3 Strategy synthesis 4 Randomization as a substitute to finite-memory 5 Conclusion and ongoing work Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 3 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion 1 Classical energy and mean-payoff games 2 Extensions to multi-dimensions and parity 3 Strategy synthesis 4 Randomization as a substitute to finite-memory 5 Conclusion and ongoing work Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 4 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Turn-based games s 0 G = ( S 1 , S 2 , s init , E ) S = S 1 ∪ S 2 , S 1 ∩ S 2 = ∅ , E ⊆ S × S s 1 s 2 P 1 states = P 2 states = s 3 s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 5 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Turn-based games s 0 G = ( S 1 , S 2 , s init , E ) S = S 1 ∪ S 2 , S 1 ∩ S 2 = ∅ , E ⊆ S × S s 1 s 2 P 1 states = P 2 states = Play π = s 0 s 1 s 2 . . . s n . . . s.t. s 0 = s init s 3 Prefix ρ = π ( n ) = s 0 s 1 s 2 . . . s n s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 5 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Pure strategies s 0 Pure strategy for P i λ i ∈ Λ i : Prefs i ( G ) → S s.t. for all ρ ∈ Prefs i ( G ), (Last( ρ ) , λ i ( ρ )) ∈ E s 1 s 2 Memoryless strategy λ pm ∈ Λ PM : S i → S i i Finite-memory strategy s 3 λ fm ∈ Λ FM : Prefs i ( G ) → S , and can i i be encoded as a deterministic Moore machine s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 5 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Integer payoff function s 0 − 2 2 G = ( S 1 , S 2 , s init , E , w ) w : E → Z s 1 s 2 0 1 3 1 s 3 − 2 0 s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 5 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Integer payoff function s 0 − 2 2 G = ( S 1 , S 2 , s init , E , w ) w : E → Z s 1 s 2 0 1 Energy level EL( ρ ) = � i = n − 1 3 1 w ( s i , s i +1 ) i =0 Mean-payoff s 3 MP( π ) = lim inf n →∞ 1 n EL( π ( n )) − 2 0 s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 5 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Energy and mean-payoff objectives s 0 Energy objective − 2 2 Given initial credit v 0 ∈ N , PosEnergy G ( v 0 ) = { π ∈ Plays( G ) | s 1 s 2 0 1 ∀ n ≥ 0 : v 0 + EL( π ( n )) ∈ N } 3 1 Mean-payoff objective Given threshold v ∈ Q , s 3 MeanPayoff G ( v ) = { π ∈ Plays( G ) | MP( π ) ≥ v } − 2 0 s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 5 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Energy and mean-payoff objectives s 0 λ 1 ( s 3 ) = s 4 − 2 2 ⊲ λ 1 wins for MeanPayoff G ( − 1 4 ) s 1 s 2 0 1 ⊲ λ 1 loses for PosEnergy G ( v 0 ), for any arbitrary high initial credit 3 1 λ 1 ( s 3 ) = s 5 ⊲ λ 1 wins for MeanPayoff G ( 1 2 ) s 3 ⊲ λ 1 wins for PosEnergy G ( v 0 ), with v 0 = 2 − 2 0 s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 5 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Decision problems Unknown initial credit problem : ∃ ? v 0 ∈ N , λ 1 ∈ Λ 1 s.t. λ 1 wins for PosEnergy G ( v 0 ) Mean-payoff threshold problem : Given v ∈ Q , ∃ ? λ 1 ∈ Λ 1 s.t. λ 1 wins for MeanPayoff G ( v ) MPG threshold v problem equivalent to EG − v unknown initial credit problem [BFL + 08]. Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 6 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Complexity of EGs and MPGs EGs MPGs Memory to win memoryless memoryless [CdAHS03, BFL + 08] [EM79, LL69] Decision problem NP ∩ coNP NP ∩ coNP Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 7 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion 1 Classical energy and mean-payoff games 2 Extensions to multi-dimensions and parity 3 Strategy synthesis 4 Randomization as a substitute to finite-memory 5 Conclusion and ongoing work Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 8 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Multi-dimensional weights s 0 G = ( S 1 , S 2 , s init , E , w ) − 2 2 w : E → Z s 1 s 2 0 1 3 1 s 3 − 2 0 s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 9 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Multi-dimensional weights s 0 G = ( S 1 , S 2 , s init , E , k , w ) (2 , 1) (1 , − 2) w : E → Z k (0 , 0) s 1 s 2 (1 , 0) ⊲ multiple quantitative aspects (0 , − 2) ( − 3 , 3) ⊲ natural extensions of energy and mean-payoff objectives and associated s 3 decision problems (0 , 1) (1 , − 1) s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 9 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion MEGs & MMPGs Finite memory suffice for MEGs [CDHR10]. Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 10 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion MEGs & MMPGs Finite memory suffice for MEGs [CDHR10]. However, infinite memory is needed for MMPGs, even with only one player! [CDHR10] (2 , 0) (0 , 2) (0 , 0) s 0 s 1 (0 , 0) ⊲ To obtain MP( π ) = (1 , 1), P 1 has to visit s 0 and s 1 for longer and longer intervals before jumping from one to the other. ⊲ Any finite-memory strategy induces an ultimately periodic play s.t. MP( π ) = ( x , y ), x + y < 2. ⊲ With lim sup as MP the gap is huge : (2 , 2). Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 10 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion MEGs & MMPGs If players are restricted to finite memory [CDHR10], ⊲ MEGs and MMPGs are still determined and they are log-space equivalent, ⊲ the unknown initial credit and the mean-payoff threshold problems are coNP-complete, ⊲ no clue on memory bounds for P 1 (for P 2 , we know it is memoryless). Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 11 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion MEGs & MMPGs If players are restricted to finite memory [CDHR10], ⊲ MEGs and MMPGs are still determined and they are log-space equivalent, ⊲ the unknown initial credit and the mean-payoff threshold problems are coNP-complete, ⊲ no clue on memory bounds for P 1 (for P 2 , we know it is memoryless). Other interesting results on decision problems on MEGs are proved in [FJLS11]. Surprisingly, given a fixed initial vector, the problem becomes EXPSPACE-hard. Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 11 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Parity s 0 G = ( S 1 , S 2 , s init , E , w ) − 2 2 s 1 s 2 0 1 3 1 s 3 − 2 0 s 4 s 5 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 12 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Parity s 0 � � G p = S 1 , S 2 , s init , E , w , p p = 1 p : S → N − 2 2 Par( π ) = min { p ( s ) | s ∈ Inf( π ) } s 1 s 2 0 1 ⊲ Parity G p = p = 1 p = 3 { π ∈ Plays( G p ) | Par( π ) mod 2 = 0 } 3 1 ⊲ canonical way to express ω -regular s 3 objectives p = 2 ⊲ achieve the energy or mean-payoff objective while satisfying the parity − 2 0 condition s 4 s 5 p = 0 p = 2 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 12 / 42

EGs & MPGs Multi-dim. & parity Synthesis Randomization Conclusion Parity s 0 p = 1 To win the energy parity objective, − 2 2 P 1 must ⊲ visit s 4 infinitely often, s 1 s 2 0 1 p = 1 p = 3 ⊲ alternate with visits of s 5 to fund future visits of s 4 . 3 1 s 3 p = 2 − 2 0 s 4 s 5 p = 0 p = 2 Strat. Synth. for Quant. Obj. Chatterjee, Randour, Raskin 12 / 42