Chapter 17 Planning Based on Model Checking Dana S. Nau University - PowerPoint PPT Presentation

Lecture slides for Automated Planning: Theory and Practice Chapter 17 Planning Based on Model Checking Dana S. Nau University of Maryland 1:19 PM February 29, 2012 Dana Nau: Lecture slides for Automated Planning 1 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Motivation c a b Intended ● Actions with multiple possible outcome c outcomes grasp(c) a b ◆ Action failures » e.g., gripper drops its load ◆ Exogenous events a b » e.g., road closed Unintended outcome ● Nondeterministic systems are like Markov Decision Processes (MDPs), but without probabilities attached to the outcomes ◆ Useful if accurate probabilities aren’t available, or if probability calculations would introduce inaccuracies Dana Nau: Lecture slides for Automated Planning 2 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Nondeterministic Systems ● Nondeterministic system : a triple Σ = ( S, A, γ ) ◆ S = finite set of states ◆ A = finite set of actions ◆ γ : S × A → 2 s ● Like in the previous chapter, the book doesn’t commit to any particular representation ◆ It only deals with the underlying semantics ◆ Draw the state-transition graph explicitly ● Like in the previous chapter, a policy is a function from states into actions ◆ π : S → A ● Notation: S π = { s | ( s,a ) ∈ π } ◆ In some algorithms, we’ll temporarily have nondeterministic policies » Ambiguous: multiple actions for some states ◆ π : S → 2 A , or equivalently, π ⊆ S × A ◆ We’ll always make these policies deterministic before the algorithm terminates Dana Nau: Lecture slides for Automated Planning 3 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Example 2 ● Robot r1 starts at location l1 Start ● Objective is to get r1 to location l4 Goal ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 4 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Example 2 ● Robot r1 starts at location l1 Start ● Objective is to get r1 to location l4 Goal ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 5 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Example 2 ● Robot r1 starts at location l1 Start ● Objective is to get r1 to location l4 Goal ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 6 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Execution Structures s5 2 s2 s3 ● Execution structure for a policy π : ◆ The graph of all of π ’s execution paths ● Notation: Σ π = ( Q , T ) Start ◆ Q ⊆ S s1 s4 ◆ T ⊆ S × S Goal ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 7 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Execution Structures s5 s2 s3 ● Execution structure for a policy π : ◆ The graph of all of π ’s execution paths ● Notation: Σ π = ( Q , T ) ◆ Q ⊆ S s1 s4 ◆ T ⊆ S × S ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 8 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Execution Structures s5 2 s2 s3 ● Execution structure for a policy π : ◆ The graph of all of π ’s execution paths ● Notation: Σ π = ( Q , T ) Start ◆ Q ⊆ S s1 s4 ◆ T ⊆ S × S Goal ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 9 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Execution Structures s5 s2 s3 ● Execution structure for a policy π : ◆ The graph of all of π ’s execution paths ● Notation: Σ π = ( Q , T ) ◆ Q ⊆ S s1 s4 ◆ T ⊆ S × S ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 10 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Execution Structures 2 ● Execution structure for a policy π : ◆ The graph of all of π ’s execution paths ● Notation: Σ π = ( Q , T ) Start ◆ Q ⊆ S s1 s4 ◆ T ⊆ S × S Goal ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 11 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Execution Structures ● Execution structure for a policy π : ◆ The graph of all of π ’s execution paths ● Notation: Σ π = ( Q , T ) ◆ Q ⊆ S s1 s4 ◆ T ⊆ S × S ● π 1 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)) } ● π 2 = { (s1, move(r1,l1,l2)), (s2, move(r1,l2,l3)), (s3, move(r1,l3,l4)), (s5, move(r1,l3,l4)) } ● π 3 = { (s1, move(r1,l1,l4)) } Dana Nau: Lecture slides for Automated Planning 12 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Types of Solutions ● Weak solution : at least one execution path reaches a goal s0 s3 a2 a0 s1 s2 Goal a1 ● Strong solution : every execution path reaches a goal s0 s3 Goal a2 a0 s1 s2 Goal a3 a1 ● Strong-cyclic solution : every fair execution path reaches a goal ◆ Don’t stay in a cycle forever if there’s a state-transition out of it a3 s0 s3 a2 a0 Goal s2 s1 a1 Dana Nau: Lecture slides for Automated Planning 13 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Finding Strong Solutions ● Backward breadth-first search ● StrongPreImg ( S ) = {( s,a ) : γ ( s,a ) ≠ ∅ , γ ( s,a ) ⊆ S } ◆ all state-action pairs for which all of the successors are in S ● PruneStates ( π , S ) = {( s,a ) ∈ π : s ∉ S } ◆ S is the set of states we’ve already solved ◆ keep only the state-action pairs for other states ● MkDet ( π ' ) ◆ π ' is a policy that may be nondeterministic ◆ remove some state-action pairs if necessary, to get a deterministic policy Dana Nau: Lecture slides for Automated Planning 14 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Example 2 π = failure π ' = ∅ S π ' = ∅ S g ∪ S π ' = { s4 } Start s4 Goal Dana Nau: Lecture slides for Automated Planning 15 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Example s5 2 π = failure s3 π ' = ∅ S π ' = ∅ S g ∪ S π ' = { s4 } π '' ← PreImage = Start {( s3,move(r1,l3,l4)), s4 (s5,move(r1,l5,l4)) } Goal Dana Nau: Lecture slides for Automated Planning 16 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Example s5 2 π = failure s3 π ' = ∅ S π ' = ∅ S g ∪ S π ' = { s4 } π '' ← PreImage = Start {( s3,move(r1,l3,l4)), s4 (s5,move(r1,l5,l4)) } Goal π ← π ' = ∅ π ' ← π ' U π '' = {( s3,move(r1,l3,l4)), (s5,move(r1,l5,l4)) } Dana Nau: Lecture slides for Automated Planning 17 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Example s5 π = ∅ 2 π ' = {( s3,move(r1,l3,l4)), s3 (s5,move(r1,l5,l4)) } S π ' = { s3,s5 } S g ∪ S π ' = { s3,s4,s5 } Start s4 Goal Dana Nau: Lecture slides for Automated Planning 18 Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/