Deterministic Planning Problems • Here, a planning problem = • State space S • Initial state s 0 , terminal set G • Primitive action set • Transition function: S × A → S • Cost function : S × A → R ∪ { ∞ } S s 0 -5 4 ∞ 8 G ∞ 3 Transitions & costs for action a 1 7
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A T1 T2 T3 T4 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 T1 T2 T3 T4 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row C A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B C • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8
Running Example: Warehouse World HLAs L D GetR U Turn D PutL 9
Running Example: Warehouse World HLAs Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9
Running Example: Warehouse World HLAs NavT( 2,3 ) NavT( 2,3 ) Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9
Running Example: Warehouse World HLAs Move( C,A ) NavT( 2,3 ) NavT( 2,3 ) Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9
Running Example: Warehouse World HLAs Act ... Move( C,A ) ... NavT( 2,3 ) NavT( 2,3 ) ... Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) ... L D GetR U Turn D PutL 9
Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act [Act] 10
Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] 10
Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] ... [NavT( left of B ), GetR, NavT( left of target ), PutR] ... 10
Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] ... [NavT( left of B ), GetR, NavT( left of target ), PutR] ... ... ... 10
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Act Move( A,C ) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act ) C , B ( e v o M . . . Act Move( B,C ) Nav( x C -1,y C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11
Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) • Nodes have optimistic & pessimistic valuations 6/6 8/8 3/3 Act ) C , B ( e v o M 1/1 . . . 5/7 7/9 Act Move( B,C ) Nav( x C -1,y C ) 2/4 Move( C,A ) Act 4/8 9/ ∞ 0/0 Move( C,B ) Act Act 3/ ∞ 8/ ∞ Move( A,C ) Act 11
Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy NavT( 0,1 ) 12
Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • NavT( 0,1 ) 6 8 10 ... ... 7 5 12
Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • NavT( 0,1 ) 6 � � 6 � � � 5 5 � 12
Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • Exact description of h = valuation of h from each s • NavT( 0,1 ) � � � � � � 6 � 6 � 6 � � � � � ... � � 5 � 5 � 5 � 12
Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • Exact description of h = valuation of h from each s • But this description has no compact, efficient representation in general • NavT( 0,1 ) � � � � � � 6 � 6 � 6 � � � � � ... � � 5 � 5 � 5 � 12
Optimistic and Pessimistic Valuations • Instead, use approximate valuations � � Exact 6 4 3 1 5 � 13
Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set � � Exact 6 4 3 1 5 � 13
Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set • Optimistic valuations never overestimate best achievable cost 1 Optimistic � � Exact 6 4 3 1 5 � 13
Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set • Optimistic valuations never overestimate best achievable cost • Pessimistic valuations never underestimate best achievable cost 1 / 4 Optimistic � � Exact 6 4 Pessimistic 3 1 5 � 13
Representing Descriptions: NCSTRIPS 14
Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA 14
Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA NavT( x t ,y t ) (Pre: At( x s ,y s )) 14
Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | 14
Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 14
Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 x ELSE: s t nil 14
Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state • An simple algorithm progresses a valuation (DNF + #) through an NCSTRIPS description to produce next valuation NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 x ELSE: s t nil 14
Angelic Hierarchical A* (AHA*) • Construct an ALT with the single plan [Act] • Loop • Select a plan with minimal optimistic cost to G • If primitive, return it • Otherwise, refine one of its HLAs • Prune dominated refinements 15
AHA*: Intuitive Picture G s 0 Act highest-level primitive 16
AHA*: Intuitive Picture G s 0 Act highest-level primitive 17
Recommend
More recommend