angelic hierarchical planning
play

Angelic Hierarchical Planning: Optimal and Online Algorithms - PowerPoint PPT Presentation

Angelic Hierarchical Planning: Optimal and Online Algorithms Bhaskara Marthi Stuart Russell Jason Wolfe MIT/Willow Garage UC Berkeley UC Berkeley bhaskara@csail.mit.edu russell@cs.berkeley.edu jawolfe@cs.berkeley.edu ICAPS 08 1


  1. Deterministic Planning Problems • Here, a planning problem = • State space S • Initial state s 0 , terminal set G • Primitive action set • Transition function: S × A → S • Cost function : S × A → R ∪ { ∞ } S s 0 -5 4 ∞ 8 G ∞ 3 Transitions & costs for action a 1 7

  2. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A T1 T2 T3 T4 8

  3. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 T1 T2 T3 T4 8

  4. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  5. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  6. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  7. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  8. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  9. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  10. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  11. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  12. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  13. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  14. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  15. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  16. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row B A • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  17. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  18. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  19. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  20. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  21. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  22. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  23. Running Example: Warehouse World Domain • Elaborated Blocks World with C discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  24. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds C • Can’t pass through blocks • Can only turn at top row A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  25. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row C A B • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  26. Running Example: Warehouse World Domain • Elaborated Blocks World with discrete spatial constraints • Gripper must stay in bounds • Can’t pass through blocks • Can only turn at top row A B C • All actions have cost 1 • Goal: have C on T4 T1 T2 T3 T4 • Can’t just move directly L, D, GetR, U, Turn, D, PutL, • Final plan has 22 steps R, R, D, GetL, L, PutL, U, L, GetL, U, Turn, R, D, D, PutR 8

  27. Running Example: Warehouse World HLAs L D GetR U Turn D PutL 9

  28. Running Example: Warehouse World HLAs Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9

  29. Running Example: Warehouse World HLAs NavT( 2,3 ) NavT( 2,3 ) Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9

  30. Running Example: Warehouse World HLAs Move( C,A ) NavT( 2,3 ) NavT( 2,3 ) Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) L D GetR U Turn D PutL 9

  31. Running Example: Warehouse World HLAs Act ... Move( C,A ) ... NavT( 2,3 ) NavT( 2,3 ) ... Nav( 2,3 ) Nav( 3,3 ) Nav( 2,3 ) ... L D GetR U Turn D PutL 9

  32. Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act [Act] 10

  33. Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] 10

  34. Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] ... [NavT( left of B ), GetR, NavT( left of target ), PutR] ... 10

  35. Running Example: Warehouse World HLAs • Plans of interest are primitive refinements of special HLA Act • Each HLA has a set of immediate [Act] refinements into action sequences ... iff at G [Move( B,C ), Act] [ ] ... [NavT( left of B ), GetR, NavT( left of target ), PutR] ... ... ... 10

  36. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) 11

  37. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans Act 11

  38. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act 11

  39. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act 11

  40. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act 11

  41. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11

  42. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11

  43. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Move( C,A ) Act Act Move( A,C ) Act 11

  44. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Act Move( A,C ) Act 11

  45. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11

  46. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11

  47. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act Move( B,C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11

  48. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) Act ) C , B ( e v o M . . . Act Move( B,C ) Nav( x C -1,y C ) Move( C,A ) Act Move( C,B ) Act Act Move( A,C ) Act 11

  49. Abstract Lookahead Trees (ALTs) • ALTs generalize lookahead trees for flat algs (e.g., A*) • Represent a set of potential plans • Basic operation: refine a plan (replace with all refs. at some HLA) • Nodes have optimistic & pessimistic valuations 6/6 8/8 3/3 Act ) C , B ( e v o M 1/1 . . . 5/7 7/9 Act Move( B,C ) Nav( x C -1,y C ) 2/4 Move( C,A ) Act 4/8 9/ ∞ 0/0 Move( C,B ) Act Act 3/ ∞ 8/ ∞ Move( A,C ) Act 11

  50. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy NavT( 0,1 ) 12

  51. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • NavT( 0,1 ) 6 8 10 ... ... 7 5 12

  52. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • NavT( 0,1 ) 6 � � 6 � � � 5 5 � 12

  53. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • Exact description of h = valuation of h from each s • NavT( 0,1 ) � � � � � � 6 � 6 � 6 � � � � � ... � � 5 � 5 � 5 � 12

  54. Modeling HLAs • An HLA is fully characterized by planning problem + hierarchy But without abstraction, lose benefits of hierarchy • • Extension of idea from “Angelic Semantics for HLAs” [MRW ‘07]: Valuation of HLA h from state s: • For each s’, min cost of any primitive refinement of h that takes s to s’ • Exact description of h = valuation of h from each s • But this description has no compact, efficient representation in general • NavT( 0,1 ) � � � � � � 6 � 6 � 6 � � � � � ... � � 5 � 5 � 5 � 12

  55. Optimistic and Pessimistic Valuations • Instead, use approximate valuations � � Exact 6 4 3 1 5 � 13

  56. Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set � � Exact 6 4 3 1 5 � 13

  57. Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set • Optimistic valuations never overestimate best achievable cost 1 Optimistic � � Exact 6 4 3 1 5 � 13

  58. Optimistic and Pessimistic Valuations • Instead, use approximate valuations • We choose a simple form: reachable set + cost bound on set • Optimistic valuations never overestimate best achievable cost • Pessimistic valuations never underestimate best achievable cost 1 / 4 Optimistic � � Exact 6 4 Pessimistic 3 1 5 � 13

  59. Representing Descriptions: NCSTRIPS 14

  60. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA 14

  61. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA NavT( x t ,y t ) (Pre: At( x s ,y s )) 14

  62. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | 14

  63. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 14

  64. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 x ELSE: s t nil 14

  65. Representing Descriptions: NCSTRIPS • Descriptions specify propositions (possibly) added/deleted by HLA • Also include a cost bound • Can condition on features of initial state • An simple algorithm progresses a valuation (DNF + #) through an NCSTRIPS description to produce next valuation NavT( x t ,y t ) (Pre: At( x s ,y s )) ~ ~ Opt: -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR s t cost ≥ | x s - x t | + | y s - y t | Pess: IF Free( x t ,y t ) ∧ ∀ x Free( x,y max ) : s ~ ~ -At( x s ,y s ), +At( x t ,y t ), -FaceR, +FaceR t cost ≤ | x s - x t | + 2 y max - y t - y s + 1 x ELSE: s t nil 14

  66. Angelic Hierarchical A* (AHA*) • Construct an ALT with the single plan [Act] • Loop • Select a plan with minimal optimistic cost to G • If primitive, return it • Otherwise, refine one of its HLAs • Prune dominated refinements 15

  67. AHA*: Intuitive Picture G s 0 Act highest-level primitive 16

  68. AHA*: Intuitive Picture G s 0 Act highest-level primitive 17

Recommend


More recommend