Pattern-Database Heuristics for Partially Observable - PowerPoint PPT Presentation

Pattern-Database Heuristics for Partially Observable Nondeterministic Planning Albert-Ludwigs-Universität Freiburg Manuela Ortlieb and Robert Mattmüller Research Group Foundations of Artificial Intelligence Department of Computer Science University of Freiburg, Germany September 19th, 2013

WHAT: POND Planning Partially observable nondeter- Motivation B 0 ministic (POND) planning: WHAT: POND Planning Given: WHY: Advance Offline Planning HOW: Informed state variables Progression Search nondeterministic and Research sensing actions Question inital state description Empirical Approach goal description Conclusion September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 2 / 20

WHAT: POND Planning Partially observable nondeter- Motivation B 0 ministic (POND) planning: WHAT: POND Planning Given: WHY: Advance Offline Planning HOW: Informed state variables Progression Search nondeterministic and Research sensing actions Question inital state description Empirical Approach goal description Conclusion Wanted: mapping from belief states to actions to reach goal state � “strong cyclic plan” September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 2 / 20

WHY: Advance Offline Planning Motivation WHAT: POND Planning Goal: Model realistic features of planning tasks like WHY: Advance Offline Planning nondeterminism and HOW: Informed Progression partial observability Search Research Purpose: Question Generate complete plan offline. Empirical Approach Avoid replanning during plan execution. Conclusion Approach: Do not reinvent the wheel. Benefit from research on heuristics in classical planning. September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 3 / 20

HOW: Informed Progression Search B 0 Motivation WHAT: POND Planning WHY: Advance Offline Planning HOW: Informed Algorithmic approach: Progression Search Progression search in Research Question belief space Empirical Approach for a strong cyclic plan Conclusion guided by distance heuristic September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 4 / 20

Research Question Domain-independent distance heuristic for belief states? Motivation Option 1: “Simplify” B Research Question 1 Apply classical Evaluating Belief States s 2 planning heuristic to Pattern-Database s 5 Heuristics individual world states. Empirical Approach 2 Aggregate h -values s 3 Conclusion over belief state. � Pros and Cons: Aggregation: ✦ easy to do h B ( B ) = h ( s 2 )+ h ( s 3 )+ h ( s 5 ) or ✪ sampling unclear h B ( B ) = max { h ( s 2 ) , h ( s 3 ) , h ( s 5 ) } ✪ aggregation unclear or ... ✪ informativeness? September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 5 / 20

Research Question Domain-independent distance heuristic for belief states? Motivation Option 2: “Lift” Research B Question Lift definitions of Evaluating Belief States heuristics to POND Pattern-Database Heuristics setting and define Empirical Approach heuristic for belief Conclusion states directly. � Pros and Cons: Compute h B ( B ) “directly”. ✪ less straightforward ✦ no sampling issue ✦ no aggregation issue ✦ more informative? September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 6 / 20

Research Question Remark: Bryce et al. (2006): “lifted” computation of h ( B ) for RPG Motivation approach using labeled uncertainty graph (LUG). Research Question Showed superiority over a “simplifying” approach (sample, Evaluating Belief States compute h RPG , aggregate). Pattern-Database Heuristics Empirical This work: Approach Comparison of “lifted” and “simplifying” approach for Conclusion pattern-database heuristic. “lift” “simplify” ≻ RPG LUG SG [Bryce et al., 2006] PDB ? ≻ ≺ ∼ ? ? [this work] September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 7 / 20

Pattern-Database Heuristics Motivation Research Question Evaluating Belief States Pattern-Database Heuristics Empirical Π Approach Conclusion September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 8 / 20

Pattern-Database Heuristics apply an abstraction α Motivation Research Question α ( Π ) Evaluating Belief States Pattern-Database Heuristics Empirical Π Approach Conclusion September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 8 / 20

Pattern-Database Heuristics apply an abstraction α Motivation Research Question α ( Π ) Evaluating Belief States Pattern-Database Heuristics Empirical Π solve abstract Approach planning task α ( Π ) Conclusion September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 8 / 20

Pattern-Database Heuristics apply an abstraction α Motivation Research Question α ( Π ) Evaluating Belief States Pattern-Database Heuristics Empirical Π solve abstract Approach planning task α ( Π ) Conclusion store abstract costs in table (PDB) 4 5 1 0 1 3 4 2 2 September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 8 / 20

Pattern-Database Heuristics Motivation Research Question Evaluating Belief States Pattern-Database Heuristics use as heuristic Empirical Π when solving Π Approach Conclusion 4 5 1 0 1 3 4 2 2 September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 8 / 20

Pattern-Database Heuristics Full vs. partial observability: Motivation Full observability: Research Question abstract state space “only” exponential in pattern size Evaluating Belief States ⇒ larger patterns possible Pattern-Database Heuristics ✦ much of the state structure taken into account Empirical ✪ (un)observability not taken into account Approach Conclusion Partial observability: abstract state space doubly exponential in pattern size ⇒ only smaller patterns possible ✪ less of the state structure taken into account ✦ (un)observability taken into account Question: How to deal with this tradeoff? September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 9 / 20

Empirical Approach Motivation Question: Research In abstraction, should we assume Question Empirical full observability (option 1) or Approach Benchmark partial observability (option 2)? Domains Belief State Sampling In abstraction, should we assume Pattern Selection Internal Comparison deterministic actions (option 1) or External Comparison nondeterministic actions (option 2)? Conclusion Way to investigate this tradeoff: purely empirical September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 10 / 20

Empirical Approach Implementation and comparison of three variants of POND Motivation PDB heuristic: Research Question Deter- Abstract Abstract Sampling, Obser- Empirical Variant miniza- problem goal aggrega- Approach vability tion type distances tion? Benchmark Domains FO-Det Belief State Sampling (“simplify full yes classical optimistic yes Pattern Selection Internal everything”) Comparison External FO-NDet Comparison (“simplify full no FOND expected yes Conclusion observation”) PO-NDet (“simplify partial no POND expected no nothing”) September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 11 / 20

Implementation Details Motivation Strong cyclic POND planner using variant of LAO* Research Question [Hansen and Zilberstein, 2001] Empirical Approach Guided by FO-Det, FO-NDet and PO-NDet PDB heuristics Benchmark Domains Canonical heuristic function, iPDB [Haslum et al., 2007] Belief State Sampling Pattern Selection Symbolic BDD representation of belief states and Internal Comparison transitions External Comparison Sampling of world states from belief states uniformly with Conclusion replacement 4GB memory limit, 30 minute time limit per run September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 12 / 20

Benchmark Domains Motivation Research Question Empirical First-Responders adapted to requiring some active Approach Benchmark sensing Domains Belief State Blocksworld adapted to requiring some active sensing Sampling Pattern Selection Internal Canadian-Traveler-Problem without probabilities and Comparison External with unit edge costs Comparison Conclusion September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 13 / 20

Belief State Sampling and Aggregation For FO-(N)Det: Motivation Research How many world states to sample from belief states? Question � experiment with Empirical Approach 1 Benchmark Domains 5 Belief State Sampling 10 Pattern Selection Internal 15 Comparison External “all” Comparison Conclusion How to aggregate values? � experiment with maximization summation September 19th, 2013 M. Ortlieb, R. Mattmüller – PDB Heuristics for POND Planning 14 / 20