Masters Thesis: Heuristic Search Under a Deadline Austin Dionne - PowerPoint PPT Presentation

Master’s Thesis: Heuristic Search Under a Deadline Austin Dionne Department of Computer Science austin.dionne at gmail.com Austin Dionne Heuristic Search Under Deadlines – 1 / 56

Acknowledgements Thanks to: Introduction Related Work ■ Wheeler Ruml (Advisor) DAS Conclusion ■ Jordan T. Thayer (Collaborator) DDT ■ NSF (grant IIS-0812141) ■ DARPA CSSG program (grant N10AP20029) Austin Dionne Heuristic Search Under Deadlines – 2 / 56

Introduction ■ Heuristic Search ■ Problem Def. ■ Thesis Statement ■ Contributions Related Work DAS Conclusion Introduction DDT Austin Dionne Heuristic Search Under Deadlines – 3 / 56

Search Is Awesome! Introduction ■ Heuristic Search ■ Problem Def. ■ Thesis Statement ■ Contributions Related Work DAS Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 4 / 56

Heuristic Search Introduction ■ Heuristic Search ■ Problem Def. ■ Thesis Statement ■ Contributions Related Work DAS Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 5 / 56

Heuristic Search (Continued) s 0 : starting state Introduction expand ( s ) : returns list of child states ( s c , c ) ■ Heuristic Search goal ( s ) : returns true if s is a goal state, false otherwise ■ Problem Def. g ( s ) : cost accumulated so far on path from s 0 to s ■ Thesis Statement h ∗ ( s ) : cost of cheapest solution under s ■ Contributions f ∗ ( s ) = g ( s ) + h ∗ ( s ) : estimated cost of best solution under s Related Work d ∗ ( s ) : number of steps to cheapest solution under s DAS h ( s ) , f ( s ) , d ( s ) : heuristic estimators of true values � Conclusion d ( s ) : unbiased estimator of d ∗ DDT Austin Dionne Heuristic Search Under Deadlines – 6 / 56

Problem Definition Introduction Given a problem and a limited amount of computation time , ■ Heuristic Search ■ Problem Def. find the best solution possible before the deadline. ■ Thesis Statement ■ Contributions Related Work ■ Problem which often occurs in practice DAS Conclusion ■ The current “best” methods do not directly consider the DDT presence of a deadline and waste effort. ■ The current “best” methods require off-line tuning for optimal performance. Austin Dionne Heuristic Search Under Deadlines – 7 / 56

Thesis Statement Introduction ■ Heuristic Search ■ Problem Def. ■ Thesis Statement ■ Contributions My thesis is that a deadline-cognizant approach which attempts Related Work to expend all available search effort towards a single final DAS solution has the potential for outperforming these methods Conclusion without off-line optimization. DDT Austin Dionne Heuristic Search Under Deadlines – 8 / 56

Contributions In this thesis we have proposed: Introduction ■ Heuristic Search ■ Problem Def. ■ Corrected single-step error model for � d ( s ) and � h ( s ) ■ Thesis Statement ■ Contributions ■ Deadline Aware Search (DAS) which can outperform Related Work current approaches DAS Conclusion ■ Extended single-step error model for calculating d ∗ and h ∗ DDT distributions on-line ■ Deadline Decision Theoretic Search (DDT) which is a more flexible and theoretically based algorithm that holds some promise Austin Dionne Heuristic Search Under Deadlines – 9 / 56

Introduction Related Work ■ Related Work ■ Related Work (Continued) ■ Related Work (Continued) ■ Current Approach ■ Our Motivation ■ Recap Related Work DAS Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 10 / 56

Related Work We are not the first to attempt to solve this problem... Introduction Related Work ■ Related Work ■ Related Work ■ Time Constrained Search (Hiraishi, Ohwada, and (Continued) ■ Related Work Mizoguchi 1998) (Continued) ■ Current Approach ■ Our Motivation ■ Contract Search (Aine, Chakrabarti, and Kumar 2010) ■ Recap DAS Conclusion Neither of these methods work well in practice! DDT Austin Dionne Heuristic Search Under Deadlines – 11 / 56

Related Work (Continued) Problem with Time Constrained Search: Introduction Related Work ■ Parameters abound! ( ǫ upper , ǫ lower , ∆w ) ■ Related Work ■ Related Work (Continued) ■ Important questions without answers: ■ Related Work (Continued) ■ Current Approach ◆ When (if ever) should we resort open list? ■ Our Motivation ■ Recap ◆ Is a hysteresis necessary for changes in w ? DAS Conclusion I could not implement a version of this algorithm that worked DDT well! Austin Dionne Heuristic Search Under Deadlines – 12 / 56

Related Work (Continued) Problem with Contract Search: Introduction Related Work ■ Not really applicable to domains with goals at a wide range ■ Related Work ■ Related Work of depths (tiles/gridworld/robots) (Continued) ■ Related Work (Continued) ■ Takes substantial off-line effort to prepare the algorithm ■ Current Approach ■ Our Motivation for a particular domain and deadline ■ Recap DAS Jordan Thayer implemented this algorithm and it does not work Conclusion well! DDT Austin Dionne Heuristic Search Under Deadlines – 13 / 56

Currently Accepted Approach Anytime Search Introduction Related Work ■ Search for a suboptimal initial solution relatively quickly ■ Related Work ■ Related Work ■ Continue searching, finding sequence of improved solutions over (Continued) ■ Related Work time (Continued) ■ Current Approach ■ Eventually converge to optimal ■ Our Motivation ■ Recap Problems: DAS Conclusion 1. Wasted effort in finding sequence of mostly unused solutions DDT 2. Based on bounded suboptimal search, which requires parameter settings ■ May not have time for off-line tuning ■ For some domains different deadlines require different settings Austin Dionne Heuristic Search Under Deadlines – 14 / 56

Our Motivation Our desired deadline-aware approach should: Introduction Related Work ■ Consider the time remaining in ordering state expansion ■ Related Work ■ Related Work (Continued) ■ Perform consistently well across a full range deadlines ■ Related Work (Continued) (fractions of a second to minutes) ■ Current Approach ■ Our Motivation ■ Recap ■ Be parameterless and general DAS Conclusion ■ Not require significant off-line computation DDT Austin Dionne Heuristic Search Under Deadlines – 15 / 56

Recap ■ Search under deadlines is a difficult and important problem Introduction Related Work ■ Related Work ■ Previously proposed approaches don’t work ■ Related Work (Continued) ■ Related Work ■ Currently used approaches are unsatisfying (Continued) ■ Current Approach ■ We propose an algorithm (DAS) which can outperform ■ Our Motivation ■ Recap these methods without the use of off-line tuning DAS Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 16 / 56

Introduction Related Work DAS ■ Motivation ■ Algorithm (1) ■ Vacillation ■ Exp Delay ■ Calc d m ax ■ Algorithm (2) Deadline Aware Search (DAS) ■ Results ■ Results ■ results ■ Conclusion Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 17 / 56

Motivation DAS pursues the best solution path which is reachable within Introduction the time remaining in the search. Related Work DAS ■ Motivation ■ Best is defined as minimal f ( s ) ■ Algorithm (1) ■ Vacillation ■ Reachability is a function of an estimate distance to a ■ Exp Delay solution � ■ Calc d m ax d ( s ) and the current behavior of the search ■ Algorithm (2) ■ Results ■ Results ■ results ■ Conclusion Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 18 / 56

DAS: High-Level Algorithm While there is time remaining before the deadline: Introduction Related Work ■ Calculate maximum allowable distance d m ax DAS ■ Motivation ■ Select node n from open list with minimal f ( n ) ■ Algorithm (1) ■ Vacillation ■ Exp Delay ■ If � d ( n ) ≤ d m ax (solution is reachable) ■ Calc d m ax ■ Algorithm (2) ■ Results ◆ Expand n , add children to open list ■ Results ■ results ■ Conclusion ■ Otherwise (solution is unreachable) Conclusion ◆ Add n to pruned list DDT Austin Dionne Heuristic Search Under Deadlines – 19 / 56

Search Vacillation Error in h ( s ) produces Search Vacillation . Introduction Related Work DAS ■ Motivation ■ Algorithm (1) ■ Vacillation ■ Exp Delay ■ Calc d m ax ■ Algorithm (2) ■ Results ■ Results ■ results ■ Conclusion Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 20 / 56

Expansion Delay Expansion Delay Introduction Related Work Maintain a running expansion counter during search. DAS ■ Motivation ■ Algorithm (1) At state expansion, define expansion delay as: ■ Vacillation ■ Exp Delay ∆e = ( current exp counter ) − ( exp counter at generation ) ■ Calc d m ax ■ Algorithm (2) ■ Results ■ Results ■ results ■ Conclusion Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 21 / 56

Expansion Delay Use mean expansion delay ∆e to calculate d m ax : Introduction Related Work d m ax = ( expansions remaining ) DAS (1) ■ Motivation ∆e ■ Algorithm (1) ■ Vacillation ■ Exp Delay ■ Calc d m ax ■ Algorithm (2) ■ Results d m ax estimates the expected number of steps that will be ■ Results explored down any particular path in the search space. ■ results ■ Conclusion Conclusion DDT Austin Dionne Heuristic Search Under Deadlines – 22 / 56