Combinatorial Search Algorithms as Rational Agents Wheeler Ruml - PowerPoint PPT Presentation

Combinatorial Search Algorithms as Rational Agents Wheeler Ruml Palo Alto Research Center ruml@parc.com Wheeler Ruml (PARC) Learning to Search Trees – 2 / 40

Motivation Research goal: “What algorithm to run?” Introduction ➢ Motivation ➢ Combinatorial fundamental properties of various algorithms ■ Optimization fundamental properties of problems ➢ Constraint ■ Satisfaction ➢ Types of Search Problems ➢ The Problem ➢ The Central Idea How to best use available information in a tree search? Previous Approaches Basic BLFS BLFS with Learning Wheeler Ruml (PARC) Learning to Search Trees – 3 / 40

Combinatorial Optimization Given: set of variables Introduction ➢ Motivation possible values for each variable ➢ Combinatorial Optimization objective function over assignments ➢ Constraint Find: assignment that minimizes objective function Satisfaction ➢ Types of Search Problems ➢ The Problem One approach: search tree for best leaf ➢ The Central Idea Previous Approaches variable 1 Basic BLFS BLFS with Learning value 1 value 2 variable 2 variable 2 value 1 value 2 value 1 value 2 1.6 2.3 1.5 3.9 Wheeler Ruml (PARC) Learning to Search Trees – 4 / 40

Constraint Satisfaction Given: set of variables Introduction ➢ Motivation possible values for each variable ➢ Combinatorial Optimization set of constraints between variables ➢ Constraint Find: complete and feasible assignment Satisfaction ➢ Types of Search Problems ➢ The Problem Treat as combinatorial optimization: ➢ The Central Idea Previous Approaches variable 1 Basic BLFS BLFS with Learning value 1 value 2 variable 2 variable 2 value 1 value 2 value 1 value 2 1 3 0 4 Wheeler Ruml (PARC) Learning to Search Trees – 5 / 40

Types of Search Problems Shortest path: find shallowest node that is a goal Introduction eg, shortest plan ➢ Motivation ➢ Combinatorial Constraint satisfaction: find any leaf node that is a goal Optimization ➢ Constraint eg, valid configuration Satisfaction ➢ Types of Search Combinatorial optimization: find best-scoring leaf node Problems ➢ The Problem eg, balanced partitioning ➢ The Central Idea Adversarial search: find best-scoring leaf we can surely reach Previous Approaches eg, chess Basic BLFS BLFS with Learning Wheeler Ruml (PARC) Learning to Search Trees – 6 / 40

Types of Search Problems Shortest path: find shallowest node that is a goal Introduction eg, shortest plan ➢ Motivation ➢ Combinatorial Optimization ➢ Constraint Satisfaction ➢ Types of Search Combinatorial optimization: find best-scoring leaf node Problems ➢ The Problem eg, balanced partitioning ➢ The Central Idea Adversarial search: find best-scoring leaf we can surely reach Previous Approaches eg, chess Basic BLFS BLFS with Learning Wheeler Ruml (PARC) Learning to Search Trees – 6 / 40

Types of Search Problems Shortest path: find shallowest node that is a goal Introduction eg, shortest plan ➢ Motivation ➢ Combinatorial Optimization ➢ Constraint Satisfaction ➢ Types of Search Problems ➢ The Problem ➢ The Central Idea Adversarial search: find best-scoring leaf we can surely reach Previous Approaches eg, chess Basic BLFS BLFS with Learning Wheeler Ruml (PARC) Learning to Search Trees – 6 / 40

The Problem For large problems or when optimum is recognizable, Introduction search order matters. ➢ Motivation ➢ Combinatorial Optimization ➢ Constraint Satisfaction ➢ Types of Search Where was the mistake? Problems ➢ The Problem ➢ The Central Idea Previous Approaches Basic BLFS BLFS with Learning Truncated depth-first is not necessarily optimal! Wheeler Ruml (PARC) Learning to Search Trees – 7 / 40

The Central Idea Where to backtrack first? Introduction ➢ Motivation ➢ Combinatorial Optimization ➢ Constraint Satisfaction ➢ Types of Search Problems ➢ The Problem ➢ The Central Idea Previous Approaches Predetermined order = strong assumptions = ad hoc = brittle Basic BLFS BLFS with Learning Use a model of leaf costs on-line to guide search. [Ruml, 2001; Boyan, 1998; Baluja, 1996] Wheeler Ruml (PARC) Learning to Search Trees – 8 / 40

Introduction Previous Approaches ➢ DFS ➢ Discrepancy Search ➢ A Best-First Approach ➢ Predicting Leaf Cost ➢ Avoid Previous Approaches Bookkeeping ➢ BLFS Basic BLFS BLFS with Learning Wheeler Ruml (PARC) Learning to Search Trees – 9 / 40

Depth-First Search (DFS) Introduction Previous Approaches ➢ DFS ➢ Discrepancy Search ➢ A Best-First Approach ➢ Predicting Leaf Cost ➢ Avoid Bookkeeping ➢ BLFS Basic BLFS BLFS with Learning 1. Prune provably bad nodes (branch and bound) 2. Sort children left to right using a heuristic ordering function h Assumes penalty at top is enormous. Wheeler Ruml (PARC) Learning to Search Trees – 10 / 40

Depth-First Search (DFS) Introduction Previous Approaches ➢ DFS ➢ Discrepancy Search ➢ A Best-First Approach ➢ Predicting Leaf Cost ➢ Avoid Bookkeeping ➢ BLFS Basic BLFS BLFS with Learning 1. Prune provably bad nodes (branch and bound) 2. Sort children left to right using a heuristic ordering function h Assumes penalty at top is enormous. Wheeler Ruml (PARC) Learning to Search Trees – 10 / 40

Discrepancy Search Harvey and Ginsberg (1995): Limited Discrepancy Search Introduction discrepancy : a choice against the heuristic ordering Previous Approaches ➢ DFS Explore all paths with k discrepancies before any with k + 1 . ➢ Discrepancy Search ➢ A Best-First Approach ➢ Predicting Leaf Cost ➢ Avoid Bookkeeping ➢ BLFS Basic BLFS BLFS with Learning Korf (1996): ILDS Also Walsh (1997), Ginsberg and Harvey (1992), Meseguer (1997) Wheeler Ruml (PARC) Learning to Search Trees – 11 / 40

A Best-First Approach Fixed order ↔ fixed predictions for leaf costs Introduction Want predicted costs to match current problem Previous Approaches ➢ DFS ➢ Discrepancy Use run-time heuristic information to help make predictions. Search ➢ A Best-First Approach ➢ Predicting Leaf Use predictions to guide search: Cost ➢ Avoid Bookkeeping Rational order: increasing predicted leaf cost = best-first ➢ BLFS Basic BLFS BLFS with Learning 1.6 2.3 2.1 3.9 1.5 2.6 3.2 4.4 [Ruml, 2002 Wheeler Ruml (PARC) Learning to Search Trees – 12 / 40

Predicting Leaf Cost Want to visit leaves in increasing order of predicted cost. Introduction Previous Approaches Where are they? ➢ DFS ➢ Discrepancy Search f ( n ) = predicted cost of best leaf at or below n ■ ➢ A Best-First Approach can use any info at n or on path from root ■ ➢ Predicting Leaf Cost want f ( n ) consistent ■ ➢ Avoid Bookkeeping ➢ BLFS Basic BLFS BLFS with Learning f ( n ) = 1 . 5 f ( n ) = 1 . 7 f ( n ) = 1 . 5 f ( n ) = 2 . 2 f ( n ) = 3 . 1 f ( n ) = 1 . 5 f ( n ) = 4 . 8 1.6 2.3 2.1 3.9 1.5 2.6 3.2 4.4 Wheeler Ruml (PARC) Learning to Search Trees – 13 / 40

Avoid Bookkeeping Want to visit leaves in increasing order of predicted cost. Introduction Previous Approaches How to keep track of them? ➢ DFS ➢ Discrepancy Search don’t — allow slight misordering ■ ➢ A Best-First Approach use iteratively increasing cost bound ■ ➢ Predicting Leaf Cost ➢ Avoid Bookkeeping Cost bound = 2 ➢ BLFS Basic BLFS BLFS with Learning f ( n ) = 1 . 5 f ( n ) = 1 . 7 f ( n ) = 1 . 5 f ( n ) = 2 . 2 f ( n ) = 3 . 1 f ( n ) = 1 . 5 f ( n ) = 4 . 8 1.6 2.3 2.1 3.9 1.5 2.6 3.2 4.4 Wheeler Ruml (PARC) Learning to Search Trees – 14 / 40

Avoid Bookkeeping Want to visit leaves in increasing order of predicted cost. Introduction Previous Approaches How to keep track of them? ➢ DFS ➢ Discrepancy Search don’t — allow slight misordering ■ ➢ A Best-First Approach use iteratively increasing cost bound ■ ➢ Predicting Leaf Cost ➢ Avoid Bookkeeping Cost bound = 3 ➢ BLFS Basic BLFS BLFS with Learning f ( n ) = 1 . 5 f ( n ) = 1 . 7 f ( n ) = 1 . 5 f ( n ) = 2 . 2 f ( n ) = 3 . 1 f ( n ) = 1 . 5 f ( n ) = 4 . 8 1.6 2.3 2.1 3.9 1.5 2.6 3.2 4.4 Wheeler Ruml (PARC) Learning to Search Trees – 14 / 40

Best-Leaf-First Search (BLFS) BLFS ( root ) Introduction Previous Approaches Visit a few leaves ➢ DFS ➢ Discrepancy Nodes-desired ← number of nodes visited so far Search Loop until time runs out: ➢ A Best-First Approach Double nodes-desired ➢ Predicting Leaf Cost Estimate cost bound that visits nodes-desired nodes ➢ Avoid Bookkeeping BLFS-expand( root , bound ) ➢ BLFS Basic BLFS BLFS with Learning BLFS-expand ( node , bound ) If leaf( node ), visit( node ) else, for each child of node : If best-completion( child ) ≤ bound BLFS-expand( child , bound ) Wheeler Ruml (PARC) Learning to Search Trees – 15 / 40

Introduction Previous Approaches Basic BLFS ➢ Indecision Search ➢ Choosing the Cost Bound ➢ Best-Leaf-First Search (BLFS) ➢ Test Domains Basic BLFS ➢ Latin Squares ➢ Random Binary CSPs BLFS with Learning Wheeler Ruml (PARC) Learning to Search Trees – 16 / 40