CS 758/858: Algorithms http://www.cs.unh.edu/~ruml/cs758 - PowerPoint PPT Presentation

CS 758/858: Algorithms http://www.cs.unh.edu/~ruml/cs758 Backtracking Local Search Wheeler Ruml (UNH) Class 25, CS 758 – 1 / 20

Backtracking ■ Hardness ■ Optimization ■ Backtracking ■ Depth-first Search ■ DFS Order ■ Problems ■ ILDS ■ ILDS Order Backtracking ■ Break Local Search Wheeler Ruml (UNH) Class 25, CS 758 – 2 / 20

Hardness NPC: SAT, vertex cover, clique, subset sum, . . . Backtracking ■ Hardness ■ Optimization greedy: local choice is optimal ■ Backtracking ■ Depth-first Search DP: poly number of options to track ■ DFS Order search: exponential number of options, often combinations ■ Problems ■ ILDS ■ ILDS Order ■ Break Local Search Wheeler Ruml (UNH) Class 25, CS 758 – 3 / 20

Combinatorial Optimization decision 1 Backtracking ■ Hardness ■ Optimization option 1 option 2 ■ Backtracking ■ Depth-first Search ■ DFS Order decision 2 decision 2 ■ Problems ■ ILDS option 1 option 2 option 1 option 2 ■ ILDS Order ■ Break Local Search (1,1) (1,2) (2,1) (2,2) A tree representation of alternatives in a small combinatorial problem. Wheeler Ruml (UNH) Class 25, CS 758 – 4 / 20

Backtracking depth-first search Backtracking child ordering ■ Hardness ■ Optimization lower bounds ■ Backtracking ■ Depth-first Search branch-and-bound ■ DFS Order duplicate detection: transposition table ■ Problems ■ ILDS ■ ILDS Order ■ Break Local Search Wheeler Ruml (UNH) Class 25, CS 758 – 5 / 20

Depth-first Search DFS ( node ) Backtracking ■ Hardness 1 If is-leaf( node ) ■ Optimization ■ Backtracking 2 Visit( node ) ■ Depth-first Search 3 else ■ DFS Order ■ Problems 4 For i from 0 to num-children ■ ILDS 5 DFS(child( node , i )) ■ ILDS Order ■ Break Local Search Wheeler Ruml (UNH) Class 25, CS 758 – 6 / 20

Depth-first Search Order Backtracking ■ Hardness ■ Optimization ■ Backtracking ■ Depth-first Search ■ DFS Order ■ Problems ■ ILDS ■ ILDS Order ■ Break Local Search Wheeler Ruml (UNH) Class 25, CS 758 – 7 / 20

Problems Are Hard Backtracking ■ Hardness ■ Optimization ■ Backtracking ■ Depth-first Search ■ DFS Order ■ Problems ■ ILDS ■ ILDS Order ■ Break Local Search 13,509 US cities (W. Cook) Wheeler Ruml (UNH) Class 25, CS 758 – 8 / 20

Problems Are Hard Backtracking ■ Hardness ■ Optimization ■ Backtracking . . . 13,508 ■ Depth-first Search ■ DFS Order ■ Problems ■ ILDS . . . 13,507 ■ ILDS Order ■ Break Local Search . . . 13,506 . . . 13,505 Wheeler Ruml (UNH) Class 25, CS 758 – 9 / 20

Problems Are Hard Backtracking ■ Hardness ■ Optimization ■ Backtracking ■ Depth-first Search ■ DFS Order ■ Problems ■ ILDS ■ ILDS Order ■ Break Local Search (S. LaValle) Wheeler Ruml (UNH) Class 25, CS 758 – 10 / 20

Improved Discrepancy Search ILDS ( node , allowance , remaining ) Backtracking ■ Hardness 1 If is-leaf( node ) ■ Optimization ■ Backtracking 2 Visit( node ) ■ Depth-first Search 3 else ■ DFS Order ■ Problems 4 If allowance > 0 ■ ILDS 5 ILDS(child( node , 1), allowance − 1 , remaining − 1 ) ■ ILDS Order ■ Break 6 If remaining > allowance Local Search 7 ILDS(child( node , 0), allowance , remaining − 1 ) start with ILDS(root, iteration , max-depth ) Wheeler Ruml (UNH) Class 25, CS 758 – 11 / 20

Discrepancy Search Order Backtracking ■ Hardness ■ Optimization ■ Backtracking ■ Depth-first Search ■ DFS Order ■ Problems ■ ILDS ■ ILDS Order ■ Break Local Search The second pass of ILDS visits all leaves with one discrepancy in their path from the root. Wheeler Ruml (UNH) Class 25, CS 758 – 12 / 20

Break asst 14 ■ Backtracking recitation: last year’s final ■ Hardness ■ ■ Optimization final exam: Wed Dec 12, 3:30-5:30pm, N101 ■ ■ Backtracking ■ Depth-first Search ■ DFS Order ■ Problems ■ ILDS ■ ILDS Order ■ Break Local Search Wheeler Ruml (UNH) Class 25, CS 758 – 13 / 20

Backtracking Local Search ■ Local Search ■ Local Search ■ Max Cut ■ Suboptimality ■ EOLQs Local Search Wheeler Ruml (UNH) Class 25, CS 758 – 14 / 20

Local Search 2.6 4.4 Backtracking 2.3 Local Search ■ Local Search 3.9 ■ Local Search ■ Max Cut ■ Suboptimality 1.5 ■ EOLQs 6.2 1.6 2.1 A graph representing an improvement-based search. Wheeler Ruml (UNH) Class 25, CS 758 – 15 / 20

Local Search hill climbing Backtracking simulated annealing Local Search ■ Local Search large neighborhood search ■ Local Search genetic algorithms ■ Max Cut ■ Suboptimality particle swarm optimization ■ EOLQs Wheeler Ruml (UNH) Class 25, CS 758 – 16 / 20

Max Cut maximize weight of edges crossing the cut w ( A, B ) Backtracking Local Search decision version is NP-complete ■ Local Search ■ Local Search ■ Max Cut simple local search: ■ Suboptimality ■ EOLQs move vertex u from A to B iff � � w uv > w uv v � = u ∈ A v ∈ B it’s possible to bound suboptimality of local minima under this neighborhood! Wheeler Ruml (UNH) Class 25, CS 758 – 17 / 20

Suboptimality of Local Search for any u in A , Backtracking � � w uv ≤ w uv Local Search ■ Local Search v � = u ∈ A v ∈ B ■ Local Search ■ Max Cut summing over all u in A , ■ Suboptimality ■ EOLQs � � 2 w uv ≤ w uv = w ( A, B ) u ∈ A,v ∈ B ( u,v ) ∈ A same from perspective of B : � � 2 w uv ≤ w uv = w ( A, B ) u ∈ A,v ∈ B ( u,v ) ∈ B add: � � 2 w uv + 2 w uv ≤ 2 w ( A, B ) ( u,v ) ∈ A ( u,v ) ∈ B Wheeler Ruml (UNH) Class 25, CS 758 – 18 / 20

Suboptimality of Local Search divide by 2: Backtracking Local Search � � w uv + w uv ≤ w ( A, B ) ■ Local Search ■ Local Search ( u,v ) ∈ A ( u,v ) ∈ B ■ Max Cut ■ Suboptimality ■ EOLQs eg, more weight crossing than within partitions let W be sum of all weight in graph. add crossing weight to both sides: W ≤ 2 w ( A, B ) W/ 2 ≤ w ( A, B ) note optimal is at most W Wheeler Ruml (UNH) Class 25, CS 758 – 19 / 20

EOLQs For example: Backtracking Local Search What’s still confusing? ■ ■ Local Search What question didn’t you get to ask today? ■ ■ Local Search ■ Max Cut What would you like to hear more about? ■ ■ Suboptimality ■ EOLQs Please write down your most pressing question about algorithms and put it in the box on your way out. Thanks! Wheeler Ruml (UNH) Class 25, CS 758 – 20 / 20