Outline DM63 HEURISTICS FOR COMBINATORIAL OPTIMIZATION 1. Example Problems The Constraint Satisfaction Problem Problem Solving The Single Machine Total Tardiness Problem Lecture 2 Heuristic Methods and Metaheuristics for 2. Heuristic Methods Construction Search 3. Metaheuristics for Construction Heuristics Complete Tree Search Rollout/Pilot Method Marco Chiarandini Limited Discrepancy Search Beam Search DM63 – Heuristics for Combinatorial Optimization Problems 2 Past Lecture Systematic search is often better suited when ... ◮ proofs of insolubility or optimality are required; ◮ time constraints are not critical; ◮ problem-specific knowledge can be exploited. ◮ Course Introduction ◮ Combinatorial Problems Local search is often better suited when ... ◮ Graph Coloring ◮ Propositional Satisfiability ◮ non linear constraints and non linear objective function; ◮ Computational Complexity ◮ reasonably good solutions are required within a short time; ◮ Solution Methods ◮ problem-specific knowledge is rather limited. ◮ Construction Heuristics for the Traveling Salesman Problem ◮ Software Development Complementarity: Local and systematic search can be fruitfully combined, e.g. , by using local search for finding solutions whose optimality is proved using systematic search. DM63 – Heuristics for Combinatorial Optimization Problems 3 DM63 – Heuristics for Combinatorial Optimization Problems 4
Outline Example Problems 1. Example Problems The Constraint Satisfaction Problem Problem Solving ◮ They are chosen because conceptually concise, intended to illustrate the The Single Machine Total Tardiness Problem development, analysis and presentation of algorithms 2. Heuristic Methods ◮ Although real-world problems tend to have much more complex formulations these problems capture their essence 3. Metaheuristics for Construction Heuristics Complete Tree Search Rollout/Pilot Method Limited Discrepancy Search Beam Search DM63 – Heuristics for Combinatorial Optimization Problems 5 DM63 – Heuristics for Combinatorial Optimization Problems 6 Preprocessing Rules Constraint Satisfaction Problem Polynomial simplifications SAT (CNF formulae) ◮ Given: a triple � X, D, C � , where X is a set of variables, D is a domain ◮ elimination of duplicates of values, and C is a set of constraints. Every constraint is a pair � t, R � , ◮ elimination tautological clauses where t is a tuple of variables and R is a relation (or a set of tuples of ◮ elimination subsumed clauses values; all these tuples having the same number of elements) ◮ elimination clauses with pure literals An evaluation of the variables is a function from variables to v : X → D . ◮ elimination unit clauses and unit propagation Such an evaluation satisfies a constraint � ( x 1 , . . . , x n ) , R � if ( v ( x 1 ) , . . . , v ( x n )) ∈ R . k -coloring ◮ Task: A solution is an evaluation that satisfies all constraints. ◮ Remove under-constrained nodes ◮ Remove subsumed nodes ◮ Merge nodes that must have the same color DM63 – Heuristics for Combinatorial Optimization Problems 7 DM63 – Heuristics for Combinatorial Optimization Problems 8
Problem Solving as by George P´ olya P´ olya’s First Principle: Understand the Problem ◮ Do you understand all the words used in stating the problem? George P´ olya’s 1945 book How to Solve It : ◮ What are you asked to find or show? ◮ Can you restate the problem in your own words? ◮ First, you have to understand the problem. ◮ Can you think of a picture or a diagram that might help you understand ◮ After understanding, then make a plan. the problem? ◮ Carry out the plan. ◮ Is there enough information to enable you to find a solution? ◮ Look back on your work. How could it be better? DM63 – Heuristics for Combinatorial Optimization Problems 9 DM63 – Heuristics for Combinatorial Optimization Problems 10 P´ olya’s Second Principle: Devise a plan There are many reasonable ways to solve problems. The skill at choosing an appropriate strategy is best learned by solving many problems. You will find choosing a strategy increasingly easy. A partial list of strategies is included: P´ olya’s third Principle: Carry out the plan ◮ Guess and check Need is care and patience, given that you have the necessary skills. Persist with the plan that you have chosen. If it continues not to work discard it and ◮ Make an orderly list choose another. Don’t be misled, this is how mathematics is done, even by ◮ Eliminate possibilities professionals. ◮ Use symmetry ◮ Consider special cases ◮ Use direct reasoning P´ olya’s fourth Principle: Review/Extend Also suggested: Much can be gained by taking the time to reflect and look back at what you ◮ Look for a pattern have done, what worked and what didn’t. Doing this will enable you to predict what strategy to use to solve future problems. ◮ Draw a picture ◮ Solve a simpler problem ◮ Use a model ◮ Work backward ◮ Be ingenious DM63 – Heuristics for Combinatorial Optimization Problems 11 DM63 – Heuristics for Combinatorial Optimization Problems 12
The Single Machine Total Tardiness Problem ”If you can’t solve a problem, then there is an easier problem you can solve: find it” Given: a set of n jobs { J 1 , . . . , J n } to be processed on a single machine and for each job J i a processing time p i , a weight w i and a due date d i . Task: Find a schedule that minimizes the total weighted tardiness � n i =1 w i · T i where T i = { C i − d i , 0 } ( C i completion time of job J i ) Example: Job J 1 J 2 J 3 J 4 J 5 J 6 Processing Time 3 2 2 3 4 3 Due date 6 13 4 9 7 17 Weight 2 3 1 5 1 2 Sequence φ = J 3 , J 1 , J 5 , J 4 , J 1 , J 6 Job J 3 J 1 J 5 J 4 J 1 J 6 C i 2 5 9 12 14 17 T i 0 0 2 3 1 0 w i · T i 0 0 2 15 3 0 DM63 – Heuristics for Combinatorial Optimization Problems 13 DM63 – Heuristics for Combinatorial Optimization Problems 14 Outline Heuristic Paradigms Construction Heuristics 1. Example Problems They are closely related to search tree techniques but correspond to a single The Constraint Satisfaction Problem path from root to leaf Problem Solving ◮ search space = partial candidate solutions The Single Machine Total Tardiness Problem ◮ search step = extension with one or more solution components 2. Heuristic Methods 3. Metaheuristics for Construction Heuristics Construction Heuristic (CH): Complete Tree Search s := ∅ Rollout/Pilot Method While s is not a complete solution: Limited Discrepancy Search | | choose a solution component c Beam Search ⌊ add the solution component to s DM63 – Heuristics for Combinatorial Optimization Problems 15 DM63 – Heuristics for Combinatorial Optimization Problems 16
Heuristic Paradigms Perturbative search ◮ search space = complete candidate solutions Construction Heuristics ◮ search step = modification of one or more solution components An important class of Construction Heuristics are greedy algorithms. ◮ iteratively generate and evaluate candidate solutions ◮ Strategy: always make the choice which is the best at the moment. ◮ decision problems: evaluation = test if solution ◮ They are not generally guaranteed to find globally optimal solutions ◮ optimization problems: evaluation = check objective function value (but sometimes they do: Minimum Spanning Tree, Single Source ◮ evaluating candidate solutions is typically computationally much cheaper Shortest Path, etc.) than finding (optimal) solutions Iterative Improvement (II): While s has better neighbors: | choose a neighbor s ′ of s | | such that f ( s ′ ) < f ( s ) | ⌊ s := s ′ DM63 – Heuristics for Combinatorial Optimization Problems 17 DM63 – Heuristics for Combinatorial Optimization Problems 18 Heuristic Methods Example: Uninformed random walk for SAT Meta-heuristics are general purpose guidance criteria to enhance these two procedure URW-for-SAT( F ,maxSteps ) heuristic paradigms. input: propositional formula F , integer maxSteps output: model of F or ∅ Meta-heuristics choose assignment ϕ of truth values to all variables in F uniformly at random; General guidance criteria over lower level heuristics. steps := 0; while not (( ϕ satisfies F ) and ( steps < maxSteps )) do Metaheuristics select x uniformly at random from { x ′ | x ′ is a variable in F and changing value of x ′ in ϕ decreases the number of unsatisfied clauses } ; steps := steps +1 ; end Construction Perturbative if ϕ satisfies F then Heuristics Search return ϕ else return ∅ In order to obtain very high quality solutions end it is often successful to combine these heuristic paradigms into an hybrid end URW-for-SAT algorithm DM63 – Heuristics for Combinatorial Optimization Problems 19 DM63 – Heuristics for Combinatorial Optimization Problems 20
Recommend
More recommend
