Introduction: Combinatorial Problems and Search slightly adapted - PDF document

HEURISTIC OPTIMIZATION Introduction: Combinatorial Problems and Search slightly adapted from slides for SLS:FA, Chapter 1 Outline 1. Combinatorial Problems 2. Two Prototypical Combinatorial Problems 3. Computational Complexity 4. Search Paradigms 5. Stochastic Local Search Heuristic Optimization 2018 2

Combinatorial Problems Combinatorial problems arise in many areas of computer science and application domains: I finding shortest/cheapest round trips (TSP) I finding models of propositional formulae (SAT) I planning, scheduling, time-tabling I vehicle routing I location and clustering I internet data packet routing I protein structure prediction Heuristic Optimization 2018 3 Combinatorial problems involve finding a grouping , ordering , or assignment of a discrete, finite set of objects that satisfies given conditions. Candidate solutions are combinations of solution components that may be encountered during a solution attempt but need not satisfy all given conditions. Solutions are candidate solutions that satisfy all given conditions. Heuristic Optimization 2018 4

Example: I Given: Set of points in the Euclidean plane I Objective: Find the shortest round trip Note: I a round trip corresponds to a sequence of points (= assignment of points to sequence positions) I solution component: trip segment consisting of two points that are visited one directly after the other I candidate solution: round trip I solution : round trip with minimal length Heuristic Optimization 2018 5 Problem vs problem instance: I Problem: Given any set of points X , find a shortest round trip I Solution: Algorithm that finds shortest round trips for any X I Problem instance: Given a specific set of points P , find a shortest round trip I Solution: Shortest round trip for P Technically, problems can be formalised as sets of problem instances. Heuristic Optimization 2018 6

Decision problems: solutions = candidate solutions that satisfy given logical conditions Example: The Graph Colouring Problem I Given: Graph G and set of colours C I Objective: Assign to all vertices of G a colour from C such that two vertices connected by an edge are never assigned the same colour Heuristic Optimization 2018 7 Every decision problem has two variants: I Search variant: Find a solution for given problem instance (or determine that no solution exists) I Decision variant: Determine whether solution for given problem instance exists Note: Search and decision variants are closely related; algorithms for one can be used for solving the other. Heuristic Optimization 2018 8

Optimisation problems: I can be seen as generalisations of decision problems I objective function f measures solution quality (often defined on all candidate solutions) I typical goal: find solution with optimal quality minimisation problem: optimal quality = minimal value of f maximisation problem: optimal quality = maximal value of f Example: Variant of the Graph Colouring Problem where the objective is to find a valid colour assignment that uses a minimal number of colours. Note: Every minimisation problem can be formulated as a maximisation problems and vice versa. Heuristic Optimization 2018 9 Variants of optimisation problems: I Search variant: Find a solution with optimal objective function value for given problem instance I Evaluation variant: Determine optimal objective function value for given problem instance Every optimisation problem has associated decision problems : Given a problem instance and a fixed solution quality bound b , find a solution with objective function value  b (for minimisation problems) or determine that no such solution exists. Heuristic Optimization 2018 10

Many optimisation problems have an objective function as well as logical conditions that solutions must satisfy. A candidate solution is called feasible (or valid ) i ff it satisfies the given logical conditions. Note: Logical conditions can always be captured by an evaluation function such that feasible candidate solutions correspond to solutions of an associated decision problem with a specific bound. Heuristic Optimization 2018 11 Note: I Algorithms for optimisation problems can be used to solve associated decision problems I Algorithms for decision problems can often be extended to related optimisation problems . I Caution: This does not always solve the given problem most e ffi ciently. Heuristic Optimization 2018 12

Two Prototypical Combinatorial Problems Studying conceptually simple problems facilitates development, analysis and presentation of algorithms Two prominent, conceptually simple problems: I Finding satisfying variable assignments of propositional formulae (SAT) – prototypical decision problem I Finding shortest round trips in graphs (TSP) – prototypical optimisation problem Heuristic Optimization 2018 13 SAT: A simple example I Given: Formula F := ( x 1 _ x 2 ) ^ ( ¬ x 1 _ ¬ x 2 ) I Objective: Find an assignment of truth values to variables x 1 , x 2 that renders F true, or decide that no such assignment exists. General SAT Problem (search variant): I Given: Formula F in propositional logic I Objective: Find an assignment of truth values to variables in F that renders F true, or decide that no such assignment exists. Heuristic Optimization 2018 14

Definition: I Formula in propositional logic : well-formed string that may contain I propositional variables x 1 , x 2 , . . . , x n ; I truth values > (‘true’), ? (‘false’); I operators ¬ (‘not’), ^ (‘and’), _ (‘or’); I parentheses (for operator nesting). I Model (or satisfying assignment ) of a formula F : Assignment of truth values to the variables in F under which F becomes true (under the usual interpretation of the logical operators) I Formula F is satisfiable i ff there exists at least one model of F , unsatisfiable otherwise. Heuristic Optimization 2018 15 Definition: I A formula is in conjunctive normal form (CNF) i ff it is of the form k ( i ) m ^ _ l ij = ( l 11 _ . . . _ l 1 k (1) ) . . . ^ ( l m 1 _ . . . _ l mk ( m ) ) i =1 j =1 where each literal l ij is a propositional variable or its negation. The disjunctions ( l i 1 _ . . . _ l ik ( i ) ) are called clauses . I A formula is in k -CNF i ff it is in CNF and all clauses contain exactly k literals ( i.e. , for all i , k ( i ) = k ). Note: For every propositional formula, there is an equivalent formula in 3-CNF. Heuristic Optimization 2018 16

Concise definition of SAT: I Given: Formula F in propositional logic. I Objective: Decide whether F is satisfiable. Note: I In many cases, the restriction of SAT to CNF formulae is considered. I The restriction of SAT to k -CNF formulae is called k -SAT . Heuristic Optimization 2018 17 Example: F := ^ ( ¬ x 2 _ x 1 ) ^ ( ¬ x 1 _ ¬ x 2 _ ¬ x 3 ) ^ ( x 1 _ x 2 ) ^ ( ¬ x 4 _ x 3 ) ^ ( ¬ x 5 _ x 3 ) I F is in CNF. I Is F satisfiable? Yes, e.g. , x 1 := x 2 := > , x 3 := x 4 := x 5 := ? is a model of F . Heuristic Optimization 2018 18

TSP: A simple example Europe Bonifaccio Circeo Maronia Corfu Troy Ustica Stromboli Asia Ithaca Messina Favignana Zakinthos Taormina Malea Gibraltar Birzebbuga Djerba Africa Heuristic Optimization 2018 19 Definition: I Hamiltonian cycle in graph G := ( V , E ): cyclic path that visits every vertex of G exactly once (except start/end point). I Weight of path p := ( u 1 , . . . , u k ) in edge-weighted graph G := ( V , E , w ): total weight of all edges on p , i.e. : k � 1 X w ( p ) := w (( u i , u i +1 )) i =1 Heuristic Optimization 2018 20

The Travelling Salesman Problem (TSP) I Given: Directed, edge-weighted graph G . I Objective: Find a minimal-weight Hamiltonian cycle in G . Types of TSP instances: I Symmetric : For all edges ( v , v 0 ) of the given graph G , ( v 0 , v ) is also in G , and w (( v , v 0 )) = w (( v 0 , v )). Otherwise: asymmetric . I Euclidean : Vertices = points in a Euclidean space, weight function = Euclidean distance metric. I Geographic : Vertices = points on a sphere, weight function = geographic (great circle) distance. Heuristic Optimization 2018 21 Computational Complexity Fundamental question: How hard is a given computational problems to solve? Important concepts: I Time complexity of a problem Π : Computation time required for solving a given instance π of Π using the most e ffi cient algorithm for Π . I Worst-case time complexity: Time complexity in the worst case over all problem instances of a given size, typically measured as a function of instance size. Heuristic Optimization 2018 22

Time complexity I time complexity gives the amount of time taken by an algorithm as a function of the input size I time complexity often described by big-O notation ( O ( · )) I let f and g be two functions I we say f ( n ) = O ( g ( n )) if two positive numbers c and n 0 exist such that for all n � n 0 we have f ( n )  c · g ( n ) I we call an algorithm polynomial-time if its time complexity is bounded by a polynomial p ( n ), ie. f ( n ) = O ( p ( n )) I we call an algorithm exponential-time if its time complexity cannot be bounded by a polynomial Heuristic Optimization 2018 23 Heuristic Optimization 2018 24