Local Search Theory Marco Chiarandini Department of Mathematics - PowerPoint PPT Presentation

DM841 D ISCRETE O PTIMIZATION Part 2 – Heuristics Local Search Theory Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

Local Search Revisited Outline Metaheuristics 1. Local Search Revisited Search Space Properties Neighborhoods Formalized Distances Landscape Characteristics 2. Metaheuristics 2

Local Search Revisited Outline Metaheuristics 1. Local Search Revisited Search Space Properties Neighborhoods Formalized Distances Landscape Characteristics 2. Metaheuristics 3

Local Search Revisited Outline Metaheuristics 1. Local Search Revisited Search Space Properties Neighborhoods Formalized Distances Landscape Characteristics 2. Metaheuristics 4

LS Algorithm Components Local Search Revisited Metaheuristics Neighborhood function Neighborhood function N π : S π → 2 S π Also defined as: N : S × S → { T , F } or N ⊆ S × S ◮ neighborhood (set) of candidate solution s : N ( s ) := { s ′ ∈ S | N ( s , s ′ ) } ◮ neighborhood size is | N ( s ) | ◮ neighborhood is symmetric if: s ′ ∈ N ( s ) ⇒ s ∈ N ( s ′ ) ◮ neighborhood graph of ( S , N , π ) is a directed graph: G N π := ( V , A ) with V = S π and ( uv ) ∈ A ⇔ v ∈ N ( u ) (if symmetric neighborhood � undirected graph) Notation: N when set, N when collection of sets or function 5

Local Search Revisited Metaheuristics A neighborhood function is also defined by means of an operator (aka move). An operator ∆ is a collection of operator functions δ : S → S such that s ′ ∈ N ( s ) ∃ δ ∈ ∆ , δ ( s ) = s ′ ⇒ = Definition k -exchange neighborhood: candidate solutions s , s ′ are neighbors iff s differs from s ′ in at most k solution components Examples: ◮ 1-exchange (flip) neighborhood for SAT (solution components = single variable assignments) ◮ 2-exchange neighborhood for TSP (solution components = edges in given graph) 6

Local Search Revisited LS Algorithm Components Metaheuristics Definition: ◮ Local minimum: search position without improving neighbors wrt given evaluation function f and neighborhood N , i.e. , position s ∈ S such that f ( s ) ≤ f ( s ′ ) for all s ′ ∈ N ( s ) . ◮ Strict local minimum: search position s ∈ S such that f ( s ) < f ( s ′ ) for all s ′ ∈ N ( s ) . ◮ Local maxima and strict local maxima : defined analogously. 7

Local Search Revisited LS Algorithm Components Metaheuristics Note: ◮ Local search implements a walk through the neighborhood graph ◮ Procedural versions of init , step and terminate implement sampling from respective probability distributions. ◮ Local search algorithms can be described as Markov processes: behavior in any search state { s , m } depends only on current position s higher order MP if (limited) memory m . 8

LS Algorithm Components Local Search Revisited Metaheuristics Step function Search step (or move): pair of search positions s , s ′ for which s ′ can be reached from s in one step, i.e. , N ( s , s ′ ) and step ( { s , m } , { s ′ , m ′ } ) > 0 for some memory states m , m ′ ∈ M . ◮ Search trajectory: finite sequence of search positions � s 0 , s 1 , . . . , s k � such that ( s i − 1 , s i ) is a search step for any i ∈ { 1 , . . . , k } and the probability of initializing the search at s 0 is greater than zero, i.e. , init ( { s 0 , m } ) > 0 for some memory state m ∈ M . ◮ Search strategy: specified by init and step function; to some extent independent of problem instance and other components of LS algorithm. ◮ random ◮ based on evaluation function ◮ based on memory 9

Local Search Revisited Outline Metaheuristics 1. Local Search Revisited Search Space Properties Neighborhoods Formalized Distances Landscape Characteristics 2. Metaheuristics 10

Local Search Revisited Neighborhood Operator Metaheuristics Goal: providing a formal description of neighborhood functions for the three main solution representations: ◮ Permutation ◮ linear permutation: Single Machine Total Weighted Tardiness Problem ◮ circular permutation: Traveling Salesman Problem ◮ Assignment: SAT, CSP ◮ Set, Partition: Max Independent Set A neighborhood function N : S → 2 S is also defined through an operator. An operator ∆ is a collection of operator functions δ : S → S such that s ′ ∈ N ( s ) ∃ δ ∈ ∆ | δ ( s ) = s ′ ⇐ ⇒ 11

Local Search Revisited Permutations Metaheuristics S n indicates the set all permutations of the numbers { 1 , 2 , . . . , n } ( 1 , 2 . . . , n ) is the identity permutation ι . If π ∈ Π( n ) and 1 ≤ i ≤ n then: ◮ π i is the element at position i ◮ pos π ( i ) is the position of element i Alternatively, a permutation is a bijective function π ( i ) = π i The permutation product π · π ′ is the composition ( π · π ′ ) i = π ′ ( π ( i )) For each π there exists a permutation such that π − 1 · π = ι π − 1 ( i ) = pos π ( i ) ∆ N ⊂ S n 12

Local Search Revisited Linear Permutations Metaheuristics Swap operator ∆ S = { δ i S | 1 ≤ i ≤ n } δ i S ( π 1 . . . π i π i + 1 . . . π n ) = ( π 1 . . . π i + 1 π i . . . π n ) Interchange operator ∆ X = { δ ij X | 1 ≤ i < j ≤ n } δ ij X ( π ) = ( π 1 . . . π i − 1 π j π i + 1 . . . π j − 1 π i π j + 1 . . . π n ) ( ≡ set of all transpositions) Insert operator ∆ I = { δ ij I | 1 ≤ i ≤ n , 1 ≤ j ≤ n , j � = i } � ( π 1 . . . π i − 1 π i + 1 . . . π j π i π j + 1 . . . π n ) i < j δ ij I ( π ) = ( π 1 . . . π j π i π j + 1 . . . π i − 1 π i + 1 . . . π n ) i > j 13

Local Search Revisited Circular Permutations Metaheuristics Reversal (2-edge-exchange) ∆ R = { δ ij R | 1 ≤ i < j ≤ n } δ ij R ( π ) = ( π 1 . . . π i − 1 π j . . . π i π j + 1 . . . π n ) Block moves (3-edge-exchange) ∆ B = { δ ijk B | 1 ≤ i < j < k ≤ n } δ ij B ( π ) = ( π 1 . . . π i − 1 π j . . . π k π i . . . π j − 1 π k + 1 . . . π n ) Short block move (Or-edge-exchange) ∆ SB = { δ ij SB | 1 ≤ i < j ≤ n } δ ij SB ( π ) = ( π 1 . . . π i − 1 π j π j + 1 π j + 2 π i . . . π j − 1 π j + 3 . . . π n ) 14

Local Search Revisited Assignments Metaheuristics An assignment can be represented as a mapping σ : { X 1 . . . X n } → { v : v ∈ D , | D | = k } : σ = { X i = v i , X j = v j , . . . } One-exchange operator ∆ 1 E = { δ il 1 E | 1 ≤ i ≤ n , 1 ≤ l ≤ k } σ ′ : σ ′ ( X i ) = v l and σ ′ ( X j ) = σ ( X j ) ∀ j � = i δ il � � � σ ) = 1 E Two-exchange operator ∆ 2 E = { δ ij 2 E | 1 ≤ i < j ≤ n } σ ′ : σ ′ ( X i ) = σ ( X j ) , σ ′ ( X j ) = σ ( X i ) and σ ′ ( X l ) = σ ( X l ) ∀ l � = i , j δ ij � � 2 E ( σ ) = 15

Local Search Revisited Partitioning Metaheuristics An assignment can be represented as a partition of objects selected and not selected s : { X } → { C , C } (it can also be represented by a bit string) One-addition operator 1 E | v ∈ ¯ ∆ 1 E = { δ v C } s : C ′ = C ∪ v and ¯ C ′ = ¯ δ v � � C \ v } s ) = 1 E One-deletion operator ∆ 1 E = { δ v 1 E | v ∈ C } s : C ′ = C \ v and ¯ C ′ = ¯ δ v � � s ) = C ∪ v } 1 E Swap operator 1 E | v ∈ C , u ∈ ¯ ∆ 1 E = { δ v C } s : C ′ = C ∪ u \ v and ¯ C ′ = ¯ δ v � � s ) = C ∪ v \ u } 1 E 16

Local Search Revisited Outline Metaheuristics 1. Local Search Revisited Search Space Properties Neighborhoods Formalized Distances Landscape Characteristics 2. Metaheuristics 17

Local Search Revisited Distances Metaheuristics Set of paths in N with s , s ′ ∈ S : Φ( s , s ′ ) = { ( s 1 , . . . , s h ) | s 1 = s , s h = s ′ ∀ i : 1 ≤ i ≤ h − 1 , � s i , s i + 1 � ∈ E N } If φ = ( s 1 , . . . , s h ) ∈ Φ( s , s ′ ) let | φ | = h be the length of the path; then the distance between any two solutions s , s ′ is the length of shortest path between s and s ′ in N : d N ( s , s ′ ) = φ ∈ Φ( s , s ′ ) | Φ | min diam ( N ) = max { d N ( s , s ′ ) | s , s ′ ∈ S } (= maximal distance between any two candidate solutions) (= worst-case lower bound for number of search steps required for reaching (optimal) solutions) Note : with permutations it is easy to see that: d N ( π, π ′ ) = d N ( π − 1 · π ′ , ι ) 18

Local Search Revisited Metaheuristics Distances for Linear Permutation Representations ◮ Swap neighborhood operator computable in O ( n 2 ) by the precedence based distance metric: d S ( π, π ′ ) = # {� i , j �| 1 ≤ i < j ≤ n , pos π ′ ( π j ) < pos π ′ ( π i ) } . diam ( G N ) = n ( n − 1 ) / 2 ◮ Interchange neighborhood operator Computable in O ( n ) + O ( n ) since d X ( π, π ′ ) = d X ( π − 1 · π ′ , ι ) = n − c ( π − 1 · π ′ ) c ( π ) is the number of disjoint cycles that decompose a permutation. diam ( G N X ) = n − 1 ◮ Insert neighborhood operator Computable in O ( n ) + O ( n log ( n )) since d I ( π, π ′ ) = d I ( π − 1 · π ′ , ι ) = n − | lis ( π − 1 · π ′ ) | where lis ( π ) denotes the length of the longest increasing subsequence. diam ( G N I ) = n − 1 19

Local Search Revisited Metaheuristics Distances for Circular Permutation Representations ◮ Reversal neighborhood operator sorting by reversal is known to be NP-hard surrogate in TSP: bond distance ◮ Block moves neighborhood operator unknown whether it is NP-hard but there does not exist a proved polynomial-time algorithm 20