Outline • I t erat ive improvement algorit hms • Hill climbing search Local Search • Simulat ed annealing • Genet ic algorit hms CS 486/ 686 Univer sit y of Wat erloo May 12, 2005 1 2 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart Examples I ntroduction Vehicle rout ing • So f ar we have st udied algorit hms which syst emat ically explore sear ch spaces – Keep one or more pat hs in memory – When t he goal is f ound, t he solut ion consist s of a pat h t o t he goal • For many problems t he pat h is unimpor t ant Channel Rout ing 3 4 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart Examples I ntroduction • I nf ormal charact erizat ion J ob shop – Combinat orial st ruct ure being opt imized scheduling – There is a cost f unct ion t o be opt imized • At least we want t o f ind a good solut ion – Searching all possible st at es is inf easible A v ~B v C – No known algorit hm f or f inding t he solut ion Boolean ~A v C v D ef f icient ly Sat isf iabilit y B v D v ~E – Some not ion of similar st at es having similar ~C v ~D v ~E cost s … 5 6 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart 1
Example - TSP Constructive method • For t he opt imal solut ion we could use A*! – But we do not really need t o know how we got t o t he solut ion – we j ust want t he solut ion • Goal is t o minimize t he lengt h of t he rout e – Can be very expensive t o run • Const ruct ive met hod: – St art f rom scrat ch and build up a solut ion • I terat ive improvement method: – St art wit h a solut ion and t ry t o improve it 7 8 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart I terative improvement methods I terative improvement methods 1. St art at some random point on t he landscape • I dea: I magine all possible solut ions laid 2. Generat e all possible point s t o move t o out on a landscape 3. Choose a point of improvement and – We want t o f ind t he highest (or lowest ) move t o it point 4. I f you are st uck t hen rest art 9 10 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart Hill- climbing I terative improvement methods 1. St art at some init ial conf igurat ion S 2. Let V=Eval(S) • What does it mean t o “generat e point s t o move t o” 3. Let N=Move_Set (S) 4. For each X i ∈ ∈ ∈ ∈ N – Somet imes called generat ing t he moveset • Depends on t he applicat ion – Let V max =max i Eval(X i ) and X max =argmax i Eval(X i ) TSP 5. I f V max ≤ V, ret urn S 6. Let S=X max and V=V max . Go t o 3 “Like t rying t o f ind t he peak of Mt Everest in t he f og”, 2-swap Russell and Norvig 11 12 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart 2
Hill Climbing Hill Climbing • I ssues wit h hill climbing – I t can get st uck! • Always t ake a st ep in t he direct ion t hat improves t he current solut ion value t he – Local maximum (local minimum) most – Plat eaus – Greedy objective function • Good t hings about hill climbing global maximum – Easy t o progr am! shoulder – Requires no memory of where we have been! local maximum "flat" local maximum – I t is import ant t o have a “good” set of moves • Not t oo many, not t oo f ew state space current state 13 14 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart I mproving on hill climbing Hill climbing example: GSAT • Plat eaus Av~BvC 1 Conf igurat ion A=1, B=0, C=1, D=0, E=1 – Allow f or sideways moves, but be caref ul ~AvCvD 1 since may move sideways f orever! Goal is t o maximize t he number of sat isf ied BvDv~E 0 clauses: Eval(conf ig)=# sat isf ied clauses • Local Maximum ~Cv~Dv~E 1 – Random rest art s: “I f at f irst you do not GSAT Move_Set: Flip any 1 variable ~Av~CvE 1 succeed, t ry, t ry again” WALKSAT (Randomized GSAT) – Random rest art s works well in pract ice Pick a random unsat isf ied clause; • Randomized hill climbing Consider f lipping each variable in t he clause – Like hill climbing except you choose a random I f any improve Eval, t hen accept t he best stat e f rom the move set , and t hen move t o I f none improve Eval, t hen wit h prob p pick t he it if it is bet t er t han current st at e. Cont inue move t hat is least bad; prob (1-p) pick a random unt il you are bored one 15 16 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart Simulated Annealing Simulated annealing • Hill climbing algor it hms which never 1. Let S be t he init ial conf igurat ion and make downhill moves ar e incomplet e V=Eval(S) – Can get st uck at local maxima (minima) 2. Let i be a random move f rom t he moveset and let S i be t he next • A r andom walk is complet e but ver y inef f icient conf igurat ion, V i =Eval(S i ) 3. I f V< V i t hen S=S i and V=V i New I dea: 4. Else wit h probabilit y p, S=S i and V=V i Allow t he algorit hm t o make some “bad” moves in order t o escape local maxima. 5. Got o 2 unt il you are bored 17 18 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart 3
Selecting moves in simulat ed annealing Simulated annealing • I f new value V i is bet t er t han old value V t hen def init ely move t o new solut ion • How should we choose t he probabilit y of • I f new value V i is worse t han old value V accept ing a “bad” move? t hen move t o new solut ion wit h probabilit y – I dea 1: p=0.1 (or some ot her f ixed value)? Exp(-(V-V i )/ T) – I dea 2: Probabilit y t hat decreases wit h t ime? Bolt zmann dist ribut ion: T> 0 is a paramet er called – I dea 3: Probabilit y t hat decreases wit h t emperat ure. I t st art s high and decreases over t ime and as V-V i increases? t ime t owards 0 I f T is close t o 0 t hen t he probabilit y of making a bad 19 move is almost 0 20 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart Properties of simulated Genetic Algorithms annealing • Problems are encoded int o a represent at ion • I f T is decreased slowly enough t hen which allows cert ain operat ions t o occur simulat ed annealing is guarant eed (in – Usually use a bit st ring t heory) t o reach best solut ion – The represent at ion is key – needs t o be t hought out caref ully – Annealing schedule is crit ical • An encoded candidat e solut ion is an individual • When T is high: Explorat ory phase • Each individual has a f it ness which is a numerical (random walk) value associat ed wit h it s qualit y of solut ion • When T is low: Exploit at ion phase • A populat ion is a set of individuals (r andomized hill climbing) • Populat ions change over gener at ions by applying st rat egies t o t hem 21 22 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart Typical genetic algorithm Crossover • Consist s of combining part s of individuals • I nit ialize: Populat ion P consist s of N random individuals (bit st rings) t o creat e new individuals • Evaluat e: f or each x ∈ P, comput e f it ness(x) • Choose a random crossover point • Loop – Cut t he individuals t here and swap t he pieces – For i=1 t o N do • Choose 2 parent s each wit h probabilit y proport ional t o 101| 0101 011|1110 f it ness scores Cross over • Crossover t he 2 parent s t o produce a new bit st ring (child) • Wit h some small probabilit y mut at e child 011| 0101 101|1110 • Add child t o t he populat ion • Unt il some child is f it enough or you get bored I mplement at ion: use a crossover mask m • Ret urn t he best child in t he populat ion Given t wo parent s a and b t he of f spring are according t o f it ness f unct ion (a m) v (b v v ~m) and (a v ~m) v (b v m) 23 24 CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart CS486/686 Lecture Slides (c) 2005 K. Larson and P. Poupart 4
Recommend
More recommend
