The Simulated Annealing Algorithm An approximation algorithm Course: CS 5130 - Advanced Data Structures and Algorithms Instructor: Dr. Badri Adhikari
Mathematical Optimization Mathematical optimization is the selection of a best element (with regard to some criteria) from some set of available alternatives Given: a function f : A to R maps some set A to the real numbers Minimization: find an element x0 in A such that f(x0) ≤ f(x) for all x in A Maximization: find an element x0 in A such that f(x0) ≥ f(x) for all x in A Example:
Examples of Space Search Problems
1. Travelling Salesman Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city? Solution to a symmetric TSP with 7 cities using brute force search. Note: Number of permutations: (7-1)!/2 = 360 http://en.wikipedia.org/wiki/File:Bruteforce.gif
2. Protein Fold Space Search
Energy function Is the computation time needed for What is energy function for TSP? energy function evaluation Sum of distances important? What is energy function for protein Why? fold search? Can we compute energy of all ? configurations in the space? Can we list all the configurations at all?
Hill Climbing vs Random Walk - Start with random values of the variables - Compute the energy function - Make small changes to the values, and see if we get a better value of the energy function - Repeat, until no better values of energy function can be obtained State is simply the set of variable values
Simulated Annealing
Annealing - want to produce materials of good properties, like strength - involves create liquid version and then solidifying example: casting - desirable to arrange the atoms in a systematic fashion, which in other words corresponds to low energy - we want minimum energy Annealing - physical process of controlled cooling
Original Paper introducing the idea
Statistical Mechanics The behavior of systems with many degrees of freedom in thermal equilibrium at a finite temperature. Combinatorial Optimization Finding the minimum of a given function depending on many variables. The Analogy ● If a liquid material cools and anneals too quickly, then the material will solidify into a sub-optimal configuration. ● If the liquid material cools slowly, the crystals within the material will solidify optimally into a state of minimum energy (i.e. ground state). ● This ground state corresponds to the minimum of the cost function in an optimization problem.
Intuition behind SA water bubble rings maze
Introduce probability - make move with some probability - such that, if the move is a good one, the probability is high we want Δ E to influence the probability - allow good moves (high Δ E) with high probability, and allow bad moves with - low probability
Counting the number of remaining SA Block Diagram (for minimization problem) (unhooked) rings = minimization problem Initial Temperature Random Configuration Count the number of UNHOOKED rings Press button 1 inner epoch 1 outer epoch Accept all good moves Is it a better move? 8 - 10 = -2 < 0 ? Loop X times Assignment: Test the algorithm with some real numbers. If the move is too bad, accept with a low probability
Control how Δ E influences probability - Function - range should be 0 and 1 (probability) - domain should be infinite - Sigmoid function ΔE=0 ΔE
How to choose value for T ? - we follow the physical world approach - cool the system gradually and hope that the system will settle to an optimal state - we initialize T to some high value - gradually decrease T - some monotonically decreasing function (cooling rate) - How to select initial and final temperature? - empirically
Key Ingredients for SA 1. A concise description of a configuration (architecture, How to determine design, topology) of the system (Design Vector). when a ring is hooked? 2. A random generator of rearrangements of the elements in a configuration (Neighborhoods). This generator The button to press encapsulates rules so as to generate only valid configurations. Perturbation function. 3. A quantitative objective function containing the trade-offs that have to be made (Simulation Model and Output A formula for calculating which configuration is better/worse. Metric(s)). Surrogate for system energy. 4. An annealing schedule of the temperatures and/or the The monotonically decreasing length of times for which the system is to be evolved. function
Reconstruction of the protein molecule 1GUU - Initial Temperature = 50 (length of the protein) - Final Temperature = 0.01 - Number of SA iterations = 3200 - Number of true C β contacts = 45 - Contact energy = Root Mean Square Deviations T i+1 = T i - 10 N epoch = 100 A movie clip demonstrating an application of monte carlo simulated annealing to reconstruct a small alpha helical protein 1GUU. https://www.youtube.com/watch?v=2p5x7XROxlo
Summary There are algorithms (approximation algorithms) for NP-hard problems. Simulated annealing algorithm is an example. The SA algorithm probabilistically combines random walk and hill climbing algorithms. The output of one SA run may be different from another SA run. Typically, we run more than once to draw some initial conclusions. In the SA algorithm we always accept good moves. The probability of accepting a bad move depends on - temperature & change in energy.
Recommend
More recommend
