Stochastic Search and Diffusion Arturo Berrones
Some basic physics: Newton’s 2nd Law of motion: F = ma = m ¨ x The force F can be expressed as the gradient of a Born 4 January 1643 [OS: 25 December 1642] [1] Woolsthorpe-by-Colsterworth , Lincolnshire, England scalar potential energy function V: Died 31 March 1727 [OS: 20 March 1727] [1] Kensington , London, England x =− dV x Resid England ence m ¨ Natio English dx nality Field Physicist, mathematician, astronomer, natural philosopher, and alchemist Instit University of Cambridge ution Alma Trinity College, University of Cambridge mater Know Newtonian mechanics n for Universal gravitation Infinitesimal calculus Classical optics http://en.wikipedia.org/wiki/Sir_Isaac_Newton
Damping : is a force that is opposed to movement, and is proportional to velocity: x =− dV x m ¨ − γ ˙ x dx If γ >> m we have the Aristotle’s Law of motion (as you can see, I only use recent references!): x =− dV x γ ˙ dx Name: Aristotle Birth: 384 BC Death: March 7, 322 BC School/tradition: Inspired the Peripatetic school and tradition of Aristotelianism Main interests: Politics, Metaphysics, Science, Logic, Ethics http://en.wikipedia.org/wiki/Aristotle Notable ideas: The Golden mean, Reason, Passion Influences: Plato Influenced: Almost all of western philosophy and science afterward
Take γ = 1. Consider the possibility of thermal fluctuations. We then have: x =− dV x ε t ˙ dx where the last term represents thermal noise. This is a model for a particle in the presence of a potential energy in a viscous fluid at a given temperature (Langevin equation)… with all sort of bio – inspired and socio – inspired algorithms for global optimization around, it may be a good idea to search for new physics – inspired algorithms.
V x 1 , x 2 , ... , x N Consider a cost function . This can be imagined as a collection of N interacting particles in a potential, under the presence of a viscous fluid and at finite temperature. As time evolves, these kind of physical systems go to a state of thermal equilibrium. In practice, this means that for large enough times, the most likely positions of the particles are those close to the global minimum of the cost function.
From the Chapman – Kolmogorov equation of stochastic processes, an equation for the probability density of the Langevin system can be found: p x This is known as the Fokker – Planck equation. Notice that is linear for the density. At infinite times, the time derivative vanishes. The maximization of p at infinite times is a task essentially as difficult as the minimization of V . My proposal is not to maximize p , but to use the Fokker – Planck equation to construct a suitable random number generator. Suppose that y is a uniformly distributed random variable. We can combine the Fokker – Planck equation with the fundamental rule for the transformation of probability densities: p x dx = g y dy
Let’s consider at first sight simple restrictions in our optimization problem, of the form . L 1n ≤ x n ≤ L 2n Due to the fact that the previous differential equation is linear, it turns out that its solution can be approximated by a linear combination of functions from a complete set, like by the evaluation of the equations in N(L-1) points and the solution of NL linear algebraic equations. Tasks of these kinds are far more simpler than the minimization of V.
Example: Michalewicz’s function: 0 ≤ x n ≤ π The search space is and m = 10 .
Michalewicz’s function is widely used as a test function for global optimization algorithms because the local structure gives little information on the location of the global optimum. Careful inspection shows that the global minimum is located in the neighborhood of (2.2, 1.5).
Other references (besides Aristotle and Newton): ?
Recommend
More recommend
