Stochastic Search and Diffusion Arturo Berrones Some basic physics: - - PowerPoint PPT Presentation

Stochastic Search and Diffusion Arturo Berrones

Some basic physics: Newton’s 2nd Law of motion: The force F can be expressed as the gradient of a scalar potential energy function V:

F=ma=m ¨ x

m ¨ x=−dV  x  dx

Newtonian mechanics Universal gravitation Infinitesimal calculus Classical optics Know n for Trinity College, University of Cambridge Alma mater University of Cambridge Instit ution Physicist, mathematician, astronomer, natural philosopher, and alchemist Field English Natio nality England Resid ence 31 March 1727 [OS: 20 March 1727][1]

Kensington, London, England

Died 4 January 1643 [OS: 25 December 1642][1]

Woolsthorpe-by-Colsterworth, Lincolnshire, England

Born

http://en.wikipedia.org/wiki/Sir_Isaac_Newton

Damping : is a force that is opposed to movement, and is proportional to velocity: If γ >> m we have the Aristotle’s Law of motion (as you can see, I only use recent references!):

m ¨ x=−dV  x  dx −γ ˙ x

Almost all of western philosophy and science afterward Influenced: Plato Influences: The Golden mean, Reason, Passion Notable ideas: Politics, Metaphysics, Science, Logic, Ethics Main interests: Inspired the Peripatetic school and tradition of Aristotelianism School/tradition: March 7, 322 BC Death: 384 BC Birth: Aristotle Name:

http://en.wikipedia.org/wiki/Aristotle

γ ˙ x=−dV  x  dx

Take γ = 1. Consider the possibility of thermal fluctuations. We then have: where the last term represents thermal noise. This is a model for a particle in the presence of a potential energy in a viscous fluid at a given temperature (Langevin equation)… with all sort of bio – inspired and socio – inspired algorithms for global

• ptimization around, it may be a good idea to search for new physics – inspired

algorithms.

˙ x=−dV  x  dx εt 

Consider a cost function . This can be imagined as a collection of N interacting particles in a potential, under the presence of a viscous fluid and at finite

• temperature. As time evolves, these kind of physical systems go to a state of

thermal equilibrium. In practice, this means that for large enough times, the most likely positions of the particles are those close to the global minimum of the cost function.

V  x1, x2 ,... , xN 

From the Chapman – Kolmogorov equation of stochastic processes, an equation for the probability density of the Langevin system can be found: This is known as the Fokker – Planck equation. Notice that is linear for the density. At infinite times, the time derivative vanishes. The maximization of p at infinite times is a task essentially as difficult as the minimization of V. My proposal is not to maximize p, but to use the Fokker – Planck equation to construct a suitable random number generator. Suppose that y is a uniformly distributed random variable. We can combine the Fokker – Planck equation with the fundamental rule for the transformation of probability densities:

p x 

p x dx=g y dy

Let’s consider at first sight simple restrictions in our optimization problem, of the form . Due to the fact that the previous differential equation is linear, it turns out that its solution can be approximated by a linear combination of functions from a complete set, like by the evaluation of the equations in N(L-1) points and the solution of NL linear algebraic equations. Tasks of these kinds are far more simpler than the minimization of V.

L1n≤xn≤L2n

Example: Michalewicz’s function: The search space is and m = 10.

0≤xn≤π

Michalewicz’s function is widely used as a test function for global optimization algorithms because the local structure gives little information on the location of the global optimum. Careful inspection shows that the global minimum is located in the neighborhood of (2.2, 1.5).

Other references (besides Aristotle and Newton):

?