Evolutionary Algorithms General Concepts Prof. Thomas Bck Natural - PowerPoint PPT Presentation

Evolutionary Algorithms General Concepts Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 1

Overview • Taxonomy of EAs • Key Features • Literature etc. Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 2

sadsad TAXONOMY Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 3

Introduction Input: Known (measured) Output: Known (measured) • Modeling Interrelation: Unknown ! ! ! ??? ! ! ! Input: Will be given • Simulation How is the result for the input? ! ! ! ! ! ! ??? • Optimization Model: Already exists ??? ! ! ! ! ! ! Objective: Will be given How (with which parameter settings) to achieve this objective? Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 4

Evolutionary Algorithms Taxonomy Evolutionary Algorithms Evolutio Genetic nary Genetic Algorithms Progra Progra Evolutionary Strategies (ES) mming (GA) mming (GP) (EP) Real- Order- Canoni Messy Derand (1, l ) ( µ, l ) coded based (1+1) CMA cal GAs GAs omized GAs GAs Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 5

Key features Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 6

Key Features ES vs GA Evolutionary Strategies Genetic Algorithms Mixed-integer capabilities Discrete representations Emphasis on mutation Emphasis on crossover Self-adaptation No self-adaptation Small population sizes Larger population sizes Deterministic selection Probabilistic selection Developed in Europe Developed in US Theory focused on convergence Theory focused on schema speed processing Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 7

Key Features of EAs (1) • Set of candidate solutions ( individuals ), forming a population . • Generating new solutions by: Reproduction : Copying an individual − Crossover (recombination): − 2 (or more) parents à 1 (or more) offspring Mutation : − 1 parent à 1 offspring Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 8

Key Features of EAs (2) • Quality measure of individuals: Fitness function , objective function . • Survival-of-the-fittest principle • Mating-of-the-fittest principle • Algorithm cycles through iterations (generations) Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 9

Main Components of EAs 1. Representation of individuals: Coding 2. Evaluation method for individuals: Fitness Initialization procedure for 1 st generation 3. 4. Definition of variation operators ( mutation, crossover ) 5. Parent (mating) selection mechanism 6. Survivor (environmental) selection mechanism 7. Technical parameters mutation rates − population size − crossover rates − Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 10

Generalized EA Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 11

Generalized Evolutionary Algorithm t := 0; initialize (P(t)); evaluate (P(t)); while not terminate do P‘(t) := mating_selection (P(t)); P‘‘(t) := variation (P‘(t)); evaluate (P‘‘(t)); P(t+1) := environmental_selection (P‘‘(t) È P(t)); t := t+1; od Variation summarizes mutation and recombination Environmental selection can take old parents into account! Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 12

Advantages of EAs 1. Widely applicable, also in cases where no good solution techniques are available Multimodalities, discontinuities, constraints − Noisy objective functions − Multiple criteria decision making problems − Implicitly defined problems (simulation models) − 2. No presumptions w.r.t. problem space 3. Low development costs, i.e., costs to adapt to new problem spaces 4. The solutions of EAs have straightforward interpretations 5. Can run interactively, always deliver solutions 6. Self-adaptation of strategy parameters Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 13

Disadvantages of EAs 1. No guarantee for finding optimal solutions within a finite amount of time. This is true for all global optimization methods. 2. No complete theoretical basis (yet), but much progress is being made. 3. Parameter tuning is sometimes based on trial and error. Solution: Self-adaptation of strategy parameters − Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 14

Two Views on EAs 1. Global random search methods Probabilistic search with high “creativity” − Diversified search − Applying local search operators − 2. Nature based search techniques Stochastic influence − Population based − Adaptive behavior − Recognizing/amplifying strong gene patterns − Exploration Exploitation Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 15

Numerical Function Optimization Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 16

History, literature Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 17

History (Incomplete) • Friedberg et al. 1958: Automatic programming, binary representations Bremermann 1958: Binary/continuous variables, recombination and • mutation, estimation p*=1/l (optimal mutation rate). Box 1957: Evolutionary operation, fractional design of experiments. • L.J. Fogel 1962 (San Diego, CA): Evolutionary Programming , finite state • machines, mutation only. H.-P. Schwefel and I. Rechenberg 1965 (Berlin, Germany): Evolution • Strategies , experimental optimization, (1+1)-strategy, mutation only. J. H. Holland 1962 (Ann Arbor, MI): Genetic Algorithms , binary variables, • mutation and recombination J. Koza 1989: Genetic Programming , S-expressions − Holland et al., ca. 1985: Classifier Systems , classification and induction − rules. Pittsburgh vs. Michigan approach. Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 18

Literature (Incomplete) • Th. Bäck: Evolutionary Algorithms in Theory and Pract ice, Oxford University Press, NY, 1996. Th. Bäck, D.B. Fogel, Z. Michalewicz: Handbook of Evolutionary Computation , • Vols. 1, 2, Institute of Physics Publishing, Bristol, UK, 2003. A.E. Eiben, J. Smith: Introduction to Evolutionary Computing , Natural • Computing Series, Springer, Berlin, 2003. D.B. Fogel: Evolutionary Computation: Toward a new philosophy of machine • intelligence , IEEE Press, 1995. Z. Michalewicz: Genetic Algorithms + Data Structures = Evolution Programs , • Springer, 1996. • H.-P. Schwefel: Evolution and Optimum Seeking , Wiley, NY, 1995. Prof. Thomas Bäck Natural Computing Group Evolutionary Algorithms Optimization 19