Last time Genetics and evolution Genetic algorithms 4/2 - 10 - PDF document

Last time � Genetics and evolution � Genetic algorithms 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Outline for today � Evolutionary computation � Overview � Genetic programming � Genetic algorithms � Genetic algorithms � Aspects of evolution � Classifier systems 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Evolutionary Computation - History � AI and ALife � Alan Turing, John von Neumann, Norbert Wiener � Self-replicate and adaptivity � Evolutionary programming � Fogel, Owens, and Walsh (1966) � Differs from genetic algorithms in three ways: • Representation: not constrained to be a string • Representation: not constrained to be a string • No crossover • Different form of mutation, and typically reduced rate of mutation during a run � Evolution strategies � Rechenberg (1965,1973), Schwefel (1975,1977) � Independently developed � Slightly different way of selection and mutation compared to EP � Recombination is possible 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 1

Evolutionary Computation - History � Genetic algorithms � John Holland (1960s) � Classifier Systems � John Holland (1976 ?) � A cross between a Post production system, a genetic algorithm and a market economy algorithm, and a market economy � A hybrid nature: Both evolution and learning � Genetic programming � John Koza (1992) � Evolving of whole programs � Resembles GA, but program fragments are used instead of strings � LISP 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU The No Free Lunch Theorem � ”The NFL theorem states that over all possible search spaces, all methods perform equally well, including the simple technique of randomly guessing.” – Flake � No single method of optimization is best for all � No single method of optimization is best for all applications � Evolutionary algorithms performs relatively well when: � there is a large number of parameters to be determined � the surface of solutions is complex, having many intermediate optima 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Genetic Programming � An attempt to deal with one of the central questions in computer science (posed by Arthur Samuel in 1959), namely � How can computers learn to solve problems without being explicitly programmed? In other words, how can computers be made to do what needs to be done, without b in t ld x ctl h being told exactly how to do it? t d it? � Any computer program can be graphically depicted as an rooted point-labeled tree with ordered branches � The search space in genetic programming is the space of all possible computer programs composed of functions and terminals appropriate to the problem domain 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 2

Genetic Programming - Steps � In applying genetic programming to a problem, there are five major preparing steps: � The set of terminals � The set of primitive functions � The set of primitive functions � The fitness measure � The parameters for controlling the run � The method for designating a result and the criterion for terminating a run � Start with an initial population of randomly generated computer programs 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Genetic Programming - Example � Koza, Rice, and Roughgarden (1992) � Foraging strategies of Anolis lizards � Questions: � ”What makes for an optimal foraging strategy?” � What makes for an optimal foraging strategy? � ”How can an evolutionary process assemble strategies that require complex calculations from simple components?” 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Genetic Programming - Example � Four variables: � The abundance a of insects � The sprint velocity v of the lizard � The coordinate x , y of the insect in the lizard’s view � A strategy is a function of these variables that returns 1 or -1 � The goal: A function that maximizes food capture per unit time 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 3

Genetic Programming - Example � 10 x 20 meter viewing area (fig 1a) � Region 1: Insects always escape � Region 2: Insects never escape � Region 3: Insects escape with probability zero on the x axis and linearly increasing with the angle to a maximum of 0.5 on the y axis � Result, the best individual at generation � 0 (fig 1b) � 12 (fig 1c) � 46 (fig 1d) 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Genetic Algorithms - Schema � How do genetic algorithm’s work? � The Schema Theorem (Holland, 1975) 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Genetic Algoritms – Example: Coevolution � Hillis (1990) � Host-parasite coevolution � Adaptation in a static environment results in � loss of diversity � loss of diversity � overfit solutions � Problem: � Evolving minimal sorting networks for sorting lists with a fixed number n of elements � Ex: (3,8), (14,8), (4,9), ... � With n = 16, best known solution is 60 comparisons 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 4

GA – Example: Coevolution � Hillis used a GA � Two criteria for networks in the population � Small size, implicitly favored through the encoding � Correctness, explicitly through the fitness function � The fitness of a network, equal to the percentage of correctly sorted cases � Spatial implementation, each individual were placed on a two-dimensional lattice � Result (with static environment): � The GA got stuck on local optima � 65 comparisons 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU GA – Example: Coevolution � Reason: � After a while the test cases were not challenging enough � Solution: � Let the test cases evolve � The network’s fitness was the percentage of � The network s fitness was the percentage of test cases in the parasite that it sorted correctly � The fitness of the parasite was the percentage of its test cases that the network sorted incorrectly � New result: � 61 comparisons 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU The Blind Watchmaker � 40% of all Americans (25% of college- educated Americans) do not believe in Darwinian evolution (M. Mitchell, 1999) � Only 40% of all Americans accept the evolution theory (Aftonbladet, 090201) y � Richard Dawkins (1996) � ”Biomorphs” � A way to teach how evolution works � Variants � SimLife � Creatures 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 5

Lamarckian Evolution � ”... the evolution of traits that are modified through experience and passed on, in their modified form, to the genotype of the next generation” – M. Mitchell � Not possible in natural systems � Not possible in natural systems � But artificial systems can use it � Needs a mean for adapting within a generation � and a way of passing new gains to the genotype of the next generation 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Lamarckian Evolution � Often more effective than Darwin evolution in static environments � Each individual can try out many possibilities in each generation � But, not so effective when the environment , ff is dynamic 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU The Baldwin Effect � Also known as genetic assimilation � ”... If learning or other forms of adaptation during individuals’ lifetime are available, the desired configuration can arise via these mechanisms, and while the trait itself will not be passed on to while the trait itself will not be passed on to offspring, the genetic background producing it will be favored. Thus, according to Baldwin, learning and other forms of within-lifetime adaptation can lead to increased survival, which can eventually lead to genetic variation that produces the trait genetically.” – M. Mitchell 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 6

Classifier Systems � Adaptation � Learning – in the lifetime of the agent � Evolution – across generations � What about adaptation in systems between learning and evolution � Culture � Culture � Social � Economic � Classifier systems combine � Genetic algorithms � Environmental feedback � Simple reinforcement learning 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Feedback and Control � Visible features usually correspond to a subset of environment � Reinforcement � What differs adaptive systems from non-adaptive � Delayed rewards and punishments � How does one find the optimal controller? 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU Classifier Systems � Rules � if condition then action � Production, Expert, and Classifier systems � Classifier systems � Classifier systems � Are mostly used to control-like problems � Almost never ”programmed” 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 7