Last time � Genetics and evolution � Genetic algorithms � Assignment 4 � Assignment 3 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Outline for today � Evolutionary computation � Overview � Genetic programming � Genetic algorithms � Aspects of evolution � Classifier systems 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Evolutionary Computation - History � Evolutionary programming � Fogel, Owens, and Walsh (1966) � Differs from genetic algorithms in three ways: • Representation: not constrained to be a string • No crossover • Different form of mutation, and typically reduced rate of mutation during a run � Evolution strategies � Rechenberg (1965,1973), Schwefel (1975,1977) � Independently developed � Slightly different way of selection and mutation compared to EP � Recombination is possible 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU 1
Evolutionary Computation - History � Genetic algorithms � John Holland (1960s) � Classifier Systems � John Holland (1976 ?) � A cross between a Post production system, a genetic algorithm, and a market economy � A hybrid nature: Both evolution and learning � Genetic programming � John Koza (1992) � Evolving of whole programs � Resembles GA, but program fragments are used instead of strings � LISP 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU The No Free Lunch Theorem � ”The NFL theorem states that over all possible search spaces, all methods perform equally well, including the simple technique of randomly guessing.” – Flake � No single method of optimization is best for all applications � Evolutionary algorithms performs relatively well when: � there is a large number of parameters to be determined � the surface of solutions is complex, having many intermediate optima 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Genetic Programming � An attempt to deal with one of the central questions in computer science (posed by Arthur Samuel in 1959), namely � How can computers learn to solve problems without being explicitly programmed? In other words, how can computers be made to do what needs to be done, without being told exactly how to do it? � Any computer program can be graphically depicted as an rooted point-labeled tree with ordered branches � The search space in genetic programming is the space of all possible computer programs composed of functions and terminals appropriate to the problem domain 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU 2
Genetic Programming - Steps � In applying genetic programming to a problem, there are five major preparing steps: � The set of terminals � The set of primitive functions � The fitness measure � The parameters for controlling the run � The method for designating a result and the criterion for terminating a run � Start with an initial population of randomly generated computer programs 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Genetic Programming - Example � Koza, Rice, and Roughgarden (1992) � Foraging strategies of Anolis lizards � Questions: � ”What makes for an optimal foraging strategy?” � ”How can an evolutionary process assemble strategies that require complex calculations from simple components?” 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Genetic Programming - Example � Four variables: � The abundance a of insects � The sprint velocity v of the lizard � The coordinate x , y of the insect in the lizard’s view � A strategy is a function of these variables that returns 1 or -1 � The goal: A function that maximizes food capture per unit time 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU 3
Genetic Programming - Example � 10 x 20 meter viewing area (fig 1a) � Region 1: Insects always escape � Region 2: Insects never escape � Region 3: Insects escape with probability zero on the x axis and linearly increasing with the angle to a maximum of 0.5 on the y axis � Result, the best individual at generation � 0 (fig 1b) � 12 (fig 1c) � 46 (fig 1d) 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Genetic Algorithms - Schema � How do genetic algorithm’s work? � The Schema Theorem (Holland, 1975) 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Genetic Algoritms – Example: Coevolution � Hillis (1990) � Host-parasite coevolution � Adaptation in a static environment results in � loss of diversity � overfit solutions � Problem: � Evolving minimal sorting networks for sorting lists with a fixed number n of elements � Ex: (3,8), (14,8), (4,9), ... � With n = 16, best known solution is 60 comparisons 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU 4
GA – Example: Coevolution � Hillis used a GA � Two criteria for networks in the population � Small size, implicitly favored through the encoding � Correctness, explicitly through the fitness function � The fitness of a network, equal to the percentage of correctly sorted cases � Spatial implementation, each individual were placed on a two-dimensional lattice � Result (with static environment): � The GA got stuck on local optima � 65 comparisons 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU GA – Example: Coevolution � Reason: � After a while the test cases were not challenging enough � Solution: � Let the test cases evolve � The network’s fitness was the percentage of test cases in the parasite that it sorted correctly � The fitness of the parasite was the percentage of its test cases that the network sorted incorrectly � New result: � 61 comparisons 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU The Blind Watchmaker � 40% of all Americans (25% of college- educated Americans) do not believe in Darwinian evolution (M. Mitchell, 1999) � Richard Dawkins (1996) � ”Biomorphs” � A way to teach how evolution works � Variants � SimLife � Creatures 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU 5
Lamarckian Evolution � ”... the evolution of traits that are modified through experience and passed on, in their modified form, to the genotype of the next generation” – M. Mitchell � Not possible in natural systems � But artificial systems can use it � Needs a mean for adapting within a generation � and a way of passing new gains to the genotype of the next generation 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Lamarckian Evolution � Often more effective than Darwin evolution in static environments � Each individual can try out many possibilities in each generation � But, not so effective when the environment is dynamic 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU The Baldwin Effect � Also known as genetic assimilation � ”... If learning or other forms of adaptation during individuals’ lifetime are available, the desired configuration can arise via these mechanisms, and while the trait itself will not be passed on to offspring, the genetic background producing it will be favored. Thus, according to Baldwin, learning and other forms of within-lifetime adaptation can lead to increased survival, which can eventually lead to genetic variation that produces the trait genetically.” – M. Mitchell 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU 6
Classifier Systems � Adaptation � Learning – in the lifetime of the agent � Evolution – across generations � What about adaptation in systems between learning and evolution � Culture � Social � Economic � Classifier systems combine � Genetic algorithms � Environmental feedback � Simple reinforcement learning 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Feedback and Control � Visible features usually correspond to a subset of environment � Reinforcement � What differs adaptive systems from non-adaptive � Delayed rewards and punishments � How does one find the optimal controller? 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Classifier Systems � Rules � if condition then action � Production, Expert, and Classifier systems � Classifier systems � Are mostly used to control-like problems � Almost never ”programmed” 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU 7
Classifier Systems � A classifier system consists of � List of classifiers • condition : message : strength • Ex: 1#0#:1001:37 � List of messages • Messages describe the ”current” environment • Temporary storage space • Actions to take � Detectors • Sensory organs, post on the message list � Effectors • Can be used to modify the environment 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Classifier Systems 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU Classifier Systems The effectors place messages on the message 1. list A match set is formed from all suitable 2. classifiers The classifiers bid against each other. A function 3. of strength and maybe specificity. An action set is formed from the highest bidders The classifiers in the action set pay a portion of 4. their bids to the other classifiers (if any) that were responsible for posting the message that matched their condition. The paid classifiers have their strengths increased as a result 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU 8
Recommend
More recommend
