Evolving Neural Networks Keith L. Downing The Norwegian University - PowerPoint PPT Presentation

Evolving Neural Networks Keith L. Downing The Norwegian University of Science and Technology (NTNU) Trondheim, Norway keithd@idi.ntnu.no March 11, 2014 Keith L. Downing Evolving Neural Networks

A Brief History of Artificial Intelligence Early (1955 - 1980) focus (and success) on tasks that humans find difficult: chess, geometry, physics... Later (1985- 2010) focus on easy human tasks, which are hard for computers. In the 1980’s, it became clear that computers lack common sense, and it’s not easy to give it to them in the same way that we give them high-level, expert knowledge of a specific domain. In the 1990’s, Situated and Embodied AI (SEAI) recognized as a promising low road to intelligence. Computers will only acquire common sense about the world by experiencing it and having to survive in it. Keith L. Downing Evolving Neural Networks

Cognitive Incrementalism Llinas (pg. 35) ...that which we call thinking is the evolutionary internalization of movement.. Mindware (pg. 135), Andy Clark, 2001 This is the idea that you do indeed get full-blown human cognition by gradually adding bells and whistles to basic (embodied and embedded) strategies of relating to the present at hand. Could sensorimotor control be the basis of common sense ? Is it the key to Artificial General Intelligence (AGI)? Keith L. Downing Evolving Neural Networks

Robot (Hans Moravec) Comparative Evolution Living Organisms Computers Sense & Act 10,000,000 years 25 years Reason 100,000 years 40 years Calculate 1,000 years 60 years Neural circuitry for cognition reuses, extends and is constrained by sensorimotor circuitry. Should AGI progress and be restricted in similar ways? Keith L. Downing Evolving Neural Networks

Artificial Neural Networks (ANNs) Utility for AGI via Cognitive Incrementalism Simple, homogeneous substrate Same, basic, neural signals carry information of perceptual, cognitive and motor nature - - no need for special representations for each aspect of intelligence. Relatively unbiased. Adapt to represent the salient aspects of a situation. Built for learning. Keith L. Downing Evolving Neural Networks

Training Artificial Neural Networks Training/Test Cases: {(d1, r1) (d2, r2) (d3, r3)....} d3 r3 Encoder Decoder r* E = r3 - r* dE/dW Cases N times, with learning Training Neural Net Test 1 time, without learning Keith L. Downing Evolving Neural Networks

Backpropagation Advantages Powerful tool for learning complex input-output mappings in diverse problem domains. Relatively simple algorithm with solid mathematical foundation. Drawbacks Requires a known, correct output for each input → impractical for training autonomous systems. Requires many training rounds, often hundreds or thousands. Can easily get stuck in local error minima during gradient descent. Recurrent networks are a problem. Biologically unrealistic Keith L. Downing Evolving Neural Networks

Evolving Artificial Neural Networks (EANNs) Genome (Direct Encoding) 0 0 1 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 Encoder Decoder Cases Fitness Test Total Neural Error Net 1 time, without learning Keith L. Downing Evolving Neural Networks

Three Levels of Adaptation POE Systems Phylogenetic or Evolutionary - Characterized by the use of 1 an EA and thus having clearly definable genotypic and phenotypic levels, genetic operators, fitness functions, etc. Ontogenetic or Developmental - Involving a non-trivial 2 genotype-to-phenotype translation. In most cases, the genotype is a recipe that, through some recursive growth process, produces the phenotype. Epigenetic or Learning - During actual performance 3 testing, the system is able to modify itself in some manner that effects future behavior. A.k.a. TRIDAP (3-way adaptive) Keith L. Downing Evolving Neural Networks

Genotypic Encodings Direct - Position (in chromosome) and bits determine a phenotypic trait, independent of all other genes. Indirect Bijective - Genes may interact in determining traits. Chromosomal position and/or bits may only be relative indicators. Indirect Generative - Genes encode parameters for development. Direct Encoding 1110001101011011..... Schedule the 3rd exam Indirect (Uncompressed) Encoding for the 10th time slot. ....1110001101011011..... Schedule the next unscheduled exam for the 10th of the unfilled time slots. Generative (Developmental) Encoding 111010100011 7 5 12 Exam 1 => Slot 4 Exam 2 => Slot 8 ......... Keith L. Downing Evolving Neural Networks

Two Early Direct Encodings Phenotype Phenotype 1 2 1 2 +.45 +.55 -.32 +.89 ? ? ? ? 3 4 3 4 ? ? -.07 +.61 Recurrent connections ignored 5 5 Weights learned by Net assumed to be back-propagation fully connected From 1 2 3 4 5 1 0 0 1 0 0 Genotype 2 0 0 0 0 0 Connection To 3 0 1 0 0 1 +.45 -.32 +.89 +.55 -.07 +.61 Table 4 1 1 0 0 0 Montana & Davis (1989) 5 0 0 1 1 0 Genotype 0 0 1 0 0 0 0 0 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 0 Miller et. al. (1989) Keith L. Downing Evolving Neural Networks

The Competing Conventions Problem Genotypes 1-11 -111 -111 1-11 X Y X Y XOR(X,Y) XOR(X,Y) A B A B C C Crossover 1-11 1-11 -111 -111 X Y X Y X ∧¬Y Y ∧¬X A B A B C C Many-to-1 genotype → phenotype mapping can hinder evolution. Keith L. Downing Evolving Neural Networks

Evolving Individual Neurons: the SANE Approach Genotype => Phenotype Mapping 0 1 2 3 Genome for Node A Weight -0.6 0.5 0.2 Genome for Node C 2 0.5 33 -0.6 130 1.2 8 0.2 311 -1.0 150 0.4 A B C 1.2 -1.0 0.4 Index Index < 127 => Input Else => Output 0 1 Moriarty & Mikkulainen (1997). Still a direct representation. Keith L. Downing Evolving Neural Networks

Cooperative Coevolution of Neurons in SANE Generation K Neurons Create Networks Assign Evaluate fitness to Networks neurons Generation K+1 Selection, Neurons mutation & recombination of neurons Neuron fitness is based on the ability to combine well (i.e. cooperate with) other neurons in forming a good neural network Circumvents competing conventions by never linking neurons together on a chromosome; it just allows good combinations to form dynamically during fitness testing. Keith L. Downing Evolving Neural Networks

Complexification ??? Primates Fishes Mammals Humans Birds Reptiles Insects Bacteria Viruses Amphibians Complexity Evolution does not necessarily favor increased complexity. Evolution searches all over the complexity spectrum, but there seem to be clear LOWER limits of complexity. Evolution found those early but continues to stretch the upper limits. Full House , Stephen Jay Gould (1996). In EANNs, it’s hard to begin with large, complex genomes; all are unfit. Can we allow genomes to gradually complexify? This entails dynamic and variable chromosome sizes. Keith L. Downing Evolving Neural Networks

Duplication and Differentiation Genotype A A B C A B C Duplication F F G H F G H Differentiation Phenotype A A' B C A A* B C Further Differentiation F U G H F X G H Useless Useful NEW Function A low-risk route to complexification, since key functionalities (e.g. F) are still present during the exploratory period when variants of A arise and their phenotypic consequences are tested . Keith L. Downing Evolving Neural Networks

Neural Complexification via Modularity, Duplication & Differentiation !"#$%& '&()*+,+- Hox Genes: a conserved modular component Evolving Brains (J. Allman, 1999) Evolution by Gene Duplication (S. Ohno, 1970) Keith L. Downing Evolving Neural Networks

Vertebrate Brain Archetype 6"3-.& 1"23$* 8%9&230(: ;$%) !"#$%%& +,-.&%/'0(# '"(")"%%$* 7&%%-$* 4-".2",5&%0. Principles of Brain Evolution (G. Streidter, 2005) Keith L. Downing Evolving Neural Networks

Problems with Complexification and Dynamic, Variable Genome Size Parents Children A B C* A B B C X A A* B C A A* C* A B C A B C* X A A* B C* A A* B C A B A* A B C* X A B C C* A B C A* Missing genes and copies of same or similar (i.e. from same ancestor) genes. Partially remedied by history-based alignment. Keith L. Downing Evolving Neural Networks

Neurevolution of Augmenting Topologies (NEAT) Genotype 1 2 3 4 5 Nodes Input Input Output Hidden Hidden Connections 1 => 5 2 => 4 5 => 3 1 => 3 4 => 3 2 => 5 4 => 5 W: 0.3 W: 0.7 W: -0.6 W: 0.5 W: -0.1 W: 0.9 W: 0.2 Phenotype 1 2 Input 0.3 0.7 0.9 4 0.2 5 Hidden 0.5 -0.1 -0.6 3 Output Stanley and Miikkulainen (2002) Historical tags + speciation allow gradual complexification. Classic version restricted to one hidden layer but many connection schemes Extremely popular (direct encoding) approach to EANNs. Basis for NERO war game (Stanley et. al., 2005). Keith L. Downing Evolving Neural Networks

Cartesian Genetic Programming (Miller, 2000, 2011) F1 C1 F2 C2 Fn Cn O1 Ok i w s i w s 1 2 0.3 1 4 0.7 1 2 1 -0.4 1 3 0.9 1 1 5 0.6 1 4 1.0 1 6 0.7 Input 1 4 Output 1.0 0.3 6 0.6 2 5 -0.4 3 0.9 Beats SANE, ESP and NEAT on several benchmarks. (Khan et. al., 2013) Keith L. Downing Evolving Neural Networks