Genetic Algorithms in Robotics Julius Mayer Universit at Hamburg - PowerPoint PPT Presentation

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg GA’s in Robotics Genetic Algorithms in Robotics Julius Mayer Universit¨ at Hamburg Fakult¨ at f¨ ur Mathematik, Informatik und Naturwissenschaften Fachbereich Informatik Technische Aspekte Multimodaler Systeme October 31, 2016 J. Mayer 1

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg GA’s in Robotics Outline 1. Introduction Motivation Classification 2. Algorithm Overview Phases 3. Application GA’s in Robotics Neuroevolution 4. Discussion Evaluation Conclusion J. Mayer 2

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Introduction - Motivation GA’s in Robotics Motivation Problems that are hard to solve with classical optimization because of ◮ too many parameters (intractability), ◮ large search space, ◮ non-differentiable objective functions, ◮ varying numbers of variables within the optimization process, ◮ a lacking mathematical function specification. [1] J. Mayer 3

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Introduction - Classification GA’s in Robotics Classification Biologic: Theory of evolution ◮ Adaptation of population to the environment ◮ Gradual, hereditary change in the individuals of a species ◮ Information storage and transfer through genomes ◮ Optimizing population fitness by reproduction [3] J. Mayer 4

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Introduction - Classification GA’s in Robotics Classification Biologic: Theory of evolution ◮ Adaptation of population to the environment ◮ Gradual, hereditary change in the individuals of a species ◮ Information storage and transfer through genomes ◮ Optimizing population fitness by reproduction [5] J. Mayer 5

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Introduction - Classification GA’s in Robotics Classification Algorithmic: Stochastic optimization ◮ Can ’solve’ NP-hard problems ◮ Solution approximation ◮ Computational complexity as prohibiting factor ◮ e.g. ant colony & particle swarm optimization [7] J. Mayer 6

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Algorithm - Overview GA’s in Robotics Algorithm 1. Initialization 2. Evaluation 3. Selection 4. Mutation 5. Terminate J. Mayer 7

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Algorithm - Phases GA’s in Robotics Phase 1. Initialization Initialize random population ◮ Choose appropriate representation ◮ Individuals as encoded problem solutions ◮ e.g. real values or binary strings 1,67 2,34 1,04 5,83 2,10 101010011011010010101 [12] J. Mayer 8

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Algorithm - Phases GA’s in Robotics Phase 2. Evaluation ◮ Choose objective function ◮ Decode & evaluate chromosome ◮ Find minima in objective function ◮ Survival of the fittest [9] J. Mayer 9

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Algorithm - Phases GA’s in Robotics Phase 3. Selection ◮ Probabilistic selection ◮ Roulette wheel selection ◮ Tournament Selection ◮ Balance exploration & exploitation dilemma J. Mayer 10

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Algorithm - Phases GA’s in Robotics Phase 4. Mutation ◮ Assign mutation rate ◮ Mutate ◮ Bitflips ◮ Normal distribution mutation ◮ Recombination [12] ◮ Crossover ◮ Permutation ◮ Mutation drives change J. Mayer 11

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Algorithm - Phases GA’s in Robotics Phase 4. Mutation ◮ Assign mutation rate ◮ Mutate ◮ Bitflips ◮ Normal distribution mutation ◮ Recombination ◮ Crossover [12] ◮ Permutation ◮ Mutation drives change J. Mayer 12

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Algorithm - Phases GA’s in Robotics Phase 5. Termination ◮ If solution is sufficiently close ◮ Performance is satisfying ◮ Change is stagnant J. Mayer 13

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Application - GA’s in Robotics GA’s in Robotics GA’s in Robotics ◮ Evolutionary robotics ◮ Path planing ◮ Multivariate parameter optimization ◮ Evolving artificial network architectures & optimizing connection weights (learning) [6] J. Mayer 14

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Application - Neuroevolution GA’s in Robotics Neuroevolution of augmenting topologies (NEAT) ◮ Uses crossover ◮ Uses speciation to protect structures ◮ Increment growth from minimal structures [10] J. Mayer 15

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Application - Neuroevolution GA’s in Robotics NEAT: Mutation ◮ Connection weights ◮ Network structure ◮ Add new connection (random weight) ◮ Add new node (split connection) [10] J. Mayer 16

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Application - Neuroevolution GA’s in Robotics NEAT: Crossover ◮ Genes are lined up according to innovation number ◮ Matching Genes are randomly chosen ◮ Disjoint / excess are taken from fitter parent [10] J. Mayer 17

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Application - Neuroevolution GA’s in Robotics NEAT for RoboCup Soccer Keepaway (robot soccer subtask) ◮ Large state space (unable to explore exhaustively) ◮ Only partial state information for each agent ◮ Continuous action space ◮ Multiple teammates need to learn simultaneously [4] J. Mayer 18

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Application - Neuroevolution GA’s in Robotics NEAT vs. Temporal Difference Methods [11] J. Mayer 19

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Application - Neuroevolution GA’s in Robotics NEAT vs. Temporal Difference Methods [11] J. Mayer 20

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Discussion - Evaluation GA’s in Robotics Evaluation Challenges Drawbacks ◮ Get GA’s parameters right ◮ Approximation instead of ◮ Solution encoding exact solution ◮ Exploration vs. exploitation ◮ Computational complexity as ◮ Premature convergence prohibiting factor (local minima) ◮ No guaranteed convergence ◮ slow convergence to global optimum ◮ big search space ◮ Non-exhaustive coverage of ◮ Find representative fitness the complete solution space function J. Mayer 21

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Discussion - Conclusion GA’s in Robotics Conclusion ◮ Inspired by biological evolution ◮ Stochastic optimization ◮ Population containing encoded problem solutions (parameters) ◮ Different applications in robotics ◮ e.g. evolving ANN (NEAT) for RoboCup Soccer ◮ No guarantee for convergence to [2] global optimum J. Mayer 22

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Discussion - Conclusion GA’s in Robotics References [1] http://rednuht.org/genetic walkers/, 2016. [2] https://cdn.meme.am/instances/41036988.jpg. 2016. [3] https://i.ytimg.com/vi/3ZJNyScv8to/maxresdefault.jpg, 2016. [4] https://i.ytimg.com/vi/HHlN0TDgllE/maxresdefault.jpg. 2016. [5] http://www.hanskottke.de/wordpress/wp-content/uploads/2012/09/evolutions hip hop.jpg. 2016. [6] http://www.orocos.org. 2016. [7] http://www.turingfinance.com/wp-content/uploads/2014/10/Self-Organizing-Feature-Map-3.png. 2016. [8] John H Holland. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. U Michigan Press, 1975. [9] Melanie Mitchell. Complexity: A Guided Tour. In Oxford , volume 1, chapter Cellular A, pages 145–159. Oxford University Press, Inc, New York, 2009. J. Mayer 23

MIN-Fakult¨ at Fachbereich Informatik Universit¨ at Hamburg Discussion - Conclusion GA’s in Robotics References (cont.) [10] Kenneth O Stanley and Risto Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary computation , 10(2):99–127, 2002. [11] Matthew E Taylor. Comparing Evolutionary and Temporal Difference Methods in a Reinforcement Learning Domain. pages 1321–1328. [12] Mattias Wahde. Biologically Inspired Optimization Methods: An introduction . WIT Press, Boston, MA, 2008. J. Mayer 24