Towards Efficient Evolutionary Design of Autonomous Robots Peter Krˇ cah Computer Center, Charles University Ovocn´ y Trh 5, 116 36 Prague 1 Czech Republic peter.krcah@ruk.cuni.cz September 23, 2008
Introduction ◮ Ultimate goal : to automatically design hardware and software for robots capable of performing given tasks ◮ Previous works : Virtual Creatures (Sims 1994), GOLEM (Lipson, Pollack 2000), Genobots (Hornby, Pollack, 2002), Miconi (2006) ◮ Fitness evaluation done in simulation ◮ Simulation is slow (only about 10x faster than reality) ◮ Single run takes typically several hours/days ◮ Possible solutions ◮ Increase computational power ◮ Use better algorithms
Introduction ◮ Ultimate goal : to automatically design hardware and software for robots capable of performing given tasks ◮ Previous works : Virtual Creatures (Sims 1994), GOLEM (Lipson, Pollack 2000), Genobots (Hornby, Pollack, 2002), Miconi (2006) ◮ Fitness evaluation done in simulation ◮ Simulation is slow (only about 10x faster than reality) ◮ Single run takes typically several hours/days ◮ Possible solutions ◮ Increase computational power ◮ Use better algorithms
Robot Morphology ◮ Morphology represented by a directed graph ◮ Each node represents a body part ◮ Each connection represents a physical joint between two parts ◮ Generative encoding inspired by Sims’ Virtual Creatures ◮ Phenotype is constructed by recursive traversal of the connections of the genotype ◮ Recursive limit in each node prevents infinite cycles
Robot Control System ◮ Control system is distributed along the robot body ◮ Each body part contains a local feedforward neural network , sensors and an effector ◮ Each neuron has one of 22 transfer functions ◮ Single independent neural network for central coordination
Problem Definition ◮ Robot is represented by a morphology graph with a nested neural network in each node: ◮ Problem 1 : How to do effective recombination ? ◮ Intuition: We need to swap corresponding parts ◮ Existing approaches: Grafting, crossover (Sims), picking (Miconi) ◮ Problem 2 : How to do define similarity measure ? ◮ We need to find correspondence of structure elements ◮ Useful feature for advanced speciation techniques
Problem Definition ◮ Robot is represented by a morphology graph with a nested neural network in each node: ◮ Problem 1 : How to do effective recombination ? ◮ Intuition: We need to swap corresponding parts ◮ Existing approaches: Grafting, crossover (Sims), picking (Miconi) ◮ Problem 2 : How to do define similarity measure ? ◮ We need to find correspondence of structure elements ◮ Useful feature for advanced speciation techniques
Problem Definition ◮ Robot is represented by a morphology graph with a nested neural network in each node: ◮ Problem 1 : How to do effective recombination ? ◮ Intuition: We need to swap corresponding parts ◮ Existing approaches: Grafting, crossover (Sims), picking (Miconi) ◮ Problem 2 : How to do define similarity measure ? ◮ We need to find correspondence of structure elements ◮ Useful feature for advanced speciation techniques
Historical Markings ◮ Introduced by K. Stanley in 2002 as a part of NEAT algorithm ◮ Algorithm for evolving neural networks ◮ Main principles : 1. Each new structure element created during evolution is assigned a unique number (a historical marking) 2. Markings are inherited ◮ Properties : 1. We can track the genes during the evolution 2. Markings can be used as an age indicators
Historical Markings ◮ Introduced by K. Stanley in 2002 as a part of NEAT algorithm ◮ Algorithm for evolving neural networks ◮ Main principles : 1. Each new structure element created during evolution is assigned a unique number (a historical marking) 2. Markings are inherited ◮ Properties : 1. We can track the genes during the evolution 2. Markings can be used as an age indicators
Historical Markings ◮ Introduced by K. Stanley in 2002 as a part of NEAT algorithm ◮ Algorithm for evolving neural networks ◮ Main principles : 1. Each new structure element created during evolution is assigned a unique number (a historical marking) 2. Markings are inherited ◮ Properties : 1. We can track the genes during the evolution 2. Markings can be used as an age indicators
Historical Markings for Robots ◮ Historical markings used for: ◮ Nodes and connections in the morphology graph ◮ Neurons and neural connections inside each body part
Historical Markings for Robots ◮ Historical markings used for: ◮ Nodes and connections in the morphology graph ◮ Neurons and neural connections inside each body part
Historical Markings for Robots ◮ Historical markings used for: ◮ Nodes and connections in the morphology graph ◮ Neurons and neural connections inside each body part
Historical Markings for Robots ◮ Historical markings used for: ◮ Nodes and connections in the morphology graph ◮ Neurons and neural connections inside each body part
Crossover and Similarity Measure 1. Align morphological nodes based on their markings 2. Align neurons inside morphological nodes based on their markings 3. Do a regular crossover or compute similarity
Similarity Measure ◮ Similarity measure of A and B ◮ 0 ... if A and B are the same ◮ 1 ... if A and B are completely different ◮ Similarity of two robots ◮ The average similarity of their corresponding nodes and connections ◮ Similarity of two nodes ◮ The average similarity of their neurons and neural connections ◮ Genes without a counterpart count as if they have a completely different counterpart
HierarchicalNEAT ◮ Speciation ◮ Explicit fitness sharing (Goldberg, Richardson, 1987) ◮ Robots divided into species based on their similarity ◮ Fitness is shared among all robots in a single species ◮ HierarchicalNEAT (hNEAT) - An algorithm for evolving morphology and control of robotic organisms ◮ Historical markings for both morphology and neural networks ◮ Crossover and similarity measure based on historical markings ◮ Speciation using explicit fitness sharing
HierarchicalNEAT ◮ Speciation ◮ Explicit fitness sharing (Goldberg, Richardson, 1987) ◮ Robots divided into species based on their similarity ◮ Fitness is shared among all robots in a single species ◮ HierarchicalNEAT (hNEAT) - An algorithm for evolving morphology and control of robotic organisms ◮ Historical markings for both morphology and neural networks ◮ Crossover and similarity measure based on historical markings ◮ Speciation using explicit fitness sharing
Experimental Setup ◮ Fitness functions : walking, jumping, swimming, light-following ◮ Winning fitness value defined for each fitness function ◮ Performance measure : number of fitness evaluations needed to find the winner ◮ Evolution fails if winner not found in the first 30 000 fitness evaluations ◮ Each configuration tested 30 times for 100 generations ◮ Single evolution : 70 minutes on 7 computers ◮ All experiments : 8 months of CPU time ( < 4 days using 70 computers) ◮ Statistical significance of the differences verified ( p < 0 . 02)
Experimental Setup ◮ Fitness functions : walking, jumping, swimming, light-following ◮ Winning fitness value defined for each fitness function ◮ Performance measure : number of fitness evaluations needed to find the winner ◮ Evolution fails if winner not found in the first 30 000 fitness evaluations ◮ Each configuration tested 30 times for 100 generations ◮ Single evolution : 70 minutes on 7 computers ◮ All experiments : 8 months of CPU time ( < 4 days using 70 computers) ◮ Statistical significance of the differences verified ( p < 0 . 02)
Measuring Performance ◮ Comparison of standard GA with grafting and crossover and hNEAT ◮ Fitness functions : walking, swimming, jumping, light-following ◮ hNEAT consistently better than standard GA
Ablation Experiments ◮ Analysis of individual components of the proposed algorithm ◮ Verification of the usefulness of mating and speciation Configuration Mean Std Failed Full hNEAT 16704.53 6339.75 8% Non-mating hNEAT 22441.38 6146.31 20% Non-speciated hNEAT 20456.40 6313.65 20% ◮ Finding the best method for creating the initial population Configuration Mean Std Failed Random - 2 nodes 11503.77 5871.19 3% Random - 3 nodes 16704.53 6339.75 8% Random - 4 nodes 29920.69 387.05 93% Uniform - 2 nodes 16811.39 8190.81 21%
Ablation Experiments ◮ Analysis of individual components of the proposed algorithm ◮ Verification of the usefulness of mating and speciation Configuration Mean Std Failed Full hNEAT 16704.53 6339.75 8% Non-mating hNEAT 22441.38 6146.31 20% Non-speciated hNEAT 20456.40 6313.65 20% ◮ Finding the best method for creating the initial population Configuration Mean Std Failed Random - 2 nodes 11503.77 5871.19 3% Random - 3 nodes 16704.53 6339.75 8% Random - 4 nodes 29920.69 387.05 93% Uniform - 2 nodes 16811.39 8190.81 21%
Recommend
More recommend
