Motivation / Problem Optimization of behavior in respect of - PDF document

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Optimization of simulated biological multi-agent systems by means of evolutionary processes Alexander Hörnlein Christoph Oechslein Frank Puppe Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Motivation / Problem • Optimization of behavior in respect of – explicit evaluation function – implicit evaluation function e.g. “the agents have to survive a certain period” • Calibration towards a predefined target behavior e.g. “the agents should act exactly as in real life ” Optimization of simulated biological multi-agent systems 2 / 23 by means of evolutionary processes 1

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Evolution as optimization • Population of potential solutions • Evaluation by means of “natural selection” • Iteration: Survivors (i.e. highly fit individuals) reproduce Optimization of simulated biological multi-agent systems 3 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Reproduction • Mutation – Offspring differs slightly - possibly advantageous – local search • Recombination – Child possibly unites the advantages of both parents – global search Optimization of simulated biological multi-agent systems 4 / 23 by means of evolutionary processes 2

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Behavior in SeSAm Activity1 Activity2 Agent • Rules Activity3 IF (in activity1) AND • Activities Condition THEN activity3 Action1 • Parameters Action2 ... • Memory • Perception Optimization of simulated biological multi-agent systems 5 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg GP approach: Mutation operators • Add new activity • Change numeric terminals activity • Change symbolic terminals • Add new rule Parameter a += 10 Parameter a += 25 Parameter b += 25 • Change rule • Change non-terminals Approach agent x Flee from agent x • Delete activity • Delete action Increase speed Focus on earth • Delete rule • Add action Optimization of simulated biological multi-agent systems 6 / 23 by means of evolutionary processes 3

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Advantage Disadvantages • Extremely powerful • Development of unnecessary or unwanted • Little constraint by initial complexity structure of behavior • Restrictions are difficult to define/set • Slow • Hard to implement within SeSAm Optimization of simulated biological multi-agent systems 7 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg GA/ES approach: Mutation operators • Change numeric terminals activity Parameter a += 25 Parameter a += 10 that’s it in principle. Approach agent x Increase speed Optimization of simulated biological multi-agent systems 8 / 23 by means of evolutionary processes 4

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Applicability of GA/ES approach within SeSAm • Actions – Use numerical terminals – Can be controlled by probabilities • Rules – Condition-parts use numerical terminals – Action-parts can be controlled by probabilities Optimization of simulated biological multi-agent systems 9 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Model modification • Define rules for any reasonable transient • Let evolution weight them • Treat actions accordingly Optimization of simulated biological multi-agent systems 10 / 23 by means of evolutionary processes 5

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Advantages Disadvantage • Sufficient powerful • Not extremely powerful • Easy to restrict: Evolution can’t break boundaries of predefined behavior • Fast • Implementation within SeSAm is ‘straight-forward’ Optimization of simulated biological multi-agent systems 11 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg SeSAm genes RULE: IF ENERGY > gene0 THEN MOVE gene0: (initial) standard lower boundary deviation upper boundary (initial) value (initial) standard deviation distribution [ ] dominance lower upper (initial) boundary value boundary Optimization of simulated biological multi-agent systems 12 / 23 by means of evolutionary processes 6

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg SeSAm genomes role agent genome behavior declaration genome egg storage gene0 declaration gene1 declaration ... family attribute allele0-0 allele1-0 gene0 gene0 allele0-1 gene1 gene1 allele1-1 ... ... ... ... Optimization of simulated biological multi-agent systems 13 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Polyploid genome • Treated threadwise • Treated genewise dominance dominance mutation mutation Optimization of simulated biological multi-agent systems 14 / 23 by means of evolutionary processes 7

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Possibilities for the gene-expression • dominant/recessive • ‘intermediary’ value0 • weighted ( dominance ) value ω ⋅ 1 i i value value1 meta gene i expression value0 ⋅ i # alleles ( dominance ) ω i i i value2 Optimization of simulated biological multi-agent systems 15 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Application from individuals to colonies 8

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Insects’ behavior hunt set seek prey insects marker marker from own fight reservoir brood care transport to nest from nest idle reservoir mate feed feed on nest lay egg on brood reservoir seek grow new nest feed Optimization of simulated biological multi-agent systems 17 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Insects’ genes hunt-factor hunt brood care-factor prey from own reservoir fight brood care transport to nest from nest egg idle reservoir queen-factor mate level energy feed on nest genes feed on brood lay egg reservoir level seek new nest grow feed genes Optimization of simulated biological multi-agent systems 18 / 23 by means of evolutionary processes 9

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Initial insects’ world Optimization of simulated biological multi-agent systems 19 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Insects’ world after 150,000 ticks Optimization of simulated biological multi-agent systems 20 / 23 by means of evolutionary processes 10

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Changes of gene-pool queen-factor hunt-factor brood care - factor Optimization of simulated biological multi-agent systems 21 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg More changes of gene-pool initial egg energy energy portion ant energy portion brood Optimization of simulated biological multi-agent systems 22 / 23 by means of evolutionary processes 11

Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg Results & Discussion • Successful evaluation in three scenarios • ES/GA approach powerful and easy to use ? Use of explicit evaluation function for greater applicability ? Accelerate optimization (through parallelism) Optimization of simulated biological multi-agent systems 23 / 23 by means of evolutionary processes Lehrstuhl für Künstliche Intelligenz - Univ. Würzburg 12