R EPRESENTING T REES IN GEP 0123456789 +/Q*c-abde WSC9 - PowerPoint PPT Presentation

D ESIGNING N EURAL N ETWORKS U SING G ENE E XPRESSION P ROGRAMMING Cândida Ferreira Gepsoft, UK candidaf@gepsoft.com WSC9 www.gepsoft.com

P LAN 1. Representing trees in GEP 2. GEP genes 3. Multigenic chromosomes 4. Representing ANNs in GEP 5. Evolving the neural network architecture 6. Example: Intragenic two-point recombination 7. Fine-tuning the weights/thresholds 8. Example: Domain-specific transposition 9. XOR: Parameters and performance 10. XOR: Two perfect solutions 11. 6-Multiplexer: Parameters and performance 12. 6-Multiplexer: A perfect unigenic solution 13. 6-Multiplexer: A perfect multigenic solution WSC9 www.gepsoft.com

R EPRESENTING T REES IN GEP 0123456789 +/Q*c-abde WSC9 www.gepsoft.com

GEP G ENES 0123456789012345678901234567890 /aQ/b*ab/Qa*b*-ababaababbabbbba head tail t = h ( n -1) + 1 h - head length t - tail length n - max arity WSC9 www.gepsoft.com

M ULTIGENIC C HROMOSOMES 012345678901234012345678901234012345678901234 Q/*b+Qababaabaa-abQ/*+bababbab**-*bb/babaaaab Sub-ET 1 Sub-ET 2 Sub-ET 3 Q a / b b * b b / Q a a b b a a b WSC9 www.gepsoft.com

R EPRESENTING ANN S IN GEP o1 5 6 h1 h2 2 3 1 4 i1 i2 D 5 6 head Dw D D 0123456789012 1 2 3 4 DDDabab654321 a b a b tail WSC9 www.gepsoft.com

E VOLVING THE N EURAL N ETWORK A RCHITECTURE Genetic Operators: » Mutation » Transposition » IS Transposition » RIS Transposition » Gene Transposition » Recombination » One-point Recombination » Intragenic Two-point Recombination WSC9 www.gepsoft.com

E XAMPLE : I NTRAGENIC T WO - POINT R ECOMBINATION a) 0123456789012345601234567890123456 W = {-0.78, -0.521, -1.224, 1.891, 0.554, 1.237, -0.444, 0.472, 1.012, 0.679} 0,1 TTababaab14393255QDbabbabb96369304 -[0] W = {-1.553, 1.425, -1.606, -0.487, 1.255, -0.253, -1.91, 1.427, -0.103, -1.625} 0,2 Qaabbbabb97872192QDbabbaaa81327963-[1] W = {-0.148, 1.83, -0.503, -1.786, 0.313, -0.302, 0.768, -0.947, 1.487, 0.075} 1,1 W = {-0.256, -0.026, 1.874, 1.488, -0.8, -0.804, 0.039, -0.957, 0.462, 1.677} 1,2 0123456789012345601234567890123456 T aabbbabb978 93255QDbabbabb96369304 -[0] Q Tababaab143 72192QDbabbaaa81327963-[1] b) c) L L Sub-NN 0,1 Sub-NN 0,2 Sub-NN 0,1 Sub-NN 0,2 T Q T Q 1.891 -1.224 1.237 1 0.679 9 1.237 -0.487 -0.487 -0.487 -0.487 8 -1.625 -1.625 -1.91 . -1.91 1 a T b a a b a a D b b D b b 1.237 -1 .224 -1.553 1.237 -1.553 1.255 1.255 a a a b a b b L L Sub-NN 1,1 Sub-NN 1,2 Sub-NN 1,1 Sub-NN 1,2 Q Q Q Q -0.503 1.488 1.488 -0.503 0.313 7 -0.503 1.677 0.075 1 -0.957 1 1 7 -0.957 7 - 6 . . . 1 . 4 8 8 1 8 . 9 3 7 7 7 . 4 8 4 0 6 - a a b b a a D b b a a D b b T b 0 .039 0.039 1.488 1.488 -0.503 1.83 0.075 a a b b a a b WSC9 www.gepsoft.com

F INE - TUNING THE W EIGHTS /T HRESHOLDS Genetic Operators: » Domain-specific Mutation » Domain-specific Transposition » Direct Mutation of Weights/Thresholds WSC9 www.gepsoft.com

E XAMPLE : D OMAIN - SPECIFIC T RANSPOSITION 01234567890123456 7890123456789012 3456 01234567890123456 7890123456789012 3456 DTQaababaabbaabba 0571745736284668 2867 DTQaababaabbaabba 0571466827457362 8466 W = {-1.64, -1.834, -0.295, 1.205, -0.807, 0.856, 1.702, W = {-1.64, -1.834, -0.295, 1.205, -0.807, 0.856, 1.702, m d -1.026, -0.417, -1.061} -1.026, -0.417, -1.061} T = {-1.14, 1.177, -1.179, -0.74, 0.393, 1.135, -0.625, T = {-1.14, 1.177, -1.179, -0.74, 0.393, 1.135, -0.625, m d 1.643, -0.029, -1.639} 1.643, -0.029, -1.639} -0.029 0.393 D D 5 5 -1.026 -0.417 9 0 2 2 0 . . 1 - T Q T Q -0.625 -0.625 -0.625 1.643 1 1 -0.417 1.702 -0.295 -1.026 -0.807 - -1.026 - . . -0.417 1.702 7 0 1.702 2 0 0.856 0 0 . . 2 8 2 5 9 0 5 7 a a a a a a a a a a b b b b WSC9 www.gepsoft.com

XOR: P ARAMETERS AND P ERFORMANCE Redundant System Compact System Number of runs 100 100 Number of generations 50 50 Population size 30 30 Number of fitness cases 4 4 Function set D T Q D T Q Terminal set a b a b Weights array length 10 10 Weights range [-2, 2] [-2, 2] Head length 4 2 Number of genes 1 1 Chromosome length 33 17 Mutation rate 0.061 0.118 One-point recombination rate 0.7 0.7 IS transposition rate 0.1 -- IS elements length 1 -- RIS transposition rate 0.1 -- RIS elements length 1 -- Dw specific transposition rate 0.1 0.1 Dw specific IS elements length 2,3,5 2,3,5 Success rate 77% 30% WSC9 www.gepsoft.com

XOR: T WO P ERFECT S OLUTIONS 1. 2. 01234567890123456 012345678901234567890123456789012 TQaTaaababbbabaaa6085977238275036 TDbabaabb88399837 W = {1.175, 0.315, -0.738, 1.694, -1.215, 1.956, W = {0.713, -0.774, -0.221, 0.773, -0.789, 1.792, -1.77, 0.443, -1.924, 1.161} -0.342, 1.088, -1.694, 1.288} T T -0.738 1.088 1.694 -1.924 1.161 1.161 a Q a T D b 0.443 0.773 -1.694 1.956 -0.342 1.088 -0.738 1.175 1.694 a b a a a a b b b WSC9 www.gepsoft.com

6-M ULTIPLEXER : P ARAMETERS AND P ERFORMANCE Unigenic System Multigenic System Number of runs 100 100 Number of generations 2000 2000 Population size 50 50 Number of fitness cases 64 64 Function set 3U 3D 3T 3U 3D 3T Terminal set a b c d d e f a b c d d e f Linking function -- O Weights array length 10 10 Weights range [-2, 2] [-2, 2] Head length 17 5 Number of genes 1 4 Chromosome length 103 124 Mutation rate 0.044 0.044 Intragenic two-point recombination rate 0.6 0.6 Gene recombination rate -- 0.1 Gene transposition rate -- 0.1 IS transposition rate 0.1 0.1 IS elements length 1,2,3 1,2,3 RIS transposition rate 0.1 0.1 RIS elements length 1,2,3 1,2,3 Weights mutation rate 0.002 0.002 Dw specific transposition rate 0.1 0.1 Dw specific IS elements length 2,3,5 2,3,5 Success rate 4% 6% WSC9 www.gepsoft.com

6-M ULTIPLEXER : A P ERFECT U NIGENIC S OLUTION TbDTTTfTTaUDcUUTTafeefebabbdabffddfcfeeeabcabfabdcfe... ...709761631479459597193997465381760511137453583952159 W = {0.241, 1.432, 1.705, -1.95, 1.19, 1.344, 0.925, -0.163, -1.531, 1.423} T -1.95 .423 3 2 4 . 1 1 D T b 1.344 5 0.925 9 - 0 . . 1 6 1 3 - 1.19 T T f T T -1.531 -0.163 1.19 1.432 1.432 -1.95 4 163 4 0.241 0 1.432 3 . 9 .432 . 1 2 5 -0. 1 a c a e e U D U U T T f 1 -1.531 1.344 1.344 1.344 1.705 1.423 . 1.423 4 4 4 -1.95 -1.95 3 3 2 . 1 e a a b b b d b f f f WSC9 www.gepsoft.com

6-M ULTIPLEXER : A P ERFECT M ULTIGENIC S OLUTION TecTDdfafabdddfa487674791701403 W = {1.126, 0.042, 1.588, -0.03, -1.91, 1.83, -0.412, 0.607, -0.294, -0.659} 1 TDcbTbadddfceacc501702156029560 W = {-1.961, 1.161, 1.588, -0.03, -1.91, 1.762, -0.412, -0.121, -0.294, -0.659} 2 TfTTUbadbcdffdce593993321226318 W = {1.558, -0.69, 0.921, 0.134, 0.468, -1.534, 0.966, 1.399, 0.023, 0.915} 3 TDTbaceaaeeacacd072636270049968 W = {1.558, 0.767, 0.076, 0.071, 0.468, -1.534, 1.387, -1.857, -1.88, 0.331} 4 Sub-NN 1 Sub-NN 2 T T 0.607 1 - 1.762 0 6 -0.412 0.042 . 9 6 5 . 1 9 - e c c T D b -0.659 1.126 1.588 -1.91 0.042 D d f T b 1.126 -0.03 -1.961 1.762 -0.412 Note: a f a d d Sub-NNs linked by OR Sub-NN 3 Sub-NN 4 T T -1.857 0.134 1.558 0.921 0.134 1.558 f T T D T b 0.921 0.331 0.966 - 1.387 9 0.468 0 8 0.134 0.921 6 . 0.331 8 6 . 0 . 9 1 - - a c a c e a a U b d b 0.023 d WSC9 www.gepsoft.com