Newfoundland & Labrador, Canada Kaizen Programming Vinícius Veloso de Melo vinicius.melo@unifesp.br Institute of Science and Technology (ICT) Federal University of São Paulo (UNIFESP)
Summary Summary ● Context ● Kaizen Programming ● Experiments ● Summary and Conclusions ● Future works 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 2 2
Context Context Not so good individuals can pass to the next generation via tournament selection in order to maintain diversity Each individual represents in the population a complete solution Check stopping Check stopping Initial criteria / insert criteria / insert population into current into current population population Calculate Calculate the fitness the fitness Evolutionary Evolutionary Solution The quality of the individual (fitness) is how good it solves the problem cycle cycle Selection Selection Evolution is driven by natural Generate Generate selection (improvement of the fittest) offspring offspring Usually random modifications 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 3 3
Context Context ● Suppose the following symbolic regression problem: – Global optimum: ● f(x) = sin(x) – Current best: ● f(x) = -(x²)/(123.91-x+tanh(10))-13.502*sin(x)+sqrt(abs((5.2134³)*x)) – GP (or similar) inserts expressions trying to reduce the error caused by the garbage expressions ● Bloat – Is it easy for GP (or similar) to get to the global optimum from this current best? ● What if we could detect which parts of the expression are good and which are bad? 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 6 6
07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 8 8
The Kaizen methodology The Kaizen methodology ● The Japanese word Kaizen means “Good Change,” and is adopted as a philosophy of work which means continuous improvement ● Kaizen Event is the term given to an event consisting of a team (of workers and managers) working together for a brief period of time to find effective solutions to identified business problems 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 9 9
07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 10 10
The Kaizen methodology The Kaizen methodology Plan-Do-Check-Act (PDCA) Plan-Do-Check-Act (PDCA) Source: http://www.binaryspectrum.com/itservices/quality_assurance.html 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 11 11
Kaizen Programming Kaizen Programming ● Kaizen Programming (KP) is a novel evolutionary tool based on the concepts of the Kaizen methodology ● KP is a computational implementation of a Kaizen event with PDCA ● KP individuals are not complete solutions, but part of it (divide and conquer strategy) – Evolution becomes a collaborative approach instead of an egocentric one 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 13 13
Kaizen Programming Kaizen Programming PDCA cycle PDCA cycle A team of experts is formed to propose ideas to solve a problem, that are put together to become a complete solution Now one can determine, exactly, which parts of the solution should be removed or improved Such property results in a reduction in bloat, smaller population sizes, and lower number of function evaluations Construct a solution (build a model) using only the new ideas or new and old ideas at the same time The quality of an idea is a measurement of its contribution to the solution The quality of the solution is how good it solves the problem 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 15 15
Kaizen Programming Kaizen Programming Application in Symbolic Regression Application in Symbolic Regression ● The creation/modification of the ideas is performed by GP (crossover and mutation) – Using the set of terminals and non-terminals, the new ideas ( K i ) are non-linear relationships among the variables, i.e.: ● K 1 =x 2 ; K 2 =sin(x); K 3 =-x+3/x ● The evaluation is performed by Ordinary Least Squares (multiple linear regression model) – ŷ = β 1 K 1 + β 2 K 2 + β 3 K 3 – β i are used to scale the ideas and are discovered by OLS ● All models generated by KP are linear in the parameters 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 16 16
Kaizen Programming Kaizen Programming Application in Symbolic Regression Application in Symbolic Regression ● The quality of the model (containing all partial solutions) is a measure of the goodness-of-fit – Adjusted R 2 : proportion of variance explained ● The quality of each solution is its importance to the model, not how good it fits ! – P-value: hypothesis test as a significance level α – Ideas with non-significant values (or very small β ) are not useful to the model ● The analysis of the model is used to guide the search instead of using natural selection 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 17 17
Remember me? Remember me? f(x) = -(x²)/(123.91-x+tanh(10))-13.502*sin(x)+sqrt((5.2134³)*x) P-value > α P-value < α P-value > α f(x) = -(x²)/(123.91-x+tanh(10))-13.502*sin(x)+sqrt((5.2134³)*x) f(x) = -13.502*sin(x) 0.0740631*(-13.502)*sin(x) 1.0*sin(x) ONE SINGLE ITERATION ! ONE SINGLE ITERATION ! 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 18 18
Kaizen Programming Kaizen Programming Application in Symbolic Regression Application in Symbolic Regression Check this constant! 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 19 19
Experiments: Symbolic regression Experiments: Symbolic regression Main results: Nguyen functions Main results: Nguyen functions 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 20 20
Experiments: Symbolic regression Experiments: Symbolic regression Main results: Nguyen functions Main results: Nguyen functions ● Artificial Bee Colony Programming (ABCP) ● Genetic Programming – Standard Crossover (SC) – No Same Mate (NSM) selection – Semantics Aware Crossover (SAC) – Context Aware Crossover (CAC) – Soft Brood Selection (SBS) – Semantic Similarity-based Crossover (SSC) ● Results taken from D. Karaboga, C. Ozturk, N. Karaboga, and B. Gorkemli. Artificial bee colony programming for symbolic regression. Information Sciences, 209(0):1 –15, 2012. 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 21 21
Experiments: Symbolic regression Experiments: Symbolic regression Main results: Nguyen functions (Karaboga et al., 2012) (Karaboga et al., 2012) Main results: Nguyen functions Why so large for only 2 terminals? 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 22 22
Experiments: Symbolic regression Experiments: Symbolic regression Main results: Nguyen functions (Karaboga et al., 2012) (Karaboga et al., 2012) Main results: Nguyen functions ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 23 23
Experiments: Symbolic regression Experiments: Symbolic regression Main results: Nguyen functions using KP Main results: Nguyen functions using KP ● Kaizen Programming – Number of experts: 8 – Maximum number of node evaluations: 1 x 10 5 – Idea improver: 90% GP Uniform Mutation / 10% GP ERC Mutation – Idea sharing: one-point crossover – 100 independent runs ● This configuration will certainly give terrible results! 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 24 24
Experiments: Symbolic regression Experiments: Symbolic regression Main results: Nguyen functions using KP Main results: Nguyen functions using KP 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 25 25
Experiments: Symbolic regression Experiments: Symbolic regression Main results: Nguyen functions Main results: Nguyen functions 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 26 26
Summary and Conclusions Summary and Conclusions ● Kaizen Programming (KP) uses a collaborative problem solving approach in which partial solutions are put together to result in a complete solution ● The final solution is a multiple linear regression model – Easier to understand (when compared to a single huge bloated solution generated by GP) and to interpret if necessary ● for instance: in the final model the best curve is an exponential component, or a sine component, etc – The resulting ideas can be seen as features extracted from the dataset. The features have different distinct accuracies to complement each other ● PCA? ICA? FFT? 07/14/14 07/14/14 GECCO 2014, VANCOUVER, BC, CANADA GECCO 2014, VANCOUVER, BC, CANADA 27 27
Recommend
More recommend
