Evolutionary algorithms paper Overview Laurits Tani - PowerPoint PPT Presentation

Evolutionary algorithms paper Overview Laurits Tani laurits.tani@gmail.com National Institute of Chemical Physics and Biophysics 15.05.2020 Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 1 / 25

ATLAS Higgs Challenge (using XGBoost) [from before] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 2 / 25

Using the default XGBoost parameters Parameter value num-boost-round 10 learning-rate 0.3 max-depth 6 gamma 0 min-child-weight 1 subsample 1 colsample-bytree 1 Threshold: 0.710 Public score Private score 3.16119 3.11380 Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 3 / 25

PSO found best hyperparameters Parameter best-value num-boost-round 153 learning-rate 0.3 max-depth 4 gamma 3.86 min-child-weight 323.6 subsample 0.830 colsample-bytree 1 Threshold: 0.841 Public score Private score 3.54407 3.45578 Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 4 / 25

GA found best hyperparameters Parameter best-value num-boost-round 500 learning-rate 0.12859 max-depth 4 gamma 5.0 min-child-weight 257.60083 subsample 1 colsample-bytree 1 Threshold: 0.841 Public score Private score 3.54464 3.44630 Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 5 / 25

Final comparison Method Threshold Public score Private score Default 0.710 3.16119 3.11380 PSO 0.841 3.54464 3.44630 GA 1 0.841 3.54407 3.45578 1 With the old set-up. Now bigger mutations (e.g from O(10) → O(100) less likely) Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 6 / 25

Conclusions Hyperparameters vary alot, result not too much Point was to automatize hyperparameter optimization to save time? Unfortunately not been able to find out the reason for much worse performance of NN and XGB, so this is not included. Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 7 / 25

Rosenbrock function Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 8 / 25

Brief reminder Used for testing the performance of optimization algorithms Known minimum at (a, a 2 ) [in our case this would be at (1, 1) since a = 1 and b = 100] To find minimum 10 6 evaluations done Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 9 / 25

Stability [10k particles, 100 iterations] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 10 / 25

Stability [10k particles, 100 iterations] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 11 / 25

Performance [10k particles, 100 iterations] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 12 / 25

Performance [10k particles, 100 iterations] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 13 / 25

GA distance performance [10000 particles, 100 iterations] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 14 / 25

GA fitness performance [10000 particles, 100 iterations] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 15 / 25

GA distance stability [1000 particles, 100 iterations] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 16 / 25

GA fitness stability [1000 particles, 100 iterations] Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 17 / 25

(modified) Gradient descent Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 18 / 25

Gradient descent Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 19 / 25

Backup slides Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 24 / 25

Rosenbrock Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 25 / 25

Evolutionary algorithms paper Overview Laurits Tani - PowerPoint PPT Presentation

Evolutionary algorithms paper Overview Laurits Tani laurits.tani@gmail.com National Institute of Chemical Physics and Biophysics 15.05.2020 Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 1 / 25 ATLAS Higgs Challenge (using

Evolutionary Algorithms CS 478 - Evolutionary Algorithms 1 Evolutionary Computation/Algorithms

Outline DM812 METAHEURISTICS Lecture 6 Evolutionary Algorithms 1. Evolutionary Algorithms

Evolutionary Algorithms To use evolutionary algorithms your must: Fitness Define your

I t Introduction to d ti t Evolutionary Algorithms Federico Nesti, f.nesti@santannapisa.it

Evolutionary Algorithms - Population management and popular algorithms Kai Olav Ellefsen

Lounge Evolutionary Algorithms Christopher Mark Gore http://www.cgore.com cgore@cgore.com

CSE CSE 460 460 Evolutionary Evolutionary Methods Methods In this section we will look at

Research Group HEAL Contact: Heuristics and Evolutionary Algorithms Laboratory Heuristic and

Electronic design with evolutionary algorithms Adrian Thompson et al. (University of Sussex, UK)

11.1 Evolutionary Strategies Evolutionary methods for continuous optimisation (Bienert,

Evolutionary Algorithms Keith L. Downing The Norwegian University of Science and Technology

Survey of Quantum-Inspired Evolutionary Algorithms Robert Nowotniak, MSc Supervisor: Jacek

Evolutionary Design By: Dianna Fox and Dan Morris Review 4 main types of Evolutionary

1 OLEMAS - OLEMAS - Agent Architecture Learning algorithm (I) Use of prototypical

1 20 July 2007 CERN Seminar 2 Introduction to evolutionary computation Evolutionary

10/20/19 Multi-objective Evolutionary Algorithms Genetic Algorithms Multi-objective

Model-Based Evolutionary Algorithms Part 1: Estimation of Distribution Algorithms Dirk Thierens

Genetic Algorithms Welcome to this introduction to Genetic Algorithms. The objective of this paper

Lab 4: 26 th March 2012 Exercise 1: Evolutionary algorithms 1. Found a problem where EAs would

Evolutionary Algorithms General Concepts Prof. Thomas Bck Natural Computing Group

Genetic Algorithms Evolutionary computation Prototypical GA An example: GABIL

Understanding Diversity and Recombination in Simple Evolutionary Algorithms Dirk Sudholt

APPLICATION OF TWO-LEVEL EVOLUTIONARY ALGORITHMS TO SELF-STRUCTURING ANTENNAS C. M. Coleman*, E.

1 University of Birmingham 3 March 2010 2 Introduction to evolutionary computation