Dynamic Fitness Functions for Genetic Improvement in Compilers and - PowerPoint PPT Presentation

Oliver Krauss MSc Dynamic Fitness Functions for Genetic Improvement in Compilers and Interpreters Kyoto 16.7.2018 Advisors: Prof. Dr. Dr. h.c. Hanspeter Mössenböck, Prof. (FH) Priv.-Doz. Dipl.-Ing. Dr. Michael Affenzeller

Abstract Genetic Improvement on Interpreter / Compiler Abstract Syntax Tree (AST) – Improve non-functional software features (performance) – Deal with large fitness landscapes – Dynamic fitness functions ... – split test suites by the complexity of the cases – Sequential or Parallel – ... don't perform well – ... show promise Page 1 | 26

Outline Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook Page 2 | 26

Outline Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook Page 3 | 26

Search Space Size ∗ – n node types (MiniC -> 158) ∗ ∗ – depth 5 , width 5 depth ∗ ∗ ∗ ∗ width Page 4 | 26

Search Space Size ∗ – n node types (MiniC -> 158) ∗ ∗ – depth 5 , width 5 depth – All Options -> n 3901 ∗ ∗ – MiniC -> 466 . 638 ∗ ∗ width Page 4 | 26

Search Space Size ∗ – n node types (MiniC -> 158) ∗ ∗ – depth 5 , width 5 depth – All Options -> n 3901 ∗ ∗ – MiniC -> 466 . 638 ∗ ∗ – depth 10 , width 10 width Page 4 | 26

Search Space Size ∗ – n node types (MiniC -> 158) ∗ ∗ – depth 5 , width 5 depth – All Options -> n 3901 ∗ ∗ – MiniC -> 466 . 638 ∗ ∗ – depth 10 , width 10 – All Options -> n 11 . 111 . 111 . 101 width – MiniC -> 80 . 814 . 253 . 363 Page 4 | 26

Outline Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook Page 5 | 26

JIT Compilation with GI I Code void main () { int i, n, GI Truffle now , prev , Graal next; prev = 0; now = 1; i = 0; n = 30; while (i < n){ if (i Optimize Compile Coco/R == 0 ) print (0); else if (i == 1) print (1); else { next = now + prev; prev = now; now = next; print(next); } i = i + 1; } } Figure: Code Interpretation and Compilation with GI (Truffle and Graal images from [1] ) Page 6 | 26

JIT Compilation with GI II MiniC – Subset of C11 [2] – Coco/R -> Creates Scanner & Parser in Java – 342 node classes (103 operators, 55 operands) Page 7 | 26

JIT Compilation with GI III Truffle [3, 1] – Interpreter based on AST – Generalized and Spezialized AST - Nodes – Self optimizing by AST-Rewriting – Can run any Guest Language on the JVM – Python, Ruby, JavaScript, ... Page 8 | 26

JIT Compilation with GI IV Graal [4, 5, 6] – Just-in-time (JIT) compiler – Agressive optimizations – Part of OpenJDK Page 9 | 26

JIT Compilation with GI V Select subtree AST from Coco/R Run in Truffle Create Stub for Optimization Collect Testdata IN OUT Get Testdata Profile 0 0 clone AST 1 1 2 1 3 2 4 3 5 5 clone & build Evaluate Population GI Population Hand best solution with Truffle evaluate to Truffle cross & mutate after x Generations Figure: GP Optimization in depth Page 10 | 26

Architectural Overview Figure: The architecture of the optimization framework in combination with Truffle and Graal Page 11 | 26

Outline Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook Page 12 | 26

Node Visitation Goal: visit all nodes fib read {} – n = 0 [v=14,s=0] n if ret || ret + == == n fib fib n 1 n 0 − − n 1 n 2 Previously Visited Visited Spezialisation Page 13 | 26

Node Visitation Goal: visit all nodes fib {} read – n = 0 [v=14,s=0] n if ret – n = 1 [v=11,s=0] || ret + == == n fib fib n n 1 0 − − n n 1 2 Visited Previously Visited Spezialisation Page 13 | 26

Node Visitation Goal: visit all nodes fib read {} – n = 0 [v=14,s=0] n if ret – n = 1 [v=11,s=0] || ret + – n = 2 [v=23,s=6] == == n fib fib n 1 n 0 − − n 1 n 2 Previously Visited Visited Spezialisation Page 13 | 26

Test Metrics I Correctness - Percentage of successful tests � tests n = 1 succeeded ( AST , test ) correctness ( AST , tests ) = � tests (1) Complexity - Percentage of nodes visited by a test � nodes visited ( AST , test ) n = 1 complexity ( AST , test ) = � nodes ( AST ) (2) Page 14 | 26

Test Metrics II Overlap - Percentage of nodes visited by two tests � nodes visited ( A ) ∩ visited ( B ) n = 1 overlap ( A , B ) = (3) max ( � nodes visited ( A ) , � nodes visited ( B ) ) n = 1 n = 1 Confidence - Percentage of nodes visited by test suite � nodes visited ( testsuite ) n = 1 confidence ( testsuite , AST ) = � nodes ( AST ) (4) Page 15 | 26

Outline Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook Page 16 | 26

Sequential Fitness Function Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs Page 17 | 26

Sequential Fitness Function Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs Page 17 | 26

Sequential Fitness Function Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs Page 17 | 26

Sequential Fitness Function Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs Page 17 | 26

Sequential Fitness Function Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs Page 17 | 26

Sequential Fitness Function Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs Page 17 | 26

Sequential Fitness Function Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs Page 17 | 26

Sequential Fitness Function Figure: Sequential algorithm, iteratively increasing the amount of tests used by the genetic algorithm (GA) to verify ASTs Page 17 | 26

Parallel Fitness Function Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached Page 18 | 26

Parallel Fitness Function Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached Page 18 | 26

Parallel Fitness Function Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached Page 18 | 26

Parallel Fitness Function Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached Page 18 | 26

Parallel Fitness Function Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached Page 18 | 26

Parallel Fitness Function Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached Page 18 | 26

Parallel Fitness Function Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached Page 18 | 26

Parallel Fitness Function Figure: Parallel algorithm, splitting a test suite into parts, and iteratively recombining them into bigger suites until the original suite is reached Page 18 | 26

Outline Motivation Background Methodology - Analyze Testdata Methodology - Dynamic Evaluation Results Discussion & Outlook Page 19 | 26

Test SetUp GA vs. Sequential FFN vs. Parallel FFN – Same configuration for all – No grafting – Comparison of solved test cases Table: Use Case statistics Use Case No. Nodes Tests Nodes visited x*2 8 5 5, 5, 5, 5, 5 Sqrt 60 5 14, 42, 42, 42, 53 Fibonacci 32 5 11, 15, 30, 30, 30 Constructed 31 5 11, 15, 20, 20, 21 Page 20 | 26

Results UseCases Fitness Functions x*2 always solved, except Parallel FFN Page 21 | 26

Results UseCases Fitness Functions x*2 always solved, except Parallel FFN Sqrt return 2 Page 21 | 26

Results UseCases Fitness Functions x*2 always solved, except Parallel FFN Sqrt return 2 Fibonacci only first 2 (Sequential FFN) Page 21 | 26

Results UseCases Fitness Functions x*2 always solved, except Parallel FFN Sqrt return 2 Fibonacci only first 2 (Sequential FFN) Constructed 3 of 5 tests solved Page 21 | 26

Results UseCases Fitness Functions x*2 always solved, GA Baseline except Parallel FFN Sqrt return 2 Fibonacci only first 2 (Sequential FFN) Constructed 3 of 5 tests solved Page 21 | 26