MuACO sm – A New Mutation-Based Ant Colony Optimization Algorithm for Learning Finite-State Machines Daniil Chivilikhin and Vladimir Ulyantsev National Research University of IT, Mechanics and Optics St. Petersburg, Russia Evolutionary and Combinatorial Optimization Track @ GECCO 2013 July 8, 2013
Motivation: Reliable software • Systems with high cost of failure – Energy industry – Aircraft industry – Space industry – … • We want to have reliable software – Testing is not enough – Verification is needed ACO for Learning FSMs 2
Introduction (1) • Automated software engineering • Model-driven development • Automata-based programming Software Model Code specification ACO for Learning FSMs 3
Introduction (2) Finite-state machine Software Model Code specification ACO for Learning FSMs 4
Finite-State Machine • S – set of states T / z 3 • s 0 ∈ S – initial state • Σ – set of input events 1 • Δ – set of output actions A / z 3 T / z 1 • δ : S ×Σ → S – transition function • λ : S ×Σ → Δ – actions function Example: 2 • two states • events = { A , T } • actions = { z 1 , z 2 , z 3 , z 4 } A / z 2 ACO for Learning FSMs 5
Automata-based programming Output Events Design programs with actions z 1 complex behavior as e 1 z 2 automated-controlled z 2 z 3 e 2 objects z 4 Automated-controlled object Events Finite-state Controlled machine object Actions ACO for Learning FSMs 6
Automata-based programming: advantages • Model before programming code, not vice versa Finite-state machine Model Code • Possibility of program verification using Model Checking • You can check temporal properties (LTL) ACO for Learning FSMs 7
Issues • Hard to build an FSM with desired structure and behavior • Several problems of learning FSMs were proven to be NP-hard • One of the solutions – metaheuristics ACO for Learning FSMs 8
Learning finite-state machines with metaheuristics • N states – number of states Software specification • Σ – input events • Δ – output actions • X = ( N states , Σ, Δ ) – search space Test Modeling examples environment Fitness function f : X → R ACO for Learning FSMs 9
Approaches to learning FSMs • Greedy heuristics – problem-specific • Reduction to SAT and CSP problems – fast – problem-specific • Evolutionary algorithms (general) – slow ACO for Learning FSMs 10
Proposed approach • Based on Ant Colony Optimization (ACO) • Non-standard problem reduction • Modified ACO algorithm ACO for Learning FSMs 11
Solution representation Transition table Output table δ λ Event Event State A T State A T 1 1 2 1 z 1 z 2 2 2 1 2 z 2 z 3 ACO for Learning FSMs 12
“Canonical” way to apply ACO • Reduce problem to finding a minimum cost path in some complete graph • Vertices – FSM transitions: – < i ∈ S , j ∈ S , e ∈ Σ , a ∈ Δ > • Each ant adds transitions to its FSM 1 → 1 [ T / z 1 ] 1 → 2 [ A / z 2 ] ACO for Learning FSMs 13
“Canonical” ACO: example • 2 states • 2 events • 1 action ACO for Learning FSMs 14
“Canonical” ACO: issues • Number of vertices in the construction graph grows as ( N states ) 2 × | Σ | × | Δ | • No meaningful way to define heuristic information • Later we show that “canonical” ACO is ineffective for FSM learning ACO for Learning FSMs 15
Proposed algorithm: MuACO sm • Mutation-Based ACO for learning FSMs • Uses a non-standard problem reduction • Modified ACO ACO for Learning FSMs 16
Problem reduction: MuACO sm vs. “canonical” • “Canonical” ACO – Nodes are solution components – Full solutions are built by ants • Proposed MuACO sm algorithm – Nodes are full solutions (FSMs) – Ants travel between full solutions ACO for Learning FSMs 17
FSM Mutations T / z 3 1 Change Change transition T / z 1 A / z 3 transition action end state 2 A / z 2 T / z 3 T / z 3 1 1 A / z 3 A / z 1 T / z 1 T / z 1 2 2 A / z 2 A / z 2 ACO for Learning FSMs 18
MuACOsm problem reduction • Construction graph – nodes are FSMs – edges are mutations of FSMs • Example ACO for Learning FSMs 19
Real search space graph ACO for Learning FSMs 20
Part of real search space (1) ACO for Learning FSMs 21
Part of real search space (2) ACO for Learning FSMs 22
Heuristic information η uv = max( η min , f ( v ) – f ( u )) u v Finite-state machines ACO for Learning FSMs 23
ACO algorithm A 0 = random FSM Improve A 0 with (1+1)-ES Graph = { A 0 } while not stop() do ConstructAntSolutions UpdatePheromoneValues DaemonActions ACO for Learning FSMs 24
Constructing ant solutions A 1 • Use a colony of ants • An ant is placed on a graph node A 2 • Each ant has a limited A number of steps A 3 • On each step the ant moves to the next node A 4 ACO for Learning FSMs 25
Ant step: selecting the next node P = 1 – P new A 1 A 1 Mutation f ( A 1 )=8 P = P new A 2 A 2 f ( A 2 )=12 A A f ( A )=10 A 3 A 3 f ( A 3 )=0 Probabilistic А 4 A 4 selection f ( A 4 )=9 Go to best mutated uv uv p FSM Av uw uw ACO for Learning FSMs 26 w { A 1 , A 2 , A 3 , A 4 }
Pheromone update • Ant path quality = max fitness value on a path τ • Update – largest pheromone value best uv deployed on edge (u, v) • Update pheromone values: ( τ ρ τ τ best = 1 ) + uv uv uv ρ • – pheromone evaporation rate 0,1 ACO for Learning FSMs 27
Differences from previous work • Added heuristic information • Changed start node selection for ants • Coupling with (1+1)-ES • More experiments (later) • More comparisons with other authors • Harder problem ACO for Learning FSMs 28
“Simple” problem: Artificial Ant • Toroidal field N × N • M pieces of food • s max time steps • Fixed position of food and the ant • Goal – build an FSM, such that the ant will eat all food in K steps Field example: John Muir Trail ACO for Learning FSMs 29
Artificial Ant: Fitness function s s 1 max last f n food s max • n food – number of eaten food pieces • s max – max number of allotted steps • s last – number of used steps eaten food f used time steps ACO for Learning FSMs 30
“Simple” problem: Artificial Ant • Two fields: – Santa Fe Trail – John Muir Trail • Comparison: – “Canonical” ACO – Christensen et al. (2007) – Tsarev et al. (2007) Santa Fe Trail – Chellapilla et al. (1999) ACO for Learning FSMs 31
“Canonical” ACO “Canonical” MuACO sm ACO State Success Success count rate, % rate, % 5 18 87 10 10 91 ACO for Learning FSMs 32
Santa Fe Trail (Christensen et al., 600 steps) MuACOsm Christensen et al 21000 Fitness evaluation count 19000 17000 15000 13000 11000 9000 7000 5000 5 7 9 11 13 15 Number of FSM states ACO for Learning FSMs 33
John Muir Trail (Tsarev et al., 2007): 200 steps MuACOsm Genetic algorithm × 10 6 Fitneess evaluation count 50 45 40 35 30 25 20 15 10 5 0 8 9 10 11 12 13 14 15 16 Number of FSM states • MuACO sm is 30 times faster for FSMs with 7 states ACO for Learning FSMs 34
“Harder” problem: learning Extended Finite-State Machines (1) ACO for Learning FSMs 35
“Harder” problem: learning Extended Finite-State Machines (2) Input data: • Number of states C and sets Σ and Δ • Set of test examples T • T i =<input sequence I j, output sequence O j > NP-hard problem: build an EFSM with C states compliant with tests T ACO for Learning FSMs 36
Learning EFSMs: Fitness function • Pass inputs to EFSM, record outputs • Compare generated outputs with references • Fitness = string similarity measure (edit distance) ACO for Learning FSMs 37
Experimental setup 1. Generate random EFSM with C states 2. Generate set of tests of total length C × 150 3. Learn EFSM 4. Experiment for each C repeated 100 times 5. Run until perfect fitness 6. Record mean number of fitness evaluations ACO for Learning FSMs 38
Learning random EFSMs ACO for Learning FSMs 39
Conclusion • Developed new ACO-based algorithm for learning FSMs and EFSMs • MuACO sm greatly outperforms GA on considered problems • Generated programs can be verified with Model Checking ACO for Learning FSMs 40
Future work • Better FSM representation to deal with isomorphism • Use novelty search • Employ verification in learning process Fitness Function MuACOsm Model Checking ACO for Learning FSMs 41
Acknowledgements • We thank the ACM for the student travel grants ACO for Learning FSMs 42
Thank you for your attention! MuACOsm – A New Mutation-Based Ant Colony Optimization Algorithm for Learning Finite-State Machines Daniil Chivililikhin Vladimir Ulyantsev This presentation online: http://rain.ifmo.ru/~chivdan/papers/gecco- 2013-presentation.pdf
Recommend
More recommend
