UKPEW 2005 Invited Talk The Future is Collaborative Performance Engineering! Jeremy Bradley Email: jb@doc.ic.ac.uk Department of Computing, Imperial College London Produced with prosper and L A T EX JTB [14/07/2005] – p.1/29
Drowning! We are drowning in modelling techniques and formalisms: SPA (Stochastic Process Algebras) SAN (Stochastic Automata Networks) SPN (Stochastic Petri Nets) LQN (Layered Queueing Networks) ... JTB [14/07/2005] – p.2/29
Drowning! We have underlying models: CTMC (Continuous Time Markov Chains) DTMC (Discrete Time Markov Chains) MDP (Markov Decision Processes) MRP (Markov Reward Processes) SMP (Semi Markov Processes) GSMP (Generalised Semi Markov Processes) ... JTB [14/07/2005] – p.3/29
Drowning! Not to mention analysis techniques: steady-state analysis transient analysis passage time analysis entropy maximisation simulation continuous approximation ... JTB [14/07/2005] – p.4/29
Drowning! ...performance query formalisms CSL/eCSL/aCSL pCTL stochastic probes NICE automata ... JTB [14/07/2005] – p.5/29
Drowning! ...and tools: ETMCC PRISM ipc/DNAmaca PEPA workbench DSPNexpress SMARTA Two Towers GreatSPN Möbius ... and many others... JTB [14/07/2005] – p.6/29
Some questions Do all the tools produce the same results for the same questions and models? Are some tools/formalisms/performance models better at capturing/modelling certain systems better than others? Which tools run faster on which type of model? Is this because they use BDDs/out-of-core solvers/MDDs/hypergraph partitioning? Are we reinventing the wheel?... in terms of tools and models? JTB [14/07/2005] – p.7/29
An answer...? The answer is another tool: Perform-DB! Not quite! More an integrated collaborative performance engineering environment Encourage a performance engineering lifecycle JTB [14/07/2005] – p.8/29
A small but significant example... Fully deterministic system: S ∼ det (1) T ∼ det (1) JTB [14/07/2005] – p.9/29
Transient analysis... π 00 ( t ) :probability of being in state 0 having started in state 0 1.2 pi_00(t) 1 0.8 Probability, p 0.6 0.4 0.2 0 0 2 4 6 8 10 12 Time, t JTB [14/07/2005] – p.10/29
With a small bit of randomness Semi deterministic system: S ∼ exp (1) T ∼ det (1) JTB [14/07/2005] – p.11/29
Transient analysis... π 00 ( t ) :probability of being in state 0 having started in state 0 0.8 Analytic solution, pi_00(t) 1 iteration 2 iterations 0.7 4 iterations 6 iterations 8 iterations 0.6 0.5 Probability 0.4 0.3 0.2 0.1 0 0 2 4 6 8 10 12 Time, t JTB [14/07/2005] – p.12/29
Stochastic Process Algebra PEPA syntax: P ✄ ✁ P ::= (a , λ ) . P P + P L P P / L A JTB [14/07/2005] – p.13/29
Stochastic Process Algebra PEPA syntax: P ✄ ✁ P ::= (a , λ ) . P P + P L P P / L A Action prefix: (a , λ ) . P JTB [14/07/2005] – p.13/29
Stochastic Process Algebra PEPA syntax: P ✄ ✁ P ::= (a , λ ) . P P + P L P P / L A Action prefix: (a , λ ) . P Competitive choice: P 1 + P 2 JTB [14/07/2005] – p.13/29
Stochastic Process Algebra PEPA syntax: P ✄ ✁ P ::= (a , λ ) . P P + P L P P / L A Action prefix: (a , λ ) . P Competitive choice: P 1 + P 2 Cooperation: P 1 ✄ ✁ L P 2 JTB [14/07/2005] – p.13/29
Stochastic Process Algebra PEPA syntax: P ✄ ✁ P ::= (a , λ ) . P P + P L P P / L A Action prefix: (a , λ ) . P Competitive choice: P 1 + P 2 Cooperation: P 1 ✄ ✁ L P 2 Action hiding: P / L JTB [14/07/2005] – p.13/29
Stochastic Process Algebra PEPA syntax: P ✄ ✁ P ::= (a , λ ) . P P + P L P P / L A Action prefix: (a , λ ) . P Competitive choice: P 1 + P 2 Cooperation: P 1 ✄ ✁ L P 2 Action hiding: P / L Constant label: A JTB [14/07/2005] – p.13/29
PEPA: Example def (AA ✄ { run } A1) ✄ ✁ { alert } (BB ✄ ✁ ✁ Sys = { run } B1) JTB [14/07/2005] – p.14/29
PEPA: Example def (AA ✄ { run } A1) ✄ ✁ { alert } (BB ✄ ✁ ✁ Sys = { run } B1) def AA = (run , ⊤ ) . (alert , r 5 ) . AA JTB [14/07/2005] – p.14/29
PEPA: Example def (AA ✄ { run } A1) ✄ ✁ { alert } (BB ✄ ✁ ✁ Sys = { run } B1) def AA = (run , ⊤ ) . (alert , r 5 ) . AA def A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 def A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 def A3 = (recover , r 1 ) . A1 JTB [14/07/2005] – p.14/29
PEPA: Example def (AA ✄ { run } A1) ✄ ✁ { alert } (BB ✄ ✁ ✁ Sys = { run } B1) def AA = (run , ⊤ ) . (alert , r 5 ) . AA def A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 def A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 def A3 = (recover , r 1 ) . A1 def BB = (run , ⊤ ) . (alert , r 5 ) . BB JTB [14/07/2005] – p.14/29
PEPA: Example def (AA ✄ { run } A1) ✄ ✁ { alert } (BB ✄ ✁ ✁ Sys = { run } B1) def AA = (run , ⊤ ) . (alert , r 5 ) . AA def A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 def A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 def A3 = (recover , r 1 ) . A1 def BB = (run , ⊤ ) . (alert , r 5 ) . BB def B1 = (start , r 1 ) . B2 + (pause , r 2 ) . B1 def B2 = (run , r 3 ) . B1 JTB [14/07/2005] – p.14/29
Types of Analysis Steady-state and transient analysis in PEPA: def 0.05 A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 Steady state: X_1 0.045 def 0.04 A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 0.035 def ⇒ 0.03 A3 = (recover , r 1 ) . A1 Probability 0.025 def AA = (run , ⊤ ) . (alert , r 5 ) . AA 0.02 0.015 AA ✄ ✁ def 0.01 Sys = { run } A1 0.005 0 0 5 10 15 20 25 30 Time, t JTB [14/07/2005] – p.15/29
Types of Analysis Steady-state and transient analysis in PEPA: def 0.05 A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 PEPA model: transient X_1 -> X_1 Steady state: X_1 0.045 def 0.04 A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 0.035 def ⇒ 0.03 A3 = (recover , r 1 ) . A1 Probability 0.025 def AA = (run , ⊤ ) . (alert , r 5 ) . AA 0.02 0.015 AA ✄ ✁ def 0.01 Sys = { run } A1 0.005 0 0 5 10 15 20 25 30 Time, t JTB [14/07/2005] – p.15/29
Types of Analysis Steady-state and transient analysis in PEPA: def 0.05 A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 PEPA model: transient X_1 -> X_1 Steady state: X_1 0.045 def 0.04 A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 0.035 def ⇒ 0.03 A3 = (recover , r 1 ) . A1 Probability 0.025 def AA = (run , ⊤ ) . (alert , r 5 ) . AA 0.02 0.015 AA ✄ ✁ def 0.01 Sys = { run } A1 0.005 0 0 5 10 15 20 25 30 Time, t JTB [14/07/2005] – p.15/29
Types of Analysis Steady-state and transient analysis in PEPA: def 0.05 A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 PEPA model: transient X_1 -> X_1 Steady state: X_1 0.045 def 0.04 A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 0.035 def ⇒ 0.03 A3 = (recover , r 1 ) . A1 Probability 0.025 def AA = (run , ⊤ ) . (alert , r 5 ) . AA 0.02 0.015 AA ✄ ✁ def 0.01 Sys = { run } A1 0.005 0 0 5 10 15 20 25 30 Time, t JTB [14/07/2005] – p.15/29
Types of Analysis Steady-state and transient analysis in PEPA: def 0.05 A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 PEPA model: transient X_1 -> X_1 Steady state: X_1 0.045 def 0.04 A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 0.035 def ⇒ 0.03 A3 = (recover , r 1 ) . A1 Probability 0.025 def AA = (run , ⊤ ) . (alert , r 5 ) . AA 0.02 0.015 AA ✄ ✁ def 0.01 Sys = { run } A1 0.005 0 0 5 10 15 20 25 30 Time, t JTB [14/07/2005] – p.15/29
Passage-time Quantiles Extract a passage-time density from a PEPA model: def A1 = (start , r 1 ) . A2 + (pause , r 2 ) . A3 def A2 = (run , r 3 ) . A1 + (fail , r 4 ) . A3 def ⇒ A3 = (recover , r 1 ) . A1 def AA = (run , ⊤ ) . (alert , r 5 ) . AA AA ✄ ✁ def Sys = { run } A1 JTB [14/07/2005] – p.16/29
The story used to be... Passage-time PDF hydra ✏✏✏✏ ✶ ipc ✲ ✲ PPPP PEPA HYDRA Passage-time CDF q Transient For state spaces of O (10 9 ) Very precise probabilistic results JTB [14/07/2005] – p.17/29
Now the story is... Stochastic simulation Dizzy ✏✏✏✏ ✶ ipc/pwb ✲ PPPP PEPA Rate equations q Numerical ODE solution For very large state spaces, e.g. 10 1000 + states Aggregate deterministic results JTB [14/07/2005] – p.18/29
Keeping track... Keeping track of: model version documentation of model version type/version of query parameter sets in model tool version numbers ...is non-trivial JTB [14/07/2005] – p.19/29
Perform-DB Allow modellers to link to/import/alter! other peoples’ models Keep track of model-result trail model → query → result Allow modellers to compare results from: different queries different parameters different models! JTB [14/07/2005] – p.20/29
Parameter Sweeping Model Rate parameter sweep Results PEPA net A ✄ ✁ L B [ B ] A def = ( up , ρ ) .A ′ + · · · JTB [14/07/2005] – p.21/29
Parameter Sweeping Model Rate parameter sweep Results PEPA net ✲ ✲ ρ = 0 . 8 A ✄ ✁ L B [ B ] A def = ( up , ρ ) .A ′ + · · · JTB [14/07/2005] – p.21/29
Recommend
More recommend
