execution trace management to support dynamic v v for
play

Execution Trace Management to Support Dynamic V&V for Executable - PowerPoint PPT Presentation

Introduction Contribution Evaluation Conclusion Execution Trace Management to Support Dynamic V&V for Executable DSMLs Ph.D defense December 3, 2015 Jury Erwan Bousse Prof. Gerti Kappel, TU Wien / rapporteur Prof. Franck Barbier, UPPA /


  1. Introduction Contribution Evaluation Conclusion Example of Petri net xDSML and model Operational Abstract Syntax semantics Net Execution Metamodel places transitions * * merges Place Place Place Place input Transition Transition Transition +tokens: int 1..* +name: string +name: string +name: string output conforms to +name: string +name: string +name: string conforms to +initialTokens: int +initialTokens: int +initialTokens: int 1..* imports run(Net) : while there is an enabled transition, fi res it. : while there is an enabled transition, fi res it. : removes a token from each input Place and adds a token to each output Place. fi re(Transition) : removes a token from each input Place, and adds a token to each output Place. Execution transformation rules (summarized) init=1 init=0 init=0 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 p2 p2 init=1 initialization − − − − − − − → Executed model Petri net model (t1 fired) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 5/32

  2. Introduction Contribution Evaluation Conclusion Example of Petri net xDSML and model Operational Abstract Syntax semantics Net Execution Metamodel places transitions * * merges Place Place Place Place input Transition Transition Transition +tokens: int 1..* +name: string +name: string +name: string output conforms to +name: string +name: string +name: string conforms to +initialTokens: int +initialTokens: int +initialTokens: int 1..* imports run(Net) : while there is an enabled transition, fi res it. : while there is an enabled transition, fi res it. : removes a token from each input Place and adds a token to each output Place. fi re(Transition) : removes a token from each input Place, and adds a token to each output Place. Execution transformation rules (summarized) init=1 init=0 init=0 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 p2 p2 init=1 initialization − − − − − − − → Executed model Petri net model (t1 fired) (t2 fired) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 5/32

  3. Introduction Contribution Evaluation Conclusion Representing executions as traces How to represent executions in order to analyze them? Example of a Petri net execution trace : run() fire(t1) fire(t2) p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 p2 p2 p2 Execution states , each storing the values of the mutable parts of the model (e.g. tokens ) Execution steps , each being the application of a transformation rule of the operational semantics, with: – small steps in between execution states (e.g. fire ) – big steps containing multiple steps (e.g. run ) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 6/32

  4. Introduction Contribution Evaluation Conclusion Representing executions as traces How to represent executions in order to analyze them? Example of a Petri net execution trace : run() fire(t1) fire(t2) p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 p2 p2 p2 Execution states , each storing the values of the mutable parts of the model (e.g. tokens ) Execution steps , each being the application of a transformation rule of the operational semantics, with: – small steps in between execution states (e.g. fire ) – big steps containing multiple steps (e.g. run ) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 6/32

  5. Introduction Contribution Evaluation Conclusion Difficulties in trace management Execution traces can be large – Requires large amounts of memory to construct a trace – Impacts processing time during analyses – example: looping small petri net (4 places and 3 transitions), 150 000 states = 130 MB Many kinds of trace manipulations – Generic (e.g. amount of steps) – Domain-specific (e.g. number of tokens that traversed a place) Many possible xDSMLs – Existing ones (e.g. Petri net, Activity diagrams) – Future ones (cf. language engineering) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 7/32

  6. Introduction Contribution Evaluation Conclusion Difficulties in trace management Execution traces can be large – Requires large amounts of memory to construct a trace – Impacts processing time during analyses – example: looping small petri net (4 places and 3 transitions), 150 000 states = 130 MB Many kinds of trace manipulations – Generic (e.g. amount of steps) – Domain-specific (e.g. number of tokens that traversed a place) Many possible xDSMLs – Existing ones (e.g. Petri net, Activity diagrams) – Future ones (cf. language engineering) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 7/32

  7. Introduction Contribution Evaluation Conclusion Difficulties in trace management Execution traces can be large – Requires large amounts of memory to construct a trace – Impacts processing time during analyses – example: looping small petri net (4 places and 3 transitions), 150 000 states = 130 MB Many kinds of trace manipulations – Generic (e.g. amount of steps) – Domain-specific (e.g. number of tokens that traversed a place) Many possible xDSMLs – Existing ones (e.g. Petri net, Activity diagrams) – Future ones (cf. language engineering) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 7/32

  8. Introduction Contribution Evaluation Conclusion Problem statement How to provide efficient execution trace management facilities for any xDSML? Three main inter-related challenges: 1 Scalability in memory to construct large traces 2 Scalability in time to process large traces 3 Usability of the execution trace data structure, to cope with the complexity of data of arbitrarily complex xDSMLs Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 8/32

  9. Introduction Contribution Evaluation Conclusion Contributions and applications Multidimensionnal domain-specific trace metamodel generation [ECMFA’15] applied to applied to Enhanced semantic Advanced and efficient differencing omniscient debugging [SLE’15] Scalable armies of model clones [MODELS’14] Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 9/32

  10. Introduction Contribution Evaluation Conclusion Contributions and applications Multidimensionnal domain-specific trace metamodel generation [ECMFA’15] applied to applied to Enhanced semantic Advanced and efficient differencing omniscient debugging [SLE’15] Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 9/32

  11. Introduction Contribution Evaluation Conclusion Contributions and applications Multidimensionnal domain-specific trace metamodel generation [ECMFA’15] applied to applied to Enhanced semantic Advanced and efficient differencing omniscient debugging [SLE’15] Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 9/32

  12. Introduction Contribution Evaluation Conclusion Outline 1 Introduction Multidimensionnal domain-specific trace metamodel generation [ECMFA’15] 2 Contribution 3 Evaluation 4 Conclusion Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  13. Introduction Contribution Evaluation Conclusion Generic trace management State of the art : generic trace metamodel [Langer et al.’14] {ordered=true} states * objects * State Object target source 1 1 Trace incoming 0..1 0..1 outgoing Event Transition transitions * event 1 qualifiedName : String States are stored in an unique sequence Each state is a snapshot (or clone) of the executed model Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 10/32

  14. Introduction Contribution Evaluation Conclusion Example of Petri net generic trace run() fire(t1) fire(t2) p1 p3 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 t1 t2 p2 p2 p2 states : Trace objects objects objects : Place : Place : Place s1 : State s2 : State s3 : State name = “p1" name = “p1" name = “p1" tokens = 1 tokens = 0 tokens = 0 : Place : Place : Place : Net : Net name = “p2" : Net name = “p2" name = “p2" tokens = 1 tokens = 1 tokens = 0 : Transition : Transition : Transition : Place : Place : Place name = “t1" name = “t1" name = “t1" name = “p3" name = “p3" name = “p3" tokens = 0 tokens = 1 tokens = 0 : Transition : Transition : Transition name = “t2" : Place name = “t2" : Place name = “t2" : Place name = “p4" name = “p4" name = “p4" tokens = 0 tokens = 0 tokens = 1 Scalability in time problem : need to visit all states Memory problem : redundancy with both immutable and mutable data Usability problem : type checks and casting Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 11/32

  15. Introduction Contribution Evaluation Conclusion Example of Petri net generic trace run() fire(t1) fire(t2) p1 p3 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 t1 t2 p2 p2 p2 states : Trace objects objects objects : Place : Place : Place s1 : State s2 : State s3 : State name = “p1" name = “p1" name = “p1" tokens = 1 tokens = 0 tokens = 0 : Place : Place : Place : Net : Net name = “p2" : Net name = “p2" name = “p2" tokens = 1 tokens = 1 tokens = 0 : Transition : Transition : Transition : Place : Place : Place name = “t1" name = “t1" name = “t1" name = “p3" name = “p3" name = “p3" tokens = 0 tokens = 1 tokens = 0 : Transition : Transition : Transition name = “t2" : Place name = “t2" : Place name = “t2" : Place name = “p4" name = “p4" name = “p4" tokens = 0 tokens = 0 tokens = 1 Scalability in time problem : need to visit all states Memory problem : redundancy with both immutable and mutable data Usability problem : type checks and casting Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 11/32

  16. Introduction Contribution Evaluation Conclusion Example of Petri net generic trace run() fire(t1) fire(t2) p1 p3 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 t1 t2 p2 p2 p2 states : Trace objects objects objects : Place : Place : Place s1 : State s2 : State s3 : State name = “p1" name = “p1" name = “p1" tokens = 1 tokens = 0 tokens = 0 : Place : Place : Place : Net : Net name = “p2" : Net name = “p2" name = “p2" tokens = 1 tokens = 1 tokens = 0 : Transition : Transition : Transition : Place : Place : Place name = “t1" name = “t1" name = “t1" name = “p3" name = “p3" name = “p3" tokens = 0 tokens = 1 tokens = 0 : Transition : Transition : Transition name = “t2" : Place name = “t2" : Place name = “t2" : Place name = “p4" name = “p4" name = “p4" tokens = 0 tokens = 0 tokens = 1 Scalability in time problem : need to visit all states Memory problem : redundancy with both immutable and mutable data Usability problem : type checks and casting Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 11/32

  17. Introduction Contribution Evaluation Conclusion Example of Petri net generic trace run() fire(t1) fire(t2) p1 p3 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 t1 t2 p2 p2 p2 states : Trace objects objects objects : Place : Place : Place s1 : State s2 : State s3 : State name = “p1" name = “p1" name = “p1" tokens = 1 tokens = 0 tokens = 0 : Place : Place : Place : Net : Net name = “p2" : Net name = “p2" name = “p2" tokens = 1 tokens = 1 tokens = 0 : Transition : Transition : Transition : Place : Place : Place name = “t1" name = “t1" name = “t1" name = “p3" name = “p3" name = “p3" tokens = 0 tokens = 1 tokens = 0 : Transition : Transition : Transition name = “t2" : Place name = “t2" : Place name = “t2" : Place name = “p4" name = “p4" name = “p4" tokens = 0 tokens = 0 tokens = 1 Scalability in time problem : need to visit all states Memory problem : redundancy with both immutable and mutable data Usability problem : type checks and casting Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 11/32

  18. Introduction Contribution Evaluation Conclusion Example of Petri net generic trace run() fire(t1) fire(t2) p1 p3 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 t1 t2 p2 p2 p2 states : Trace for (object : state.getObjects()) { if (object instanceof Place) { objects objects objects : Place Place p = (Place) object; : Place : Place s1 : State s2 : State s3 : State .. name = “p1" name = “p1" name = “p1" tokens = 1 tokens = 0 tokens = 0 } else if (object instanceof Transition) { Transition t = (Transition) object; : Place : Place : Place ... : Net : Net name = “p2" : Net name = “p2" name = “p2" } tokens = 1 tokens = 1 tokens = 0 } : Transition : Transition : Transition : Place : Place : Place name = “t1" name = “t1" name = “t1" name = “p3" name = “p3" name = “p3" tokens = 0 tokens = 1 tokens = 0 : Transition : Transition : Transition name = “t2" : Place name = “t2" : Place name = “t2" : Place name = “p4" name = “p4" name = “p4" tokens = 0 tokens = 0 tokens = 1 Scalability in time problem : need to visit all states Memory problem : redundancy with both immutable and mutable data Usability problem : type checks and casting Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 11/32

  19. Introduction Contribution Evaluation Conclusion Example of Petri net generic trace run() fire(t1) fire(t2) p1 p3 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 t1 t2 p2 p2 p2 states : Trace for (object : state.getObjects()) { if (object instanceof Place) { objects objects objects : Place Place p = (Place) object; : Place : Place s1 : State s2 : State s3 : State .. name = “p1" name = “p1" name = “p1" tokens = 1 tokens = 0 tokens = 0 } else if (object instanceof Transition) { Transition t = (Transition) object; : Place : Place : Place ... : Net : Net name = “p2" : Net name = “p2" name = “p2" } tokens = 1 tokens = 1 tokens = 0 } : Transition : Transition : Transition : Place : Place : Place name = “t1" name = “t1" name = “t1" name = “p3" name = “p3" name = “p3" tokens = 0 tokens = 1 tokens = 0 : Transition : Transition : Transition name = “t2" : Place name = “t2" : Place name = “t2" : Place name = “p4" name = “p4" name = “p4" tokens = 0 tokens = 0 tokens = 1 Scalability in time problem : need to visit all states Memory problem : redundancy with both immutable and mutable data Usability problem : type checks and casting Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 11/32

  20. Introduction Contribution Evaluation Conclusion Example of Petri net generic trace run() fire(t1) fire(t2) p1 p3 p4 p1 t1 p3 t2 p4 p1 t1 p3 t2 p4 t1 t2 p2 p2 p2 states : Trace for (object : state.getObjects()) { if (object instanceof Place) { objects objects objects : Place Place p = (Place) object; : Place : Place s1 : State s2 : State s3 : State .. name = “p1" name = “p1" name = “p1" tokens = 1 tokens = 0 tokens = 0 } else if (object instanceof Transition) { Transition t = (Transition) object; : Place : Place : Place ... : Net : Net name = “p2" : Net name = “p2" name = “p2" } tokens = 1 tokens = 1 tokens = 0 } : Transition : Transition : Transition : Place : Place : Place name = “t1" name = “t1" name = “t1" name = “p3" name = “p3" name = “p3" tokens = 0 tokens = 1 tokens = 0 : Transition : Transition : Transition name = “t2" : Place name = “t2" : Place name = “t2" : Place name = “p4" name = “p4" name = “p4" tokens = 0 tokens = 0 tokens = 1 Scalability in time problem : need to visit all states Memory problem : redundancy with both immutable and mutable data Usability problem : type checks and casting Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 11/32

  21. Introduction Contribution Evaluation Conclusion Approach: generating a domain-specific trace metamodel To provide usability and scalability in memory Generative approach to automatically derive a domain-specific trace metamodel for a given xDSML Domain-specific : domain concepts are directly accessible Precise : only concepts related to execution are kept Automated : save language engineers the design of a complex metamodel, which is time-consuming and error-prone To provide scalability in time Multidimensional navigation facilities, e.g. browsing a trace according to the values reached by a specific mutable element Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 12/32

  22. Introduction Contribution Evaluation Conclusion Overview imports Abstract Syntax merges Multidimensional Multidimensional domain-speci fi c Domain Speci fi c trace metamodel Operational semantics Trace Metamodel generator xDSML Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 13/32

  23. Introduction Contribution Evaluation Conclusion Trace metamodel generation – Steps concepts Trace {ordered=true} rootSteps Steps 0..* Base classes 1 {ordered=true} /subSteps 0..* /parentStep – small step = standalone <<abstract>> 0..1 Step transformation rule <<abstract>> <<abstract>> SmallStep BigStep – big step = rule relying on other rules {ordered=true} subSteps <<abstract>> 0..* FireStep RunStep Reification of rules into RunSubStep 2 runParentStep 0..1 step classes {ordered=true} {ordered=true} fireSequence runSequence Steps made accessible as 3 * * sequences or as a Petri net abstract syntax (partial) Transition caller Net containment tree 1 caller +name: String transitions 1 * Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 14/32

  24. Introduction Contribution Evaluation Conclusion Trace metamodel generation – Steps concepts Trace {ordered=true} rootSteps Steps 0..* Base classes 1 {ordered=true} /subSteps 0..* /parentStep – small step = standalone <<abstract>> 0..1 Step transformation rule <<abstract>> <<abstract>> SmallStep BigStep – big step = rule relying on other rules {ordered=true} subSteps <<abstract>> 0..* FireStep RunStep Reification of rules into RunSubStep 2 runParentStep 0..1 step classes {ordered=true} {ordered=true} fireSequence runSequence Steps made accessible as 3 * * sequences or as a Petri net abstract syntax (partial) Transition caller Net containment tree 1 caller +name: String transitions 1 * Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 14/32

  25. Introduction Contribution Evaluation Conclusion Trace metamodel generation – Steps concepts Trace {ordered=true} rootSteps Steps 0..* Base classes 1 {ordered=true} /subSteps 0..* /parentStep – small step = standalone <<abstract>> 0..1 Step transformation rule <<abstract>> <<abstract>> SmallStep BigStep – big step = rule relying on other rules {ordered=true} subSteps <<abstract>> 0..* FireStep RunStep Reification of rules into RunSubStep 2 runParentStep 0..1 step classes {ordered=true} {ordered=true} fireSequence runSequence Steps made accessible as 3 * * sequences or as a Petri net abstract syntax (partial) Transition caller Net containment tree 1 caller +name: String transitions 1 * Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 14/32

  26. Introduction Contribution Evaluation Conclusion Trace metamodel generation – Steps concepts Step classes are inferred by Trace static analysis of the model {ordered=true} rootSteps transformation. Steps 0..* {ordered=true} /subSteps 1 Analysis of the code 0..* /parentStep <<abstract>> 0..1 Step <<abstract>> <<abstract>> SmallStep BigStep {ordered=true} subSteps <<abstract>> 0..* FireStep RunStep RunSubStep runParentStep 0..1 {ordered=true} {ordered=true} fireSequence runSequence 2 Creation of a call graph * * Petri net abstract syntax (partial) 3 Pre-processing of the call graph (e.g. methods Transition caller Net 1 caller +name: String transitions overriding) 1 * 4 Discovery of big/small steps Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 14/32

  27. Introduction Contribution Evaluation Conclusion Trace metamodel generation – Steps concepts Step classes are inferred by Trace static analysis of the model {ordered=true} rootSteps transformation. Steps 0..* {ordered=true} /subSteps 1 Analysis of the code 0..* /parentStep <<abstract>> 0..1 Step <<abstract>> <<abstract>> SmallStep BigStep {ordered=true} subSteps <<abstract>> 0..* FireStep RunStep RunSubStep runParentStep 0..1 {ordered=true} {ordered=true} fireSequence runSequence 2 Creation of a call graph * * Petri net abstract syntax (partial) 3 Pre-processing of the call graph (e.g. methods Transition caller Net 1 caller +name: String transitions overriding) 1 * 4 Discovery of big/small steps Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 14/32

  28. Introduction Contribution Evaluation Conclusion Trace metamodel generation – Steps concepts Trace {ordered=true} rootSteps Steps 0..* Base classes 1 {ordered=true} /subSteps 0..* /parentStep – small step = standalone <<abstract>> 0..1 Step transformation rule <<abstract>> <<abstract>> SmallStep BigStep – big step = rule relying on other rules {ordered=true} subSteps <<abstract>> 0..* FireStep RunStep Reification of rules into RunSubStep 2 runParentStep 0..1 step classes {ordered=true} {ordered=true} fireSequence runSequence Steps made accessible as 3 * * sequences or as a Petri net abstract syntax (partial) Transition caller Net containment tree 1 caller +name: String transitions 1 * Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 14/32

  29. Introduction Contribution Evaluation Conclusion Trace metamodel generation – Steps concepts Trace {ordered=true} rootSteps Steps 0..* Base classes 1 {ordered=true} /subSteps 0..* /parentStep – small step = standalone <<abstract>> 0..1 Step transformation rule <<abstract>> <<abstract>> SmallStep BigStep – big step = rule relying on other rules {ordered=true} subSteps <<abstract>> 0..* FireStep RunStep Reification of rules into RunSubStep 2 runParentStep 0..1 step classes {ordered=true} {ordered=true} fireSequence runSequence Steps made accessible as 3 * * sequences or as a Petri net abstract syntax (partial) Transition caller Net containment tree 1 caller +name: String transitions 1 * Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 14/32

  30. Introduction Contribution Evaluation Conclusion Trace metamodel generation – States concepts Trace {ordered=true} States executionStates tracedPlaces 0..* * TracedPlace ExecutionState 1 Reification of mutable properies parent states into value classes 1 1..* {ordered=true} 2 Values stored as sequences , for tokensSequence 0..* TokensValue each model object tokensValues +tokens: int 0..* 3 Execution state = set of values Petri net abstract syntax 4 Can be browsed by states or originalObject 1 Execution Metamodel input value sequences Place Transition 1..* +name: String +name: String Place output +initialTokens: int 1..* +tokens: int places transitions * * Net merges Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 15/32

  31. Introduction Contribution Evaluation Conclusion Trace metamodel generation – States concepts Trace {ordered=true} States executionStates tracedPlaces 0..* * TracedPlace ExecutionState 1 Reification of mutable properies parent states into value classes 1 1..* {ordered=true} 2 Values stored as sequences , for tokensSequence 0..* TokensValue each model object tokensValues +tokens: int 0..* 3 Execution state = set of values Petri net abstract syntax 4 Can be browsed by states or originalObject 1 Execution Metamodel input value sequences Place Transition 1..* +name: String +name: String Place output +initialTokens: int 1..* +tokens: int places transitions * * Net merges Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 15/32

  32. Introduction Contribution Evaluation Conclusion Trace metamodel generation – States concepts Trace {ordered=true} States executionStates tracedPlaces 0..* * TracedPlace ExecutionState 1 Reification of mutable properies parent states into value classes 1 1..* {ordered=true} 2 Values stored as sequences , for tokensSequence 0..* TokensValue each model object tokensValues +tokens: int 0..* 3 Execution state = set of values Petri net abstract syntax 4 Can be browsed by states or originalObject 1 Execution Metamodel input value sequences Place Transition 1..* +name: String +name: String Place output +initialTokens: int 1..* +tokens: int places transitions * * Net merges Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 15/32

  33. Introduction Contribution Evaluation Conclusion Trace metamodel generation – States concepts Trace {ordered=true} States executionStates tracedPlaces 0..* * TracedPlace ExecutionState 1 Reification of mutable properies parent states into value classes 1 1..* {ordered=true} 2 Values stored as sequences , for tokensSequence 0..* TokensValue each model object tokensValues +tokens: int 0..* 3 Execution state = set of values Petri net abstract syntax 4 Can be browsed by states or originalObject 1 Execution Metamodel input value sequences Place Transition 1..* +name: String +name: String Place output +initialTokens: int 1..* +tokens: int places transitions * * Net merges Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 15/32

  34. Introduction Contribution Evaluation Conclusion Trace metamodel generation – States concepts Trace {ordered=true} States executionStates tracedPlaces 0..* * TracedPlace ExecutionState 1 Reification of mutable properies parent states into value classes 1 1..* {ordered=true} 2 Values stored as sequences , for tokensSequence 0..* TokensValue each model object tokensValues +tokens: int 0..* 3 Execution state = set of values Petri net abstract syntax 4 Can be browsed by states or originalObject 1 Execution Metamodel input value sequences Place Transition 1..* +name: String +name: String Place output +initialTokens: int 1..* +tokens: int places transitions * * Net merges Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 15/32

  35. Introduction Contribution Evaluation Conclusion Resulting Petri net trace metamodel Trace {ordered=true} {ordered=true} executionStates rootSteps 0..* States 0..* Steps ExecutionState <<abstract>> {ordered=true} {ordered=true} Step tracedPlaces endingState endedSteps /subSteps * +() 0..1 0..* 0..* +() +() TracedPlace {ordered=true} +() +() startingState startedSteps +() +() 1 0..* parent +() +() /parentStep 1 states 0..1 1..* {ordered=true} tokensSequence <<abstract>> <<abstract>> 0..* SmallStep BigStep TokensValue tokensValues +tokens: int {ordered=true} 0..* subSteps <<abstract>> 0..* FireStep RunStep RunSubStep runParentStep Petri net abstract syntax 0..1 originalObject 1 input {ordered=true} {ordered=true} Place caller Transition 1..* fireSequence runSequence 1 +name: String * * +name: String output +initialTokens: int 1..* transitions places * * Net caller 1 Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 16/32

  36. Introduction Contribution Evaluation Conclusion Example of Petri net trace (states only) (s0) (s1) (s2) p1 p3 p4 p1 p3 p4 p1 t1 p3 t2 p4 t1 t2 t1 t2 p2 p2 p2 exeTrace : Trace s1 : ExecutionState s2 : ExecutionState s0 : ExecutionState tracedPlaces executionState executionState executionState tokensValues tokensValues tokensValues p1 : TracedPlace : Place - name = "p1" tokens originalObject Trace - initialTokens = 1 : TokensValue : TokensValue tokens = 1 tokens = 0 value : Place p2 : TracedPlace object state - name = "p2" tokens originalObject model state - initialTokens = 1 Trace : TokensValue : TokensValue model element tokens = 1 tokens = 0 : Place p3 : TracedPlace - name = "p3" tokens originalObject : TokensValue - initialTokens = 0 Trace : TokensValue : TokensValue tokens = 0 tokens = 0 tokens = 1 : Place p4 : TracedPlace - name = "p4" tokens originalObject - initialTokens = 0 Trace : TokensValue : TokensValue tokens = 0 tokens = 1 Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 17/32

  37. Introduction Contribution Evaluation Conclusion Conclusion Summary Precise capture of the domain of the traces of an xDSML Multidimensional navigation facilities to ease queries Questions Usability and memory consumption improved with domain-specificness? Processing time reduced thanks to multidimensional navigation? ⇒ Application to two different V&V techniques + measurements Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 18/32

  38. Introduction Contribution Evaluation Conclusion Outline 1 Introduction Enhanced semantic 2 Contribution differencing 3 Evaluation Advanced and efficient omniscient debugging [SLE’15] 4 Conclusion Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  39. Introduction Contribution Evaluation Conclusion Evaluation Research questions (summarized) As compared to a generic clone-based trace metamodel: RQ1: Usability of generated trace metamodels RQ2: Scalability in time of trace manipulations RQ3: Scalability in memory of traces Considered V&V techniques 1 Semantic differencing – Manually defined trace manipulations – Evaluation of RQ1 and RQ2 2 Omniscient debugging – Generated trace manipulations – Evaluation of RQ2 and RQ3 Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 19/32

  40. Introduction Contribution Evaluation Conclusion Evaluation Research questions (summarized) As compared to a generic clone-based trace metamodel: RQ1: Usability of generated trace metamodels RQ2: Scalability in time of trace manipulations RQ3: Scalability in memory of traces Considered V&V techniques 1 Semantic differencing – Manually defined trace manipulations – Evaluation of RQ1 and RQ2 2 Omniscient debugging – Generated trace manipulations – Evaluation of RQ2 and RQ3 Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 19/32

  41. Introduction Contribution Evaluation Conclusion Semantic model differencing Context Model differencing : analyzing and understanding the changes made to a model during its development Mostly done syntactically (e.g. version control systems) The case of xDSMLs A change in an executable model ≡ a change in its behavior Hence, model differencing must be done semantically One approach: traces comparison before and after a change. Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 20/32

  42. Introduction Contribution Evaluation Conclusion Semantic model differencing Context Model differencing : analyzing and understanding the changes made to a model during its development Mostly done syntactically (e.g. version control systems) The case of xDSMLs A change in an executable model ≡ a change in its behavior Hence, model differencing must be done semantically One approach: traces comparison before and after a change. Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 20/32

  43. Introduction Contribution Evaluation Conclusion Existing base approach [Langer et al.’14] uses uses RQ1 Syntactic Abstract Clone-Based Generic Semantic Match Rules Syntax Execution Trace Metamodel Match Rules merges conform to conform to 1 3 2 Model 1 Execution Trace 1 Syntactic Operational Semantic Semantic Matching Semantics Di ff erencing di ff erences Model 2 Execution Trace 2 RQ2 Input Syntactic correspondances Requires the manual definition of semantic match rules specific to the xDSML Relies on a generic trace metamodel – usability (RQ1) : match rules are hard to write – scalability in time (RQ2) : match rules are long to execute Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 21/32

  44. Introduction Contribution Evaluation Conclusion Enhanced approach 0 uses uses RQ1 Syntactic Abstract Generator Multidimensional Domain-Speci fi c Semantic Match Rules Syntax (contribution) Execution Trace Metamodel Match Rules merges conform to conform to 1 3 2 Model 1 Execution Trace 1 Syntactic Operational Semantic Semantic Matching Semantics Di ff erencing di ff erences Model 2 Execution Trace 2 RQ2 Input Syntactic correspondances Enhancement Additional prior step : generation of a domain-specific trace metamodel with our contribution New semantic match rules relying on multiple dimensions What we measure: – usability (RQ1) : complexity of the match rules – scalability in time (RQ2) : execution time of the match rules Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 22/32

  45. Introduction Contribution Evaluation Conclusion Evaluation results Case study: fUML xDSML and models from [Maoz et al.’11] Elements Generic Domain-specific Lines of code 90 44 Statements 35 16 Complexity of Operations 8 3 Operation calls 35 24 rules (RQ1) Loops 5 4 Type checks 4 1 Conditionals 11 3 100000 matching time (ms) Lg 10^x 10000 1000 Rules execution 100 time (RQ2) 10 generic trace domain-specific trace 1 400 450 500 550 600 650 700 750 800 number of states Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 23/32

  46. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  47. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  48. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  49. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration (1) stepInto Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  50. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration (1) stepInto Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  51. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration (1) stepInto Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  52. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration (1) stepInto Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  53. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration (1) stepInto (3) backInto Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  54. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration (1) stepInto (3) backInto (4) backOut Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  55. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration (1) stepInto (3) backInto (4) backOut Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  56. Introduction Contribution Evaluation Conclusion Model omniscient debugging: terminology and scope Interactive debugging Weak omniscient debugging (without any trace) Forward exploration Omniscient debugging (with a partial or complete trace) Backward exploration Multidimensional Deterministic, omniscient debugging without engine restart Multidimensional exploration Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 24/32

  57. Introduction Contribution Evaluation Conclusion Omniscient debugging for any xDSML? Generic omniscient debugging possible, based on steps Two key problems: responsiveness and understandability when debugging models from arbitrarily complex xDSML Approach 1 Efficient generated domain-specific trace management facilities for responsiveness – generated trace metamodel (using our contribution) – generated trace manipulations (e.g. “jump back”) – RQ2 and RQ3 can be evaluated 2 Advanced generic multidimensional omniscient debugging services for understandability Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 25/32

  58. Introduction Contribution Evaluation Conclusion Omniscient debugging for any xDSML? Generic omniscient debugging possible, based on steps Two key problems: responsiveness and understandability when debugging models from arbitrarily complex xDSML Approach 1 Efficient generated domain-specific trace management facilities for responsiveness – generated trace metamodel (using our contribution) – generated trace manipulations (e.g. “jump back”) – RQ2 and RQ3 can be evaluated 2 Advanced generic multidimensional omniscient debugging services for understandability Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 25/32

  59. Introduction Contribution Evaluation Conclusion Advanced omniscient debugging New advanced debugging services to navigate according to the different dimensions of the model run(net) A p4 Model p2 t2 States p1 t1 p3 p5 t3 P1 Values P2 Values P3 Values P4 Values P5 Values Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 26/32

  60. Introduction Contribution Evaluation Conclusion Advanced omniscient debugging New advanced debugging services to navigate according to the different dimensions of the model run(net) fire(t1) A B p4 p4 Model p2 t2 p2 t2 States p1 t1 p1 t1 p3 p5 p3 p5 t3 t3 P1 Values P2 Values P3 Values P4 Values P5 Values Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 26/32

  61. Introduction Contribution Evaluation Conclusion Advanced omniscient debugging New advanced debugging services to navigate according to the different dimensions of the model run(net) fire(t1) fire(t2) A B C p4 p4 p4 Model p2 t2 p2 t2 p2 t2 States p1 t1 p1 t1 p1 t1 p3 p5 p3 p5 p3 p5 t3 t3 t3 P1 Values P2 Values P3 Values P4 Values P5 Values Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 26/32

  62. Introduction Contribution Evaluation Conclusion Advanced omniscient debugging New advanced debugging services to navigate according to the different dimensions of the model run(net) fire(t1) fire(t2) A B C p4 p4 p4 Model p2 t2 p2 t2 p2 t2 States p1 t1 p1 t1 p1 t1 p3 p5 p3 p5 p3 p5 t3 t3 t3 jump(A) P1 Values P2 Values P3 Values P4 Values P5 Values Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 26/32

  63. Introduction Contribution Evaluation Conclusion Advanced omniscient debugging New advanced debugging services to navigate according to the different dimensions of the model run(net) fire(t1) fire(t2) A B C p4 p4 p4 Model p2 t2 p2 t2 p2 t2 States p1 t1 p1 t1 p1 t1 p3 p5 p3 p5 p3 p5 t3 t3 t3 jump(A) stepValue(p4.tokens) P1 Values P2 Values P3 Values P4 Values P5 Values Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 26/32

  64. Introduction Contribution Evaluation Conclusion Evaluation of responsiveness Same case study: fUML models from [Maoz et al.’11] Comparison of our advanced debugger with two others: weak (no trace) and clone-based (generic trace) 100 TIME (MS) “Jump back” 10 1 execution 0.1 time (RQ2) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 MODEL ID MultiDimDebugger CloneBasedDebugger WeakDebugger 800 MEM. USAGE (KB) 600 Memory 400 consumption 200 0 (RQ3) 55 60 65 70 75 80 85 90 NUM STATES TRAVERSED MultiDimDebugger CloneBasedDebugger WeakDebugger Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 27/32

  65. Introduction Contribution Evaluation Conclusion Note regarding evaluation of time RQ2 (time) was evaluated in two different ways: Semantic differencing : the whole trace is read and compared Omniscient debugging : a single state is read to jump back Results are improved in both cases: Thanks to the dimensions when reading the whole trace Thanks to the precise capture of the execution concepts , when reading a single state Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 28/32

  66. Introduction Contribution Evaluation Conclusion Outline 1 Introduction 2 Contribution 3 Evaluation 4 Conclusion Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  67. Introduction Contribution Evaluation Conclusion Conclusion How to provide efficient execution trace management facilities for any xDSML? Contribution: new generative approach to manage execution traces of any xDSML Validation in a realistic context with applications to two existing V&V techniques using a real-world xDSML Results show that: – RQ1: Reduces complexity of domain-specific manipulations – RQ2: Reduces processing time by 75% – RQ3: Reduces memory consumption by 75% Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 29/32

  68. Introduction Contribution Evaluation Conclusion Short-term perspectives Customization of generated trace metamodels to better fit expected trace manipulations, e.g. ignore unused parts of the metamodel Represent execution branches sharing a common prefix, within a single execution trace Domain-specific property language to define temporal properties over traces (cf. [Maoz et al.’14] ) Domain-specific debugging services using the underlying domain-specific trace metamodel (cf. [Chi¸ s et al.’15] ) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 30/32

  69. Introduction Contribution Evaluation Conclusion Long-term perspectives Execution traces as core modeling artifacts “Live modeling” : instant visualization of the impact of a change in a model regarding its behavior Versionning of the behavior of models: “trace of traces” Generating everything for an xDSML Generating domain-specific mutation operators Generating a DSML to define test suites for an xDSML . . . Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 31/32

  70. Introduction Contribution Evaluation Conclusion Publications International conferences: Model omniscient debugging : E. Bousse, J. Corley, B. Combemale, J. Gray, and B. Baudry. “Supporting Efficient and Advanced Omniscient Debugging for xDSMLs”. SLE 2015. Trace metamodel generation : E. Bousse, T. Mayerhofer, B. Combemale, and B. Baudry. “A Generative Approach to Define Rich Domain-Specific Trace Metamodels”. ECMFA 2015. Model cloning : E. Bousse, B. Combemale, and B. Baudry. “Scalable Armies of Model Clones through Data Sharing”. MODELS 2014. + 4 international workshops papers Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 32/32

  71. Appendix Appendix Erwan Bousse Trace Management for Dynamic V&V for xDSMLs 33/32

  72. Appendix Outline 5 Appendix Evaluation Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  73. Appendix Types of semantics (I) Abstract syntax Execution metamodel Abstract syntax Abstract syntax Execution metamodel of A of A of A of B of B conforms to all conform to conforms to conforms to all conform to a a_exe (1) a b b_exe (1) initialization A to B initialization transf. A step B step a_exe (2) b_exe (2) B step A step a_exe (3) b_exe (3) Figure: Operational semantics Figure: Translational semantics Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  74. Appendix Types of semantics (II) Abstract syntax Abstract syntax Execution metamodel Execution metamodel of A of B of B of A conforms to conforms to all conform to all conform to a b b_exe (1) a_exe (1) A to B initialization B to A transf. B step transf. b_exe (2) A step (indirect) ... B steps b_exe (i) a_exe (2) B to A B step transf. b_exe (i+1) A step (indirect) ... B steps b_exe (i+n) a_exe (3) B to A transf. Figure: Translational semantics with back-annotation Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  75. Appendix Implicit steps Figure: Implicit step illustration <<abstract>> <<abstract>> RunImplicitStep SmallStep BigStep {ordered=true} subSteps <<abstract>> 0..* FireStep RunStep RunSubStep runParentStep 0..1 Figure: Petri net implicit step class Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  76. Appendix State of the art (summary) Versioning [Hartmann et al.’14] Generic structures Clone-based [Langer et al.’14] OO Software [Hamou-Lhadj et al.’10] Parallel soft. [Eschweiler et al.’11] Trace data Domain-specific structures structures fUML [Mayerhofer et al.’12] . . . Manual [Heged¨ us et al.’10] Domain-specific structure definition approaches Generative [Meyers et al.’14] Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  77. Appendix State of the art (table) Figure: Execution trace data structures Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  78. Appendix Screenshot GEMOC debugger (during execution) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  79. Appendix Screenshot GEMOC debugger (after jump back) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  80. Appendix fUML case study fUML xDSML subset of UML 57 classes fUML Models Case study of Maoz et al. to evaluate a semantic differencing operator Models drawn from industrial sources Multiple sequences of models, each sequence with a change 40 models considered, sizes range from 36 to 51 objects Source: http://www.se-rwth.de/materials/semdiff/ Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  81. Appendix Omniscient debugger architecture Generators Generic Multidim. Generic Trace DS Trace Omniscient Metamodel Metamodel Debugger Interface (generated) Execution Operational State Domain- Engine Semantics Manager speci fi c (DS) Execution Trace Depends on / Execution Executable Trace Uses Metamodel Model Constructor Conforms to Produces xDSML DS Trace Manager Modi fi es (generated) Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

  82. Appendix Example: a metamodel, a model, and runtime counterparts API Modeling framework <<interface>> <<interface>> code generation A B e.g. EMF +getB(): List<B> +getX(): int +getI(): int +setX(int) +setI(int) -b +getJ(): int * +setJ(int) Metamodel "AB" BImpl implements +x: int A AImpl B b +i: int +i: int * +x: int +(mut) j: int +j: int instance of instance of instance of conforms to :BImpl :BImpl :B :B :AImpl x = 5 x = 7 :A x = 5 x = 7 i = 1 representation of j = 2 i = 1 j = 2 Runtime representation Model "abb" Example of system Erwan Bousse Trace Management for Dynamic V&V for xDSMLs

Recommend


More recommend