Prahladavaradan Sampath † Joint work with Rajeev A.C † and Ramesh S ‡ * This work was carried out when the authors were part of General Motors India Science Lab † Mathworks ‡ ECI Lab, General Motors R&D, Warren
Model-Based Development To develop complex software systems Model Validate Refine Auto-generate code Employs high-level modeling languages Formal syntax less ambiguous than natural language Formal semantics enables automated analyses Highly tool intensive Syntax checking, Simulation, Analysis, Test generation, Code generation (Collectively called model processors) Advantages Less development time, ease of re-design Early verification and debugging Model-based test-case generation Automatic code generation
Code Generator Code generators are tools that take as input “models” in a modelling language and output various artifacts: Code Other models (one man’s model is another man’s code) Examples of code-generators Rhapsody code-generator Matlab/Stateflow simulator Lex/Yacc Query optimizers ...
Approaches to Verify Code Generators Formally verifying the code generator White-box, one-time, interactive, strong guarantee Testing the code generator Black-box, one-time, automated, weak guarantee Manual / automated test generation Special ATG methods to handle syntactic and semantic structure of inputs and outputs Model based testing (most common in practice) Black-box, every-run, automated, weak guarantee Translation validation Black-box, every-run, automated, strong guarantee
Different Approaches Proving a code generator m:models, i:inputs: ModelExec(m, i) CodeExec(CodeGen(m),i) Testing a code generator Formany m:models, Formany i:inputs: ModelExec(m, i) CodeExec(CodeGen(m),i) Translation validation : fix a model m i:inputs: ModelExec(m, i) CodeExec(CodeGen(m),i)
Translation Validation Mathematical proof of equivalence between model and program Every translation is followed by validation Strengths Strong guarantee Does not require source code of translator Automated Weaknesses Validation has to be done after every run of the translator Computation intensive Based on the following assumptions Formal semantics of the modeling and programming languages are available Behaviours of the model and program are finite in number A mapping can be identified between model elements and program elements Verification conditions can be proved
Tool Architecture
Step-1 Obtain all behaviours of the given Stateflow model Using a formal semantics for Stateflow Generate all possible inference trees corresponding to the given model Using inference rules in semantics Iterate over all “proofs” using a Hoare logic style semantics Assumes “bounded” behaviour – no loops!
Step-2 Generate verification conditions from inference trees As Hoare tuples: {Pre-condition} Ch {Post-condition} Active states before and after execution Identify from the structure of the inference tree Variable values before and after execution Extract the sequence S of guards and actions from the inference tree Guards: boolean conditions over variables, presence/absence of events Actions: variable assignments, event broadcasts Compute wp(S, true) wp(x ← exp, P) = P[x/exp], wp(event + (e), P) = P && e + , wp(event - (e), P) = P && e - Symbolically execute S with respect to wp(S, true) Assuming wp(true, S) = P(x 1 , …, x n ), we compute symsim(P, S) = Q(x 1 , …, x n , x’ 1 , …, x’ n )
Example: Shift_logic 37 inference trees = 37 unique behaviours An inference tree: Pre-condition: all states are inactive (WP calculation) Post-condition: gear_state, first, selection_state and steady_state are active, gear == 1 (Symbolic simulation) {true} gear = 1 {true}
Example: Shift_logic 37 inference trees = 37 unique behaviours An inference tree: Pre-condition: all states are inactive (WP calculation) Post-condition: gear_state, first, selection_state and steady_state are active, gear == 1 (Symbolic simulation) {true} gear = 1 {gear == 1}
Step-3 Identify the mapping between model elements and code elements Files: md.c, md_data.c, md.h, md_private.h Chart ch: function void md_ch(void) Events: integer variable _sfEvent_md_ with values from {md_event_e1, …, md_event_en, CALL_EVENT} State s: field is_active_s ( boolean ) and field is_s ( {md_IN_s1, …, md_IN_sn, md_IN_NO_ACTIVE_CHILD} ) in structure variable md_DWork History junction in s: field was_s ( boolean ) in md_DWork Local variables: fields in structure variable md_B Inputs: fields in structure variable md_U
Step-4 Prove the verification conditions on C code Annotate the generated C code with {Pre-condition} and {Post- condition} Use the mapping between model elements and code elements Prove using C model-checker CBMC Failed proof can provide a test-case showing the difference between the behaviours of model and code
Annotated Code C file: atc.c
Some Case-studies Shift_logic in ATC demo model: 37 verification conditions A number of models with history junctions, event broadcasts, graphical functions, multi-level transitions, etc. HVAC controller models
Challenges Semantics of modelling language Is our formalization correct? Binary Yes/No answer is not great Can we do better?
User Feedback Generate test-cases from proofs Any proof visualization techniques? Tabulation of all cases and reporting? …
Testing the Semantics Formal Meta-model Test Specification What would we like to test? Syntax and Semantics of Test Generator Stateflow Model + Input/Output Input event / output Stateflow model action sequence Code Generator Under Test Test Harness
Testing the Semantics Test case Expected Model Inputs Outputs ≡? Code Generator Under Test Actual Execution Code Outputs Test Harness
Examples of Semantic Rules Semantics for a lexical analyzer
Examples of Semantic Rules Semantics for a simple while-language
Examples of Semantic Rules Inference rules for Stateflow: Entering an atomic state s by a transition Entering an OR state by a transition, and its child state by default transition
Generating Test-Cases Generate a set of “proof - trees” based on coverage criteria Given a particular behaviour as a generated “proof - tree” Compute possible models, inputs and outputs that give rise to the given behaviour {b} Invert semantics! If b then x := e1 else skip x := e2 {x = e2[e1[x’/x]/x]}
Reveals Subtle Bugs/Issues History junction bug: Inputs: e1 e2 Expected: D2 C1 X1 T1 E1 Actual: D2 C1 X1 T1 C4 T4 E1 Above bug in V6.2.1, fixed in V7.0
Reveals Subtle Bugs/Issues
References An Axiomatic Semantics for Stateflow. In preparation Translation Validation for Stateflow to C. Under submission CoGenTe: A Tool for Code Generator Testing. IEEE/ACM International Conference on Automated Software Engineering (ASE’10), Antwerp, Belgium, 2010. Behaviour Directed Testing of Auto-code Generators. IEEE International Conference on Software Engineering and Formal Methods (SEFM'08), Cape Town, SA, 2008. Verification of Model Processing Tools . Safety-Critical Systems Session, SAE World Congress & Exhibition ( SAE'08 ), Detroit, USA, 2008. How to Test Program Generators? A Case Study using flex . IEEE International Conference on Software Engineering and Formal Methods (SEFM'07), London, UK, 2007. Testing Model-Processing Tools for Embedded Systems . IEEE International Real-Time and Embedded Technology and Applications Symposium ( RTAS'07 ), Bellevue, WA, USA, April 2007.
Recommend
More recommend