Testing of CAN-Bus Devices with TTCN-3 A. Deinlein, A. Ulrich Siemens AG, Germany {andreas.deinlein, andreas.ulrich}@siemens.com 2 Testing of CAN-Bus Devices with TTCN-3 agenda � Introduction � Testing tool selection � TTCN-3 test architecture � Experiences and results ��������������������������������� 1
3 Testing of CAN-Bus Devices with TTCN-3 project domain What we do at Siemens MED CO ID - Development of components for clinical imaging devices - Here: collimators for X-ray devices collimator ��������������������������������� 4 Testing of CAN-Bus Devices with TTCN-3 keywords table stand CAN > collimator CAN CAN - controls shape and density of X-rays in medical devices CAN - Automatic Collimator with up to 12 motor driven axles X-ray generator > CAN > embedded software (Controller Area Network) - controls the axles, handles user collimator communicates over the interaction and communicates CAN fieldbus via CANopen with a with the master master (tester) - Infineon 16-Bit � C (XC167) > TTCN-3 > black-box test automation testing language standardized by automated test of the integrated ETSI embedded device via the CAN interface ��������������������������������� 2
5 Testing of CAN-Bus Devices with TTCN-3 requirements to the test automation tool � CAN protocol supported � Ease of use: simple test case design � Automatic report generation � Real-time constraints (deterministic response time of the tester of < 5 ms) � Configuration of test execution (test selection and test parameters) � Execution of concurrent test threads � Capable to perform over-night regression tests ��������������������������������� 6 Testing of CAN-Bus Devices with TTCN-3 test automation – tool selection Systematic tool comparison 1) Feature matrix TTCN-3 CANoe (Vektor Informatik) CAN supported adaptable yes ease of use new language graphical tool abstract test architecture yes limited report generation freely adaptable limited real-time constraints tool dependant yes configuration of execution yes yes concurrent test cases yes limited supported HW devices freely adaptable only Vector CAN-devices over-night regression tests yes yes total points 203 187 April 2005 TTCN-3 selected ��������������������������������� 3
7 Testing of CAN-Bus Devices with TTCN-3 test automation – tool selection concurrent testflow with 3 PTCs: Evaluation of TTCN-3 products I 2) - CANopen PTC: cyclic 100ms timer - motor axle PTC: move to all possible positions - executed under windows with priority HIGH ��� ��� 1.) timer accuracy: ��� �������������� ������� myTimer. start( 0.1 ) ; ��� ������������� alt { ������������ ��� �������������� [] myTimer. timeout { ������������ myTimer. start( 0.1 ) ; ��������������� �������������� ������������ repeat ; ����������� ������������ ������������ } � �� �������������� } ������������ �������������� � �� � timer accuracy := deviation from cyclic 100ms timer ������������ � �� ������������ � �� 2.) tester response time: �������������� ������������ � �� �������������� colPort. send (startAxlePPM(getMessageID(axle,RPDO1))); ������������ � �� �������������� ackTimer. start ; �������������� ������������ alt { � �� �������������� [] colPort. receive (axleStateAck(getMessageID(axle,TPDO1))) { ������������ � �� ackTimer. stop ; !!! !!! "���������#������������$ "���������#������������$ ������������ } � �� time [] ackTimer. timeout { ���� setverdict(fail); ���� stop ; ���� } } colPort. send (resetAckAxlePPM(getMessageID(axle,RPDO1))); � tester response time := time from receiving a message till sending the response ��������������������������������� 8 Testing of CAN-Bus Devices with TTCN-3 test automation – tool selection Evaluation of TTCN-3 products II 2) RT Performance Danet Toolbox 2.8p5 Telelogic TAU/Tester 2.4 tester response time max.: 0,9 ms; av.: 0,8 ms max.: 0,8 ms; av.: 0,5 ms % = 0,43 % = 0,76 timer accuracy April 2005 frequency 100 � � � required performance achieved � Frequency chart for 90 timer accuracy � � 0,15% of timeouts outside of � � 80 expected 5ms window 70 (Danet + Telelogic) 60 � CANoe: � = 0,54; no timeouts � � � outside 5 ms window !!! 50 CANalyzer 40 Danet 30 Telelogic 20 Danet Toolbox 10 0 selected 98,10 98,40 98,70 99,00 99,30 99,60 99,90 100,20 100,50 100,80 101,10 101,40 101,70 102,00 102,30 102,60 102,90 CANalyzer timeout value 103,20 103,50 103,80 ��������������������������������� 4
9 Testing of CAN-Bus Devices with TTCN-3 test architecture TTCN-3 module concept: • only “behaviour.3mp” differs between test cases libTCI_TM.a (Danet) • other modules are part of test libraries Codecs (libCodec.a) behaviour.3mp • Codecs automatically generated artis_architecture.3mp canopen_functions.3mp �� �� from XML-File with CAN message artis_functions.3mp canopen_datatypes.3mp representation artis_datatypes.3mp canopen_testdata.3mp artis_testdata.3mp canopen_declarations.3mp artis_declarations.3mp • TRI-Library from Danet extended TRI-Library (libTri.a) (Danet) PA-Library (libnt.a) (Danet) with support for CAN-HW • Vector XL devices • I+ME Actia Level X • can4linux (in work) ��������������������������������� 10 Testing of CAN-Bus Devices with TTCN-3 abstract test design Testdeveloper artis_architecture.3mp XML Testspecifier type port CCollimator_PortType message ... generated type component MTC behaviour.3mp ... canopen_datatypes.3mp module behaviour { type record NMT_MessageType { modulepar { integer config}; canopen_testdata.3mp integer messageID, import from artis_architecture all; integer node_ID, import ... template NMT_MessageType resetNMT integer cs … } with { encode "Canopen" } testcase tcTemplate() canopen_functions.3mp runs on MTC system TSI function fNMTReset (in integer nodeid) { runs on MTC fRegisterCollimatorIDs(config); { map ( self :colPort, system :colPort); timer emcyTimer := 20.0; //***** USER AREA ******************* fTeststep("NMT-Reset"); colPort. send (resetNMT(nodeid)); fNMTReset(64); emcyTimer. start (25.0); //********************************** alt { [] colPort. receive (bootup(1792 + nodeid)) { unmap ( self :colPort, system :colPort); repeat ; } } [] emcyTimer. timeout { control { fText("EMCY-Timeout abgelaufen"); execute (tcTemplate(), 500.0); setverdict(fail); } } } } fTeststepEnd(); ��������������������������������� } 5
11 Testing of CAN-Bus Devices with TTCN-3 test automation with ttcn-3 – test structure teststand TTCN-3 tester light emitter SUT image field CCD camera ��������������������������������� 12 Testing of CAN-Bus Devices with TTCN-3 test automation with ttcn-3 - workflow Workflow: HTML HTML Results test- test- Results test- test- transform EXE EXE (XML) spec. script (XML) spec. script execute compile PDF PDF ��������������������������������� 6
13 Testing of CAN-Bus Devices with TTCN-3 project status � Productive use since June 2005 � Test scripts for 7 collimator product variants realized (planned for about 20 variants) � About 50 test cases implemented (with about 100 lines of TTCN-3 code each) � About 6000 lines of TTCN-3 code in library modules ��������������������������������� 14 Testing of CAN-Bus Devices with TTCN-3 achieved improvements in days Test automation used 100 85% of errors 90 80 found in comp. IT 70 result from test test efforts 60 50 automation 40 30 16% of all errors 20 10 are found in 0 codereview unittest component IT reliability system IT reliability tests tests 60 50 40 found errors Only 3 errors are 30 found in system 20 integration tests 10 0 codereview unittest component IT reliability tests system IT ��������������������������������� 7
Recommend
More recommend