Situation coverage testing for autonomous robots Patrizio Pelliccione Associate Professor, University of L’Aquila, Italy Associate Professor, Chalmers|GU www.patriziopelliccione.com H2020-ICT-2016
In the near future…
In the near future??
Situation coverage • Take care about the range and diversity of the situations in which we simulate and/or test autonomous robots • Make sure that the situation coverage of our testing is adequate
Situation coverage for autonomous robots Mission, set of tasks, that the robot should accomplish Context in which the mission will be executed
Situation coverage for autonomous robots Mission, set of tasks, that the robot should accomplish Context in which the mission will be executed
What are the typical missions robots are asked to perform?
Specification Patterns • What kind of missions are specified in practice? • Identification of missions already specified in practice (i.e. papers, documents of robotic companies) • Definition of a catalogue of mission specification patterns • Tool support for assisting users in the specification of missions via the use and instantiation of patterns Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, http://roboticpatterns.com Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019
Specification Patterns • Methodology • Collection of mission requirements • We collected mission requirements from papers within the robotic scientific literature and from software engineering venues (mostly absent) • Classification of mission concerns • We linguistically decomposed each mission requirement into terms describing concerns inherent in it • Utilizing the concerns obtained from the mission requirements in the form of terms, we created a graph structure that represents relations between mission concerns • If a node represents some temporal behavior that concerns movement, it is a candidate pattern • Pattern identification and formulation Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, http://roboticpatterns.com Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019
How to formulate the patterns? “A robot r shall visit the two locations l1 and l2 in this order”
How to formulate the patterns? “A robot r shall visit the two locations l1 and l2 in this order” l1 and then l2
How to formulate the patterns? Ambiguity “A robot r shall visit the two locations l1 and l2 in this order” l1 and then l2 Is it possible to visit l2 before l1 and then to visit l2?
How to formulate the patterns? Intuitive but rigorous “A robot r shall visit the two locations l1 and l2 in this order” l1 and then l2 Is it possible to visit l2 before l1 and then to visit l2? Φ1=<>((r in l1) && <>(r in l2)) vs. φ2 = φ1 && ((!r in l2)U(r in l1))
Patterns identification Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, http://roboticpatterns.com Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019
Specification Patterns Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, http://roboticpatterns.com Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019
Specification Patterns Claudio Menghi, Christos Tsigkanos, Patrizio Pelliccione, Carlo Ghezzi, and Thorsten Berger, http://roboticpatterns.com Specification Patterns for Robotic Missions, Transactions on Software Engineering (TSE), 2019
An example of pattern
Automating tool: PsAlM http://roboticpatterns.com/psalm/ Claudio Menghi, Christos Tsigkanos, Thorsten Berger, Patrizio Pelliccione, “PsALM: Specification of Dependable Robotic Missions,” in 41st International Conference on Software Engineering (ICSE), Demonstrations Track , 2019.
Main components of the tool http://roboticpatterns.com/psalm/ Claudio Menghi, Christos Tsigkanos, Thorsten Berger, Patrizio Pelliccione, “PsALM: Specification of Dependable Robotic Missions,” in 41st International Conference on Software Engineering (ICSE), Demonstrations Track , 2019.
Validation • RQ1 : To what extent are real-world, natural-language mission requirements expressible using our pattern catalog? • RQ2 : To what extent are real-world mission specifications expressible using our pattern catalog? • RQ3 : Does the pattern catalogue support the formulation of mission requirements and specifications in a set of real-world scenarios defined in collaboration with our industrial partners?
Validation • RQ1 : How effective is the pattern catalog in capturing mission requirements and producing mission specifications? • We investigated • How the pattern catalog supports the specification of mission requirements • How the pattern catalog reduces ambiguities in mission requirements • We collected mission requirements in natural language from available requirements in 11 different robotic applications produced from Spectra (428 mission requirements) and LTLMoP (16 mission requirements) • The results show that most of the mission requirements (370 over 436 ) were expressible using the pattern catalog, which is a reasonable coverage for pattern catalog usage. The 66 mission requirements that are not covered suggested the introduction of new patterns
How to specify complex missions?
Domain Specific Language to specify missions • Specification patterns enable the specification of “simple” missions and of one single robot • PsAlM supports the composition of patterns to specify more complex missions, but it supports only AND and OR operators • We developed PROMISE to overcome these limitations • Patterns are basic building blocks • We added operators to compose patterns for specifying complex missions for multi-robots Sergio Garcia, Patrizio Pelliccione, Claudio Menghi, Thorsten Berger, Tomas Bures, “Higher-Level Mission Specification for Multiple Robots,” in 12th ACM SIGPLAN International Conference on Software Language Engineering (SLE) , 2019.
Operators of the DSL
Operators of the DSL
Promise tool chain Abstraction Layers DSL: Domain Specific Language Intermediate Language Patterns Co4robots Planner Spectra Simbad NuSMV
SERA (Self- adaptive dEcentralized Robotic Architecture) robot status local mission Central High level ] 4 Robot shared map t o local mission[robot1] station specification manager b [robot state, mission status o r Mission [ n high level specification environment] o Communication & i s Local mission manager Management s i collaboration manager m Global mission manager Layer l a c local mission o l 1 ] global mission o t b o r [ n o s i s m i Global mission decomposer a l c Adaptation manager Information manager o l Change plan [plan status] Management [gestures [call status] robot state Layer Plan executor environment] Legend call Component method Motion planning & control Gestures recognition SLAM sharing call notify Control Layer component unit Detection executor Robot interface Sensors interface Hardware & drivers S. García, C. Menghi, P. Pelliccione, T. Berger and R. Wohlrab, "An Architecture for Decentralized, Collaborative, and Autonomous Robots," 2018 IEEE International Conference on Software Architecture (ICSA) , Seattle, WA, 2018, pp. 75-7509.
Validation • RQ1 : does PROMISE effectively support the specification of complex missions? We specified three of the missions proposed for the 2018’s edition of RoboCup@Home (Stage II) + the mission in the example
Validation • RQ2 : how simple is it to specify missions with PROMISE? • All the participants of the second study were able to completely solve the three proposed blocks and • We conducted two user studies: validate them using the generated natural • The first to have feedback to improve the language language file within the expected time frame (30 • The second, the final validation minutes for each block). • They were also able to validate at least the first two blocks through simulation within the time frame.
Situation coverage testing for autonomous robots Patrizio Pelliccione Associate Professor, University of L’Aquila, Italy Associate Professor, Chalmers|GU www.patriziopelliccione.com H2020-ICT-2016
Recommend
More recommend
