Verification Strategy & Tools for IoT Systems Design Review Team - PowerPoint PPT Presentation

Verification Strategy & Tools for IoT Systems Design Review Team 2013 : Wissam Razouki, Long Phan, Balsha Maric Faculty Adviser : Shalabh Gupta Sponsor : Carrier Massimiliano D’Angelo and James Fan

Overview ● Introduction Background ● ○ IoT Structure ○ Project Tasks/Requirements ○ IoT Tech ○ Types of IoT Testing ● Design Approaches ○ Testing Methods Framework ○ ○ Testing Architectures Types of HVAC to Consider ○ ● Timeline ● What’s next

Introduction ● Many types of IoT (Internet of Things) systems; need a way to test them ● Four main components of IoT system: ○ Devices - sensors push data to cloud and receive commands through gateways ○ Gateway - used to interface between sensors/actuators and cloud ○ Cloud - stores/analyzes data Applications - usually mobile/web app to monitor ○ data and/or control machine (actuators) ● Our focus: end-to-end communication between Carrier Wi-Fi thermostat interface and gateway , focusing on HVAC systems

Simple IoT Structure Telemetry - Diagnostics data from sensors Commands - Issued by apps, executed by actuators Servers used to store and analyze data, and interface with gateways/apps

Modified IoT Structure Testing Architecture

Technologies Used in IoT ● RFID - uses radio frequency NFC - mostly used in mobile devices ● ● Bluetooth - close range wireless comm RFID ● Z-Wave - low power RF tech. Mainly used for home automation (lamp controlling, etc.) Wi-Fi - most common in IoT ● ● 2G/3G/4G/5G ● Others

Assigned Tasks from Sponsor Task 1: Define a reference IoT system architecture Task 2: Review literature for state-of-art in IoT system verification Task 3: Synthesize proposal for IoT testing verification workflow Task 4: Toolchain selection to support the IoT testing verification workflow Task 5: (Partial) Demonstration of the proposed workflow and toolchain

Types of IoT Testing to Focus on Our goal is to implement a system capable of performing the following: ● Interoperability testing - test how well devices interact with each other and make use of information/data ● Stability/reliability testing - test how much the system is prone to failure, especially in extreme conditions ● Performance testing - test speed of data transfer across entire system (latency) ● Usability testing - test if the system is providing/can provide the required data

Challenges to Consider ● Complexity of hardware-network-software mesh Device interaction in real time ● ○ Security, backward compatibility, upgrade issues ● Real-time data testing is difficult Need to consider many different UIs (user interfaces) ● ● Operation under various network availabilities/strengths ● Many network protocols need to be tested (MQTT, CoAP, HTTP, etc) ● New field; limited research material and tools

IoT Testing Architecture - Testing-as-a-Service (TaaS) ● In this system three main testing phases are ○ Interoperability Testing ○ Conformance Testing ○ Semantics Validation Testing ● Tests are outsourced and ran sequentially Cloud-based automated IoT testing oneM2M: Machine-to-machine standard for IoT IUT: Implementation Under Test

Another Example ● Tests are broken up even more

Possible Testing Methods/Tools ● Wireshark - Network layer Free and open-source software for analyzing packets sent over the internet (data integrity and ○ transfer speeds) Apache JMeter - App layer ● ○ Java open-source software that analyze and measure the performance of web applications or a variety of services (test functional behavior) ● Build small IoT systems and run simulations - System IoTIFY process

Heat Pump Types of HVAC Systems ● Heat Pump Rooftop Unit ● ● Water Source Heat Pump ● Packaged HVAC ● Split System HVAC Ductless System ●

Required budget: little to none

What to do next... Fall 2019: ● Continue research in IoT testing architectures and methods Meet with sponsor to discuss various architectures and pick one ● ● Review literature on chosen architecture ○ Modifications may be needed to satisfy our goals ● Propose a workflow Spring 2020: Select the necessary tools to implement workflow ● ○ IoT system(s) is/are needed in order for testing to begin ○ Extensive testing and simulations required to validate our design ○ Modification to original architecture might be needed (Partially) Demonstrate their application on a sample IoT system ● ○ Final testing and video demo

References 1. https://www.guru99.com/iot-testing-challenges-tools.html 2. https://www.softwaretestinghelp.com/internet-of-things-iot-testing/ 3. https://www.buildings.com/news/industry-news/articleid/21020/title/drive-hvac-efficiency-with-internet -of-things-functionality 4. https://www.propmodo.com/how-iot-is-taking-over-the-hvac-industry/ 5. http://twentyonecelsius.com.au/blog/everything-you-need-to-know-about-hvac-systems/ 6. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8281514 7. https://thesai.org/Downloads/Volume10No5/Paper_45-IoT_Testing_as_a_Service.pdf 8. https://www.clariontech.com/blog/iot-testing-framework 9. https://radiostud.io/iot-testing-tools-building-resilient-iot-platforms/