managing app testing device clouds issues and
play

Managing App Testing Device Clouds: Issues and Opportunities Mattia - PowerPoint PPT Presentation

Jan 18, 2024 •261 likes •321 views

Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version

  1. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by a device farm engineer) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

  2. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by a device farm engineer) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

  3. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by device farm engineers) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

  4. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by a device farm engineer) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

  5. Managing App Testing Device Clouds: Issues and Opportunities Mattia Fazzini Alessandro Orso Problem Description Study Methodology Android Mobile App Testing Devices and Tests Considered AWS Compatibility Device Farm Android Version #Devices CST ver. #Modules #Tests J ELLY B EAN 4.2 2 50 17268 4.2.2_r1 Test Suite J ELLY B EAN 4.3 5 58 18013 4.5_r2.2-cts K IT K AT 4.4 38 73 24039 cts-4.4_r4 L OLLIPOP 5.0 11 93 33929 cts-5.0_r9 L OLLIPOP 5.1 6 96 34207 cts-5.1_r28 M ARSHMALLOW 6.0 15 114 37613 cts-6.0_r32 N OUGAT 7.0 7 124 43494 cts-7.0_r32 Could Testing Infrastructures N OUGAT 7.1 2 133 43598 cts-7.1_r29 Oreo 8.0 11 195 59459 cts-8.0_r20 AWS Device Farm Firebase Test Lab App Center O REO 8.1 4 210 60383 cts-8.1_r16 P IE 9 9 289 84182 cts-9.0_r9 ‣ We used the Compatibility Test Suite to analyze the behavior of 110 devices on the Device farm. We collected a total of 12 million test executions. Study Results and Analysis Future Work ‣ Create a taxonomy of the failures ‣ Identify per-vendor and per-version failures ‣ We observed 9,778 failures and all devices experienced at least one failure ‣ Investigate automatic generation of ‣ We investigated general issues with the cloud infrastructure compatibility-based tests ‣ 58 tests failed on all devices (27 confirmed by a device farm engineer) ‣ Study light-weight monitoring of cloud-based ‣ We investigated security-related issues test executions ‣ 1,307 security-related test failures (caused by 153 different tests)

Recommend

Q-Clouds: Managing Performance Interference Effects for QoS-Aware Clouds Ripal Nathuji Aman

Q-Clouds: Managing Performance Interference Effects for QoS-Aware Clouds Ripal Nathuji Aman

Q-Clouds: Managing Performance Interference Effects for QoS-Aware Clouds Ripal Nathuji Aman Kansal Alireza Ghaffarkhah Presented by Joshua Davis Motivation and Background Cloud computing Off load processing and storage Charged per

549 views • 28 slides
When you look up into the sky, you will often see clouds. No two clouds are the same, and there

When you look up into the sky, you will often see clouds. No two clouds are the same, and there

When you look up into the sky, you will often see clouds. No two clouds are the same, and there are many different types of clouds. Different kinds of clouds lead to different types of weather. Cirrus clouds are the highest group of clouds that

271 views • 5 slides
Solving Device Tree Issues Use of device tree is mandatory for all new ARM systems. But the

Solving Device Tree Issues Use of device tree is mandatory for all new ARM systems. But the

Solving Device Tree Issues Use of device tree is mandatory for all new ARM systems. But the implementation of device tree has lagged behind the mandate. The first priority has been correct function. Lower priorities include device tree

2.4k views • 183 slides
Seamless Offloading of Web App Computations From Mobile Device to Edge Clouds via HTML5 Web

Seamless Offloading of Web App Computations From Mobile Device to Edge Clouds via HTML5 Web

Seamless Offloading of Web App Computations From Mobile Device to Edge Clouds via HTML5 Web Worker Migration Hyuk Jin Jeong Seoul National University SoCC 2019 Virtual Machine & Optimization Laboratory Department of Electrical and

824 views • 29 slides
Treatment Device Testing Protocols in the Garden State:

Treatment Device Testing Protocols in the Garden State:

Treatment Device Testing Protocols in the Garden State: Current Status of Laboratory Testing and Field Longevity Monitoring Washington Stormwater Center 2013

387 views • 16 slides
ykpang@beyondsecurity.com Fuzz Testing for Embedded Device Security Assurance (EDSA) ISASecure

ykpang@beyondsecurity.com Fuzz Testing for Embedded Device Security Assurance (EDSA) ISASecure

Empowering Security Teams ykpang@beyondsecurity.com Fuzz Testing for Embedded Device Security Assurance (EDSA) ISASecure EDSA Certification Communication/Network Robustness Testing (CRT) CRT examines the capability of the device to

859 views • 33 slides
Practical Issues with HIV Practical Issues with HIV Testing, CD4 Count and Viral Testing, CD4

Practical Issues with HIV Practical Issues with HIV Testing, CD4 Count and Viral Testing, CD4

Practical Issues with HIV Practical Issues with HIV Testing, CD4 Count and Viral Testing, CD4 Count and Viral Load testing Load testing 2 ND Phnom Penh Symposium on HIV 2 ND Phnom Penh Symposium on HIV Marie Downer, CDC; Chuop Sokheng, NIPH,

104 views • 8 slides
State of the Art and Open Issues in Software Testing State of the Art and Open Issues in

State of the Art and Open Issues in Software Testing State of the Art and Open Issues in

State of the Art and Open Issues in Software Testing State of the Art and Open Issues in Software Testing Software Testing d e a t g t i s e v i n d n a Most used approach 18 13.5 9 4.5 0 2000 2002 2004 2006 2008 2010

975 views • 84 slides
The Missing Satellites of the Magellanic Clouds Testing LCDM Predictions on Small Scales Nitya

The Missing Satellites of the Magellanic Clouds Testing LCDM Predictions on Small Scales Nitya

The Missing Satellites of the Magellanic Clouds Testing LCDM Predictions on Small Scales Nitya Kallivayalil Small Galaxies, Cosmic Questions, Durham, UK, 2019 Ultra-Faint Intrigue [M * (0.11) 10 4 M ; M halo ~ 10 8 M ]

241 views • 11 slides
Solving Device Tree Issues - part 3 Using devicetree is painful. The framework does not help to

Solving Device Tree Issues - part 3 Using devicetree is painful. The framework does not help to

Solving Device Tree Issues - part 3 Using devicetree is painful. The framework does not help to develop correct system descriptions (device tree source) and drivers. Errors are often not visible or are hard to understand. But tools and

2.09k views • 140 slides
Solving Device Tree Issues - part 2 Debugging devicetree issues is painful. Last year I

Solving Device Tree Issues - part 2 Debugging devicetree issues is painful. Last year I

Solving Device Tree Issues - part 2 Debugging devicetree issues is painful. Last year I presented some under-development tools and techniques to debug devicetree issues. This year I will provide an update on the status of those tools and

1.02k views • 85 slides
IDENTIFYING & MANAGING GCP COMPLIANCE RISKS FOR THE PHARMACEUTICAL, BIOTECH & DEVICE

IDENTIFYING & MANAGING GCP COMPLIANCE RISKS FOR THE PHARMACEUTICAL, BIOTECH & DEVICE

IDENTIFYING & MANAGING GCP COMPLIANCE RISKS FOR THE PHARMACEUTICAL, BIOTECH & DEVICE INDUSTRIES Karen Weaver Epstein, Becker & Green Health Care & Life Sciences Practice APPLICABLE REGULATIONS 21 CFR 312 - Investigational New

527 views • 24 slides
Improving Agility and Elasticity in Bare-metal Clouds Yushi Omote , Takahiro Shinagawa ,

Improving Agility and Elasticity in Bare-metal Clouds Yushi Omote , Takahiro Shinagawa ,

Improving Agility and Elasticity in Bare-metal Clouds Yushi Omote , Takahiro Shinagawa , Kazuhiko Kato University of Tsukuba, The University of Tokyo 1 Bare-metal Clouds An IaaS for high performance and device functionality

230 views • 21 slides
Handheld Infrared Device Dr Richard Stewart, Managing Director, Ziltek Pty Ltd Conference on

Handheld Infrared Device Dr Richard Stewart, Managing Director, Ziltek Pty Ltd Conference on

Rapid Mapping of Soils in China using a Handheld Infrared Device Dr Richard Stewart, Managing Director, Ziltek Pty Ltd Conference on Remediation of Chlorinated and Recalcitrant Compounds, 8-12 April 2018 Acknowledgements 2 Contaminated Soil in

399 views • 20 slides
Social Clouds Creating a research agenda Andrew Lippman MIT Media Lab October, 2010 Clouds and

Social Clouds Creating a research agenda Andrew Lippman MIT Media Lab October, 2010 Clouds and

Social Clouds Creating a research agenda Andrew Lippman MIT Media Lab October, 2010 Clouds and Mists: 1 st Cloud Defined by access Familiar as the internet Result of a nexus: PC, backbone, ISPs Web Opened vast doors to creativity Clouds and

592 views • 24 slides
SeeTest Continuous Testing Platform 1 Layout (Visual) Testing Integrate visual testing into your

SeeTest Continuous Testing Platform 1 Layout (Visual) Testing Integrate visual testing into your

Layout (Visual) Testing SeeTest Continuous Testing Platform 1 Layout (Visual) Testing Integrate visual testing into your CI/CD flow to detect and fix responsive design issues before production Create automated tests to verify UI responsiveness

621 views • 8 slides
Session 3: Hydrology & Clouds 3:00- 5:30 PM Session 3: Hydrology & Clouds 3:00- 5:30 PM

Session 3: Hydrology & Clouds 3:00- 5:30 PM Session 3: Hydrology & Clouds 3:00- 5:30 PM

Session 3: Hydrology & Clouds 3:00- 5:30 PM Session 3: Hydrology & Clouds 3:00- 5:30 PM Session 3: Hydrology & Clouds 3:00- 5:30 PM Session 3: Hydrology & Clouds 3:00- 5:30 PM Theme: Basically a continuation of climate talks

453 views • 19 slides
RESOURCE MANAGEMENT RESOURCE MANAGEMENT STRATEGIES FOR SDR CLOUDS STRATEGIES FOR SDR CLOUDS Vuk

RESOURCE MANAGEMENT RESOURCE MANAGEMENT STRATEGIES FOR SDR CLOUDS STRATEGIES FOR SDR CLOUDS Vuk

Department of Signal Theory and Communications UNIVERSITAT POLITCNICA DE CATALUNYA RESOURCE MANAGEMENT RESOURCE MANAGEMENT STRATEGIES FOR SDR CLOUDS STRATEGIES FOR SDR CLOUDS Vuk Marojevic Ismael Gomez Pere Gilabert Gabriel Montoro

399 views • 27 slides
Mobile Device Forensics - Tool Testing Richard Ayers Disclaimer Certain commercial entities,

Mobile Device Forensics - Tool Testing Richard Ayers Disclaimer Certain commercial entities,

Mobile Device Forensics - Tool Testing Richard Ayers Disclaimer Certain commercial entities, equipment, or materials may be identified in this presentation in order to describe an experimental procedure or concept adequately. Such

257 views • 23 slides
October 26, 2009 Current Issues Wii Remote Payload Testing Status Tasks for this

October 26, 2009 Current Issues Wii Remote Payload Testing Status Tasks for this

October 26, 2009 Current Issues Wii Remote Payload Testing Status Tasks for this week Purchasing Issues Jason Boyd is the class administrator contact He says it is easier if we send him forms via email Agenda

462 views • 6 slides
Nquire ask anything Anis Abboud, Chris Snyder, Mario Finelli Device 1 Device 2 Device 1

Nquire ask anything Anis Abboud, Chris Snyder, Mario Finelli Device 1 Device 2 Device 1

Nquire ask anything Anis Abboud, Chris Snyder, Mario Finelli Device 1 Device 2 Device 1 Device 2 Device 1 Device 2 Device 1 Device 2 Device 1 Device 2 Device 1 Device 2 Device 1 Device 2 Device 1 Device 2 Device 1 Device 2

821 views • 49 slides
Spectacles, Eugene, and Kepler Managing Synthetic Biology Device Development Lesia Bilitchenko,

Spectacles, Eugene, and Kepler Managing Synthetic Biology Device Development Lesia Bilitchenko,

Spectacles, Eugene, and Kepler Managing Synthetic Biology Device Development Lesia Bilitchenko, Joanna Chen, Adam Liu, Richard Mar, Thien Nguyen, Nina Revko, Bing Xia Berkeley Software, iGEM 2009, MIT, November 1, 2009 Project Motivation

199 views • 18 slides
Hardware and Device Drivers Device virtualization Device drivers and security Bjrn

Hardware and Device Drivers Device virtualization Device drivers and security Bjrn

Outline Faculty of Computer Science Institute for System Architecture, Operating Systems Group What's so different about device drivers? Hardware access Writing device drivers on L4 Reuse of legacy drivers Hardware and Device

619 views • 12 slides
Employment Issues For Troubled Companies and Companies In Transition DOUG DEXTER GARY KAPLAN

Employment Issues For Troubled Companies and Companies In Transition DOUG DEXTER GARY KAPLAN

OCTOBER 2010 Employment Issues For Troubled Companies and Companies In Transition DOUG DEXTER GARY KAPLAN Program Outline Managing Down-Sizing And Wind-Down Issues WARN Act Issues And Traps For The Unwary Managing Employment

377 views • 21 slides

More recommend