Smart Testing with AI Using Data Mining Presented by: - PDF document

¡ ¡ T16 ¡ Analytics ¡in ¡Testing ¡ Thursday, ¡October ¡3rd, ¡2019 ¡1:30 ¡PM ¡ ¡ ¡ ¡ ¡ Smart ¡Testing ¡with ¡AI ¡Using ¡Data ¡ Mining ¡ ¡ Presented ¡by: ¡ ¡ ¡ ¡ Lorna ¡Smyth ¡ ¡ Smartbear ¡ ¡ Brought ¡to ¡you ¡by: ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ 888 -­‑-­‑-­‑ 268 -­‑-­‑-­‑ 8770 ¡ ·√·√ ¡904 -­‑-­‑-­‑ 278 -­‑-­‑-­‑ 0524 ¡-­‑ ¡info@techwell.com ¡-­‑ ¡http://www.starwest.techwell.com/ ¡ ¡ ¡ ¡ ¡ ¡ ¡

¡ Lorna ¡Smyth ¡ ¡ Lorna ¡Smyth ¡is ¡from ¡Galway ¡on ¡the ¡west ¡coast ¡of ¡Ireland. ¡She ¡is ¡currently ¡a ¡technical ¡ sales ¡engineer ¡for ¡SmartBear's ¡test ¡automation ¡tools. ¡She ¡has ¡worked ¡in ¡various ¡IT ¡ roles ¡over ¡the ¡past ¡eight ¡years, ¡including ¡technical ¡analyst, ¡data ¡migration ¡test ¡lead, ¡ and ¡software ¡licensing ¡specialist. ¡She ¡specializes ¡in ¡test ¡automation ¡and ¡delivers ¡ webinars, ¡technical ¡demonstrations, ¡and ¡deep-­‑dive ¡troubleshooting ¡calls ¡on ¡test ¡ automation ¡for ¡SmartBear’s ¡customers. ¡Lorna ¡is ¡an ¡advocate ¡for ¡women ¡in ¡IT ¡and ¡is ¡ heavily ¡involved ¡in ¡hosting ¡local ¡networking ¡events. ¡She ¡also ¡participates ¡in ¡ community ¡outreach ¡to ¡drive ¡awareness ¡of ¡IT ¡roles ¡in ¡the ¡West ¡of ¡Ireland. ¡Lorna ¡is ¡ an ¡adventure ¡and ¡sports ¡enthusiast, ¡which ¡has ¡led ¡her ¡to ¡living ¡in ¡Canada, ¡Norway, ¡ and ¡Austria, ¡following ¡her ¡love ¡of ¡the ¡mountains. ¡Lorna ¡has ¡recently ¡returned ¡to ¡ Ireland ¡and ¡is ¡loving ¡life ¡working ¡in ¡test ¡automation. ¡ ¡

8/19/2019 Lorna Smyth Solutions Engineer for Test Automation tools 1

8/19/2019 ✓ What is Artificial Intelligence(AI)? ✓ How is AI transforming the world? ✓ What is Data Mining? ✓ How can AI address Software Testing Challenges? ✓ Benefits and Challenges of AI in Testing ✓ How can you Embrace AI? What is AI? “Your biggest fear?” “Something that is going to change the world…” “ Intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans.” “Computers that mimic cognitive functions that humans associate with the human mind, such as learning and problem solving” “Emphasizes the creation of intelligent machines that work and react like humans” “a conglomeration of concepts and technologies that mean different things to different people” 2

8/19/2019 What is AI? AI is the broad category of methodologies that teach a computer to perform tasks as an “intelligent” person would. Neural networks Machine learning Deep learning the networks of hardware and a technique using algorithms helps machines learn to go software that approximate to teach machines to learn. deeper into data to the web of neurons in the recognize patterns. human brain. Data Mining How is AI transforming the world? 3

8/19/2019 Impact of AI • AI will impact global GDP by $15.7 trillion by 2030. (Source: PwC) • 72% of Business leaders termed AI as a business advantage. (Source :PwC) • By 2020, it’s predicted robots will be able to flirt and make jokes. (Source: Google) • 77% of devices we use feature one form of AI or another. (Source: PEGA) • Machine learning engineers usually earn up to $115k per annum. • Automated bots could automate up to 38% of the jobs in the US, 30% of the jobs in the UK, 21% in Japan, and 35% in Germany. (Source: PwC) AI is transforming all industries, not just software testing… 4

8/19/2019 …personalizing content, making our lives easier through email responses.. …chat bots and targeted ads…. 5

8/19/2019 Ever wondered how the Wimbledon editorial team focus their content? IBM Watson’s AI 6

8/19/2019 HEALTHCARE Virtual Administrative Nursing Workflow Assistants Robot Fraud Assisted Detection Surgery Preliminary Dosage Error Diagnosis Reduction Image Based Connected Diagnosis Machines Clinical Trial Patient Engagement Participation Bumrungrad International Manipal Comprehensive Gachon University, Gil Hospital, Thailand: Hospital, India: Medical Center, South Korea: 83% concordance 73% concordance 49% concordance 49% 83% 73% 211 patients with breast, 638 patients with breast 656 patients with colon colorectal, gastric and cancer cancer lung cancer 7

8/19/2019 FINTECH • Digital Financial Coach • Client Risk Profile – automate categorization of clients High to Low; recommend relevant financial products. • Fraud Detection – using algorithms to identify fraudlent claims. • Contract Analyser – Contract analysis is a repetitive internal task which can be delegated to a machine. • JP Morgan saved 360,000 hours over a year from its employees’ load in only a few seconds. Impact of AI in Software Testing? • Test Case Design • Test Management • Test Execution • Test Result Analysis 8

8/19/2019 What is Data Mining? Image source: AIAnalyticsHub 9

8/19/2019 Data Mining Techniques: Sequential Classification Regression Prediction Patterns Association Clustering Outer Rules Data Mining Techniques: Classification: Retrieve information and classify into different classes. • E.g. Analyse loan applicant data, class as ‘risky’ or ‘safe’. • E.g. Analyse test data/ conditions, If ‘test’ run on Windows X, then class as ‘likely to fail’. Clustering: Identify data that is like each other. • E.g. Segregate applicants with similar incomes. • E.g. Segregate defects with similar causes of failure (e.g. Performance, Functional, System). Regression: Identify and analyze relationship between variables. • E.g. Credit Policy, Loan Purpose, Income, Credit Score, Spend History. • E.g. Operating System, Browser, Environment, Resolution. 10

8/19/2019 Data Mining Techniques: Association Rules: Identify associations between two or more items – discovers hidden patterns.. • E.g. A single, female aged between 35-45 with account of over $5000 is likely to need a loan between $10,000-$15,000. • E.g. Performance tests that are ran on Mac OS X are 20% likely to fail Outer Detection: Identify items that do not match an expected pattern/behavior. • E.g. Unexpected Spend/Transaction on a bank statement. • E.g. Defect/Error types that are unexpected or haven’t been found before. Data Mining Techniques: Sequential Patterns: Identify patterns in transaction data for a certain time period. • E.g. Identifying a pattern of spending at certain times of the year. • E.g. Identifying patterns of performance issues during holiday season. Prediction: combination of data mining techniques, analyzes past events or instances in a sequence for predicting a future event. • E.g. Predict an individuals overall creditworthiness. • E.g. Predictive defect finding based on past defects found. 11

8/19/2019 Data Mining Process Understand Understand Prepare & Business Process Data Data Problem Evaluate, Verify & Model Data Train & Test Deploy Model What do you mean by modelling the data? 12

8/19/2019 Modeling Data: Neural Network example Let’s train an AI…. 13

8/19/2019 AI and Software Testing We Want Faster Release Cycles Continuous Autonomous Manual Bulky Robust More time for Testing Testing Testing Automation Automation Business Goals Weeks Tools Tools Speed SDLC Time Taken Quality Waterfall Early Cost Agile Agile Culture Devops CI/CD Innovation Smart Hours Low Automation Capability High 14

8/19/2019 Why now? • Many useful machine learning systems are relatively easy to build. • The availability of, and ability to store and process large amounts of data, make training machine learning systems easier and more accurate. • Software testing industry can now see the potential of adopting AI in testing. ➢ Testing is a good use case as it’s built on a logical foundation ➢ Companies are struggling to keep up with user expectations. Current Automated UI testing challenges TEST MAINTENANCE AND TEST DESIGN EXECUTION • Unreliable Object Recognition • Test refactoring • Test Framework Design • Test scalability • Inadequate Prioritization • Inadequate Documentation 15

8/19/2019 INTELLIGENT TEST DESIGN Advanced Framework Object Generation Recognition Risk Responsive Web Design Profiling What does Intelligent Test Design Look Like? Object Recognition Automatically recognize new objects and updates to add them to the DOM and structure without manual effort Framework Generation Automatically scan your application to recommend a test framework Risk Profiling Prioritize tests based on business risk by environment, configuration, and different aspects of your application 16