how high will it be using machine learning models to
play

How High Will It Be? Using Machine Learning Models to Predict - PowerPoint PPT Presentation

Oct 26, 2022 •102 likes •333 views

How High Will It Be? Using Machine Learning Models to Predict Branch Coverage in Automated Testing G. Grano, T. Titov, S. Panichella, H. Gall MaLTeSQuE@SANER 2018, 20, Campobasso (Italy) grano@ifi.uzh.ch giograno90 U.S. Economy Impact 1

  1. How High Will It Be? Using Machine Learning Models to Predict Branch Coverage in Automated Testing G. Grano, T. Titov, S. Panichella, H. Gall MaLTeSQuE@SANER 2018, 20, Campobasso (Italy) grano@ifi.uzh.ch giograno90

  2. U.S. Economy Impact 1 $59.5 billion $22.2 billion avoidable 0.6% GDP 1 http://www.abeacha.com/NIST_press_release_bugs_cost.html 2 — Giovanni Grano @ s.e.a.l.

  3. Continuous Integration The way we develop and release code is rapidly changing In an ideal world, at every single commit the entire test suite should be executed ... ... but companies like Google commit about 16.000 changes per day! 3 — Giovanni Grano @ s.e.a.l.

  4. Test Data Generation Pretty active research area in the last years Mature tools able to generate test suites with high coverage: > EvoSuite > Randoop > ... > and many more! How do they fit in such a CI/CD environment? 4 — Giovanni Grano @ s.e.a.l.

  5. Continuous Testing Generation 2 Continuous integration enhanced with automated test generation It raises many questions: > testing order > how much time to spend per class 2 Campos et al - Continuous Test Generation: Enhancing Continuous Integration with Automated Test Generation 5 — Giovanni Grano @ s.e.a.l.

  6. Test Suite Augmentatio n 3 Automatic generation considering code changes and their effect on the previous codebase Hardy doable to the expensive amount of time needed to generate tests 3 Xu et al - Directed Test Suite Augmentation: Techniques and Tradeoffs 6 — Giovanni Grano @ s.e.a.l.

  7. Coverage Prediction Knowing a priori the coverage achieved by test data generation tools > maximize the coverage for the entire system given an amount of time > budget allocation for critical components 7 — Giovanni Grano @ s.e.a.l.

  8. Research Questions

  9. > RQ1: Which type of features can we leverage to train machine learning models to predict the branch coverage achieved by test data generation tools? > RQ2: To what extend can we predict the coverage achieved by test data generation tools? 9 — Giovanni Grano @ s.e.a.l.

  10. Project Selection Open Source Projects from Defect4j > Apache Cassandra > Apache Ivy > Google Guava > Google Dagger Guava Cassandr Dagger Ivy a LOC 78,525 220,573 848 50,430 Java 538 1,474 43 464 Files 10 — Giovanni Grano @ s.e.a.l.

  11. Training Set > Run EvoSuite > Labeled data (with coverage) > Computed metrics = input variables Coverage Threats: Metrics Extractor > we did it once 11 — Giovanni Grano @ s.e.a.l.

  12. Feature Selection Goal: capture complexity

  13. Package Level Features Computed with JDepend 4 Name Description Ca indicator of the package's responsibility Ce indicator of the package's independence A abstract classes / total number of classes I indicator of the package's resilience to change ... ... 4 https://github.com/clarkware/jdepend 13 — Giovanni Grano @ s.e.a.l.

  14. CK and OO Feature ck tool provided by Aniche 5 Name Description CBO coupling between objects DIT depth of inheritance tree NOC number of children NOSF number of static field ... ... 5 https://github.com/mauricioaniche/ck 14 — Giovanni Grano @ s.e.a.l.

  15. Java Reserved Keywords To capture additional complexity Previously used in Information Retrieval as a feature 6 52 Java reserved keywords 6 Sanderson et al The history of information retrieval research 15 — Giovanni Grano @ s.e.a.l.

  16. Grid Selection best hyper-parameters 3-cross fold validation Feature Transformation z-score 16 — Giovanni Grano @ s.e.a.l.

  17. Algorithms Huber Regression Support Vector Regression Multi-Layer Perception 17 — Giovanni Grano @ s.e.a.l.

  18. Results RQ1 Huber SVR MLP Tool's Average EvoSuit 0.255 0.216 0.242 0.238 e Randoop 0.172 0.088 0.139 0.131 Average 0.213 0.152 0.191 18 — Giovanni Grano @ s.e.a.l.

  19. Results RQ2 Time Math Lang Tool's Aver. EvoSuit 0.255 0.330 0.289 0.291 e Randoo 0.168 0.262 0.246 0.225 p Aver. 0.211 0.296 0.267 19 — Giovanni Grano @ s.e.a.l.

  20. Conclusion Knowing a priori the coverage achieved by test data generation tools might ease important decisions; > we took the first steps, investigating well known features > well know features (from gut feeling) give reasonable results 20 — Giovanni Grano @ s.e.a.l.

  21. Future work Improvements > larger dataset > branch-level features > feature analysis > differences between employed tools 21 — Giovanni Grano @ s.e.a.l.

Recommend

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead Cynthia Rudin, Duke University Presenters : Sreya Dutta Roy, Ziqian Lin 1 Overview Introduction Explainable ML Vs

1.04k views • 39 slides
Deploying Machine Learning Models on The Edge Deploying Machine Learning Models on The Edge Yan

Deploying Machine Learning Models on The Edge Deploying Machine Learning Models on The Edge Yan

Deploying Machine Learning Models on The Edge Deploying Machine Learning Models on The Edge Yan Zhang, Mathew Salvaris Microsoft https://github.com/microsoft/deploy-MLmodels-on-iotedge Cloud Analytics Edge Analytics Device/Sensor Analytics

373 views • 34 slides
The role of models@run.time in self-explanation in the era of Machine Learning Antonio Garcia,

The role of models@run.time in self-explanation in the era of Machine Learning Antonio Garcia,

The role of models@run.time in self-explanation in the era of Machine Learning Antonio Garcia, Juan Marcelo Parra-Ullauri and Nelly Bencomo 14th International Workshop on Models@run.time September 17th, 2019 Introduction Machine learning is

616 views • 34 slides
Good Machine Learning = Generalization Goal of machine learning: build models that generalize

Good Machine Learning = Generalization Goal of machine learning: build models that generalize

Good Machine Learning = Generalization Goal of machine learning: build models that generalize well to predicting new data Overfitting: fitting the training data too well, so we lose generality of model o Example: linear regression vs.

120 views • 11 slides
Auditing Machine Learning Models for Individual Bias and Unfairness Songkai Xue Department of

Auditing Machine Learning Models for Individual Bias and Unfairness Songkai Xue Department of

Auditing Machine Learning Models for Individual Bias and Unfairness Songkai Xue Department of Statistics, University of Michigan Joint work with Mikhail Yurochkin and Yuekai Sun Introduction High-stakes decision making involves Recidivism

478 views • 30 slides
Moven Machine/Deep Learning Models Distribution Relying on the Maven Infrastructure Sergio

Moven Machine/Deep Learning Models Distribution Relying on the Maven Infrastructure Sergio

Moven Machine/Deep Learning Models Distribution Relying on the Maven Infrastructure Sergio Fernndez (Redlink GmbH) November 14th, 2016 - Sevilla SSIX aims to exploit the predictive power of Social Media on Financial Markets High-Level

724 views • 13 slides
Efficient Algorithms for Learning Sparse Models from High Dimensional Data Yoram Singer Google

Efficient Algorithms for Learning Sparse Models from High Dimensional Data Yoram Singer Google

Efficient Algorithms for Learning Sparse Models from High Dimensional Data Yoram Singer Google Research Machine Learning Summer School, UC Santa Cruz, July 18, 2012 1 Sparsity ? THE HIGHER MINIMUM WAGE THAT WAS SIGNED INTO LAW ... WILL BE

855 views • 74 slides
CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map

CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map

CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map Machine Learning Approaches Supervised Semi-supervised Unsupervised Reinforcement Learning Learning Learning Learning Classification Regression

1.06k views • 57 slides
Learning in Graphical Models Andrea Passerini passerini@disi.unitn.it Machine Learning Learning

Learning in Graphical Models Andrea Passerini passerini@disi.unitn.it Machine Learning Learning

Learning in Graphical Models Andrea Passerini passerini@disi.unitn.it Machine Learning Learning in Graphical Models Learning graphical models Parameter estimation We assume the structure of the model is given We are given a dataset of

268 views • 23 slides
Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning:

Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning:

Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning: instances, concepts, and hypotheses Specific learners Learning Hypothesis/ Labeled algorithm Model h Decision trees data New example

519 views • 41 slides
Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University April 5, 2011 Today: Readings: Kernels: Bishop Ch. 6.1 Latent Dirichlet Allocation topic models optional: Social network analysis

298 views • 13 slides
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 18, 2015 Today: Readings: Bishop chapter 8, through 8.2 Graphical models Bayes Nets: Representing distributions

852 views • 31 slides
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University October 13, 2011 Today: Readings: Required: Graphical models Bishop chapter 8, through 8.2 Bayes Nets: Conditional

149 views • 10 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Dhruv Batra Virginia Tech What is this class about? Some of the most exciting developments in Machine Learning, AI, Statistics &

925 views • 61 slides
Logistic regression on Sonar Machine Learning Toolbox Classification models Categorical

Logistic regression on Sonar Machine Learning Toolbox Classification models Categorical

MACHINE LEARNING TOOLBOX Logistic regression on Sonar Machine Learning Toolbox Classification models Categorical (i.e. qualitative) target variable Example: will a loan default? Still a form of supervised learning Use a

315 views • 27 slides
Stealing Machine Learning Models via Prediction APIs Florian Tramr, Fan Zhang, Ari Juels,

Stealing Machine Learning Models via Prediction APIs Florian Tramr, Fan Zhang, Ari Juels,

Stealing Machine Learning Models via Prediction APIs Florian Tramr, Fan Zhang, Ari Juels, Michael K. Reiter, Thomas Ristenpart Usenix Security Symposium Austin, Texas, USA August, 11 th 2016 Machine Learning (ML) Systems (1) Gather labeled

213 views • 17 slides
Interval-valued regression and classication models in the framework of machine learning Lev

Interval-valued regression and classication models in the framework of machine learning Lev

A statement of the standard machine learning problem A new approach for regression and classication modeling Regression models Classication models Interval-valued regression and classication models in the framework of machine learning

214 views • 18 slides
Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 22, 2011 Today: Readings: Recommended: Clustering Jordan Graphical Models Mixture model clustering Muphy Intro

655 views • 10 slides
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 25, 2015 Today: Readings: Bishop chapter 8 Graphical models Mitchell chapter 6 Bayes Nets: Inference Learning

639 views • 34 slides
Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Mixture Models: definition Mixture models: Linear supper-position of mixtures or components | =

1.04k views • 39 slides
TensorFlow: A System for Learning-Scale Machine Learning Google Brain The Problem Machine

TensorFlow: A System for Learning-Scale Machine Learning Google Brain The Problem Machine

TensorFlow: A System for Learning-Scale Machine Learning Google Brain The Problem Machine learning is everywhere This is in large part due to: 1. Invention of more sophisticated machine learning models 2. Availability of large

330 views • 22 slides
High Performance Machine Learning: Advances, Challenges and Opportunities Eduardo Rodrigues

High Performance Machine Learning: Advances, Challenges and Opportunities Eduardo Rodrigues

High Performance Machine Learning: Advances, Challenges and Opportunities Eduardo Rodrigues Lecture @ ERAD-RS - April 11th, 2019 IBM Research Artificial Intelligence Deep Blue (1997) AI and Machine Learning AI ML

647 views • 63 slides
Coresets for Data-efficient Training of Machine Learning Models Baharan Mirzasoleiman*, Jeff

Coresets for Data-efficient Training of Machine Learning Models Baharan Mirzasoleiman*, Jeff

Coresets for Data-efficient Training of Machine Learning Models Baharan Mirzasoleiman*, Jeff Bilmes**, Jure Leskovec* * * ** Machine Learning Becomes Mainstream Personalized medicine Robotics Finance Autonomous cars 2 Data is the Fuel

704 views • 22 slides
An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning http://www.cs.iastate.edu/~cs573x/bbsilab.html Machine Learning Software Preparing Data Building Classifiers Interpreting Results Machine Learning Software

496 views • 26 slides

More recommend