Continuous integration in a social-coding world Bogdan Vasilescu, Stef van Schuylenburg, Jules Wulms, Alexander Serebrenik, Mark G. J. van den Brand @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Integration “[used to be] a long and unpredictable process” (Martin Fowler, 2000) Development stops Integration starts @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Continuous Integration speeds up collaborative software development by reducing integration problems integration with every commit @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Continuous Integration speeds up collaborative software development by reducing integration problems integration with every commit Report Initiate CI process T est Tests Build @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
“There are 7.1M people collaborating right now across 15.9M repositories on GitHub. Developers from all around the world are building amazing things together. Their story is our story.” https://github.com/about/press How are GitHub developers using CI? @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
How are GitHub developers using CI? 80 60 count 40 20 0 Java Python Ruby Programming.language (1) Most projects are configured to use CI, but less than half actually do. not configured configured but not using configured and using @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Contributing to IronCat’s repository Jean-Luc Picat Octocat Spocktocat IronCat <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Jean-Luc Collaborators Picat Octocat Spocktocat <push> (commit) IronCat <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Jean-Luc Picat Octocat I want to add a new feature / fix an issue but I don’ t have commit rights Spocktocat IronCat <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Jean-Luc Picat Octocat Spocktocat <created> Repository <forked> IronCat <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Jean-Luc Picat Octocat I can implement the change in my own fork. But how to signal IronCat? Spocktocat <push> (commit) <created> Repository <forked> IronCat <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Jean-Luc Picat Octocat Can you please pull my change into your repository> Spocktocat <push> <pull request> (commit) <created> Repository <forked> IronCat <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Jean-Luc Picat Octocat Spocktocat <push> <pull request> (commit) <created> Repository <forked> IronCat Nice work! I’ll merge your change <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Jean-Luc Picat Octocat Spocktocat <member of> <push> <pull request> (commit) <created> Repository <forked> IronCat Nice work! <merge commit> I’ll merge your change <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub workflow Jean-Luc Picat Octocat <push> <push> <pull request> Spocktocat Direct contributions (Collaborators) Indirect contributions IronCat (Pull requests) <push> <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
unprecedented low barrier to entry for potential contributors L. A. Dabbish, H. C. Stuart, J. Tsay, and J. D. Herbsleb, “ Social coding in GitHub: transparency and collaboration in an open software repository ,” in CSCW 2012 scalability challenges when integrating automated tests (many) outside contributions R. Pham, L. Singer, O. Liskin, K. Schneider et al., “ Creating a shared understanding of testing culture on a social coding site ,” in ICSE 2013 @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
How are GitHub developers using CI? Distribution of p − values from the Chi^2 tests 80 60 count Odds Ratios 40 0.01 0.05 0.1 0.5 1 20 0 Java Python Ruby 0.2 0.5 1.0 2.0 5.0 10.0 20.0 Programming.language (1) Most projects are configured to (2) Pull requests are much more likely to result use CI, but less than half actually do. in successful builds than direct commits not configured configured but not using configured and using @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GHTorrent sample G. Gousios, “ The GHTorent dataset and tool suite ,” in MSR 2013 G. Gousios, B. Vasilescu, A. Serebrenik, A. Zaidman, “ Lean GHTorrent: GitHub data on demand ,” in MSR 2014 223 large and active repositories * not forks * ≥ 1 year * both commits and pull requests * Java, Python, or Ruby * ≥ 10 changes last month * ≥ 10 contributors @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub + Travis-CI Jean-Luc Picat Octocat <push> <push> <pull request> Spocktocat Direct contributions (Collaborators) Indirect contributions IronCat (Pull requests) <push> <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
GitHub + Travis-CI Jean-Luc Picat Octocat <push> <push> <pull request> Spocktocat IronCat <push> <created> Repository @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Methods and Data @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Methods and Data No Stop ? .travis.yml Y es #passed #failed #passed #failed rails/rails #passed #failed rails/rails builds builds #passed #failed rails/rails builds builds #passed #failed rails/rails builds builds rails/rails builds builds commits 5767 3715 builds builds commits 5767 3715 commits 5767 3715 commits 5767 3715 pull reqs 2450 1337 commits 5767 3715 pull reqs 2450 1337 pull reqs 2450 1337 pull reqs 2450 1337 pull reqs 2450 1337 @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Methods and Data No Stop ? .travis.yml Y es #passed #failed #passed #failed rails/rails #passed #failed rails/rails builds builds #passed #failed rails/rails builds builds #passed #failed rails/rails builds builds rails/rails builds builds commits 5767 3715 builds builds commits 5767 3715 commits 5767 3715 commits 5767 3715 pull reqs 2450 1337 commits 5767 3715 pull reqs 2450 1337 pull reqs 2450 1337 80 pull reqs 2450 1337 pull reqs 2450 1337 60 count 40 not configured (17 / 223) configured but not using � 20 (93 / 223) configured and using 0 (113 / 223) Java Python Ruby Programming.language @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Methods and Data No Stop ? .travis.yml Y es #passed #failed #passed #failed rails/rails #passed #failed rails/rails builds builds #passed #failed rails/rails builds builds #passed #failed rails/rails builds builds rails/rails builds builds commits 5767 3715 builds builds commits 5767 3715 commits 5767 3715 commits 5767 3715 pull reqs 2450 1337 commits 5767 3715 pull reqs 2450 1337 pull reqs 2450 1337 pull reqs 2450 1337 Distribution of p − values from the Chi^2 tests pull reqs 2450 1337 Does type matter for passed/failed? ² + Stouffer 0.01 0.05 0.1 0.5 1 @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Methods and Data No Stop ? .travis.yml Y es #passed #failed #passed #failed rails/rails #passed #failed rails/rails builds builds #passed #failed rails/rails builds builds #passed #failed rails/rails builds builds rails/rails builds builds commits 5767 3715 builds builds commits 5767 3715 commits 5767 3715 commits 5767 3715 pull reqs 2450 1337 commits 5767 3715 pull reqs 2450 1337 Odds Ratios pull reqs 2450 1337 pull reqs 2450 1337 pull reqs 2450 1337 Which type succeeds more often? 0.2 0.5 1.0 2.0 5.0 10.0 20.0 Odds ratios @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
How are GitHub developers using CI? Distribution of p − values from the Chi^2 tests 80 60 count Odds Ratios 40 0.01 0.05 0.1 0.5 1 20 0 Java Python Ruby 0.2 0.5 1.0 2.0 5.0 10.0 20.0 Programming.language (1) Most projects are configured to (2) Pull requests are much more likely to result use CI, but less than half actually do. in successful builds than direct commits @b_vasilescu @aserebrenik #icsme14 ICSME 2014, Victoria, BC, Canada
Recommend
More recommend
