1 Refactoring Code for Overview of My Research Asynchronous - - PDF document

1

INTERACTIVE PROGRAM TRANSFORMATIONS Danny Dig

What do you see yourself doing after you earn your Bachelor’s?

• A. Get a job as a programmer/tester etc., on the tech track
• B. Get a job with the government
• C. Go to grad school (M.S. or Ph.D.)
• D. Start my own company
• E. Other

2

What can I do today so that I get my dream SE job next year?

Create a folder/CV with your CS361 work (professional documentation, process, TDD, Design Patterns) Write lots of code (read even more code) Contribute to OSS

• your CV is your GitHub profile

3

Software Evolution as Done Today

Change is the heart of software development:

• add features, fix bugs, support new hardware/UI/OS

Today’s program development is very crude

• change carried manually, through low-level edits
• changes are almost never reused
• versioning tools focus on changes to lines of code

Change is too ad-hoc making software development error- prone, time-consuming, and $$$

• 2/3rd of software costs due to software evolution, some

industrial surveys claiming 90%

4

My View of Tomorrow: Programming is Program Transformation

Change needs to move to a higher-level of abstraction Program transformations as first-class:

• most changes carried through automated program

transformations

• even manual edits become transformations
• programs as sequence of program transformations

Q1: Analyze what software changes occur in practice? Q2: How can we automate them? Q3: Can we represent programs as transformations? Archive, retrieve, and visualize them? Q4: Can we infer higher-level transformations?

5

Interactive Source-to-Source Program Transformations

Programmer + Tool >> Tool | Programmer

search, remember, compute domain knowledge

6

2

Interactive Analysis & Transformations

Refactor end-user software Retrofit Parallelism Introduce Asynchronous Code Upgrade library-based programs

Overview of My Research

Generate tests

TOSEM15,STVR15, ECOOP14,ISSTA13, FSE13,ICST13,FSE12, ICSE11,SOFTW11, OOPSLA09, ICSE09, ASE09a FSE14d,ICSM12 ICST13, TSE10, ASE09b, ISSTA08, FSE07 ECOOP13,ECOOP12, TSE08, ICSE08, ICSE07,ECOOP06, JSME06,ICSM05 ICSE14, FSE14

7

Refactoring Code for Asynchronous Execution on Mobile

By 2016, 300B apps downloaded annually Running I/O blocking operations synchronously freezes the UI and frustrates users My research enables retrofitting asynchronous execution via interactive refactoring (complex analysis and transformation)

• inverting flow of control, callbacks
• novel analyses determine non-interference with main thread

Generated hundreds of patches to correctly introduce asynchrony, accepted by OSS developers

8

Our Refactorings for Parallelism

Refactorings for thread-safety

• make class immutable [ICSE'11]
• convert to Atomic* classes [ICSE'09]
• use concurrent collections [ICSE'09]
• infer region annotations [ASE'09]
• atomic check-then-act operations [ICST’13]

Refactorings for throughput

• parallel recursive divide-and-conquer [ICSE'09]
• loop parallelism via ParallelArray [OOPSLA’10]
• loop parallelism via lambda-enabled functional operators

[FSE’13]

Refactorings for scalability

• Atomic*, concurrent collections [ICSE'09]

9

Interprocedural analyses:

• control, data-flow
• points-to
• constraint-based

10

x, y -> x + y

λ expressions

Hallmark of functional languages Some languages had λ from first version (Smalltalk, Python, JS) λ retrofitted in OO languages (C++, C#, Java)

• unobtrusive parallelism
• collections

From Imperative to Functional Programming through Refactoring [Gyori, Franklin, Dig, Lahoda: FSE’13]

11

• 1. Anonymous Inner Class to lambda
• 2. for loop to functional operations

Cleaner code Less clutter

How can I λ-ify my Java code?

LambdaFicator refactorings: unobtrusive parallelism explicit semantics

Advantages of iterating with functional

• perations

1. private void redify() { 2. int n = 4;// amount of parallelism 3. Thread[] threads = new Thread[n]; 4. final List<ElementRule> rules = 5. properties.getRules(); 6. int size = rules.size(); 7. for ( int i = 0; i < n; i++) { 8. final int from = i * size / n; 9. final int to = (i + 1) * size / n;

• 10. threads[i] = new Thread(new

Runnable() {

• 11. @Override
• 12. public void run() {
• 13. for ( Block b :

blocks)

• 14. if(b.getColor() ==

BLUE)

• 15. b.setColor(RED);
• 16. }
• 17. });
• 18. threads[i].start();
• 19. }
• 20. for ( int i = 0; i < n; i++) {
• 21. try {
• 22. threads[i].join();
• 23. } catch

( InterruptedException ex) {

private void redify() { for ( Block b : blocks) if(b.getColor() == BLUE) b.setColor(RED); } private void redify() { blocks.parallelStream() .filter(b -> b.getColor() == BLUE) .forEach(b -> { b.setColor(RED); }); }

External Iteration -> Internal Iteration

DEMO

3

13

Refactorings are widely applicable

• 46% of for loops successfully refactored

Refactorings improve code quality:

• first refactoring removes boilerplate code (52% AST node reduction)
• second refactoring makes loop semantics explicit

LambdaFicator saves programmer effort

• first refactoring changed avg. of 412 SLOC/project in 11 seconds
• second refactoring changed avg. of 1368 SLOC/project in 11 seconds

Empirical evaluation

We ran LambdaFicator on 10 projects, ~1Million SLOC

Refactoring meets Spreadsheets [Badame, Dig – ICSM’12]

Spreadsheets have been around for 4,000 years

• Plimpton 322, Babylonian clay tablet showing the Pythagorean

triplets, 1800BC

• contains a copy-paste error: number in row 9 is copied from 8

The number of spreadsheet end-users is at least 100M => 10x more errors than in professional programs

14

Spreadsheets that are changing are prone to errors

We studied a corpus of 3,700 real world spreadsheets Spreadsheets are riddled with problems:

• decreasing maintainability (e.g., duplicated expressions)
• decreasing performance (e.g., redundant computation)

End-users have to maintain spreadsheets

• life-span: 5 years, end-users/sheet: 13 [Hermans – ICSE’11]

Smells mask errors that cost [Burnett – CACM’04]

• TransAlta lost $24M in 2003 due to cut-and-paste error in Excel
• The Fed made a copy-paste error in their consumer credit statement,

could have led to $4B loss

15

Our Refactorings

• 1. Replace Awkward Formula
• 2. Extract Literal
• 3. Guard Call
• 4. String to Dropdown
• 5. Introduce Cell Name
• 6. Make Cell Constant
• 7. Extract Column

Implemented in our tool, REFBOOK, world’s first interactive refactoring tool for Microsoft Excel

16 17

Extract Column Refactoring Extract Column Refactoring needs to change elements “similar” to the one selected

Need to ensure that all expressions in the column have the “same” formula, despite not being exact matches Precondition:

• formulas in the column of

selected expression are consistent Two formulas are consistent if their ASTs are isomorphic Refactoring steps for each row:

• compute the corresponding expression (e.g., B2+C2+D2+E2)
• insert corresponding expression in the new column
• replace references to expression with the name of new cell

18

4

3-pronged Evaluation shows REFBOOK is useful

• 1. Survey
• 2. Controlled experiment
• 3. Case study

RQ1: Do refactorings improve spreadsheet quality? Yes, for 4 refactorings users preferred refactored sheets RQ2: Can REFBOOK make the process reliable? Yes, REFBOOK safe, manual refactorings had errors RQ3: Do automated refactorings improve human productivity? Yes, 2x-20x productivity improvement RQ4: Are the refactorings applicable? Yes, 32% of formulas are candidates for ExtractColumn

19

Practical Impact of My Research

Several of our tools ship with official release of Eclipse alone had 13M downloads in 2014 Influenced official concurrency libraries of Java and .NET Our tools used in industry Hundreds of accepted OSS patches http://learnparallelism.net (120,000 visitors)

20

Collaborators

• Alexandria Shearer (ugrad)
• Anda Bereckzy (ugrad)
• Lyle Franklin (ugrad)
• Alex Gyori (ugrad)
• Alex Sikora (ugrad)
• John Marrero (ugrad)
• Mihai Tarce (ugrad)
• Jack Ma (ugrad)
• Lorand Szakacs (ugrad)
• Sandro Badame (MS)
• Fredrik Kjolstadt (MS)
• Kely Garcia (MS)
• Kashif Manzoor (MS)
• Can Comertoglu (MS)
• Caius Brindescu (PhD)
• Mihai Codoban (PhD)
• Michael Hilton (PhD)
• Sergey Shmarkatiuk (PhD)
• Brett Daniel (PhD)
• Stas Negara (PhD)
• Cosmin Radoi (PhD)
• Semih Okur (PhD)
• Yu Lin (PhD)
• Rob Bochinno (PhD)
• Mohsen Vakillian (PhD)
• Stephen Heumann (PhD)
• Jeff Overbey (PhD)
• Ralph Johnson
• Darko Marinov
• Marc Snir
• Brian Bailey
• Vikram Adve
• Sarita Adve
• Frank Tip (U of Waterloo)
• Julian Dolby (TJ Watson)
• Danny Soroker (TJ Watson)
• Ramón Cáceres (TJ Watson)
• Andreas Schade (TJ Watson)
• Susan Spraragen (TJ Watson)
• Alpana Tiwari (TJ Watson)
• Tien Nguyen (IA State)
• Tao Xie (NCSU)
• Kunal Taneja (NCSU)
• John Marrero (MIT)
• Michael Ernst (U Washington)
• Shay Artzi (MIT)
• Adam Kiezun (MIT)
• Marius Minea (UPT)
• Iulian Neamtiu (UC Riverside)
• Bill Griswold (UCSD)
• Vibhu Mohindra (ACL Wireless)
• Jan Lahoda (Oracle)

21

Future Work: Change-Oriented Programming Environment

Enable a software revolution similar with the industrial revolution Semi-automated transformations at center of software development

• View programs as a composition of transformations
• Enable programmers to write, script, modify, and replay

transformations

22