Bugs, au Naturale. % 0 % 0 0 1 0 l 1 a r l a u r t a - PowerPoint PPT Presentation

Bugs, au Naturale. % 0 % 0 0 1 0 l 1 a r l a u r t a u t N a N Premkumar Devanbu DECAL Laboratory public class FunctionCall { public static void funct1 () { System.out.println ("Inside funct1"); } University of California, Davis public static void main (String[] args) { % int val; 0 0 System.out.println ("Inside main"); 1 funct1(); l a System.out.println ("About to call funct2"); r u val = funct2(8); t a System.out.println ("funct2 returned a value of " + val); N System.out.println ("About to call funct2 again"); val = funct2(-3); System.out.println ("funct2 returned a value of " + val); } public static int funct2 (int param) { System.out.println ("Inside funct2 with param " + param); return param * 2; } } ICSE 2012, “On the Naturalness of Software” Tiger, Tiger, Hmmmm…. Burning 
 Bright…

Meanwhile, back in Redmond.. TIGER!! 
 RUN!!! (or Bangalore, or Shangahi, or Sunnyvale..) Why is your feature “Let us change our traditional attitude to the construction of behind Schedule? programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do... Code, Code, Code, Code, Code… On the Uniqueness of Code (FSE 2010)

First, Some Differences

Cross-Entropy per token The Skeptic asks.. Brown Java 10 Is it just that C, Java, Python... are syntactically 7.5 simpler than English? 5 2.5 0 1-gram 2-gram 3-gram 4-gram 5-gram

Is buggy code odd? Buggy vs. Not. Title 0.7 15 Line Entropy 0.61 Effect Size (Cohen’s D) 0.5 0.53 10 0.43 0.4 0.36 5 0.29 0.25 0.2 0.18 0.18 0 non_buggy buggy fixed 0.0 0 17 33 50 67 83 100 Defective Line Count

Defective line count Title 90 15 Line Entropy 81 72 10 Percent of total bugs 67.5 60 56 5 45 44 35 0 low medium high 22.5 24 duration duration duration 15 0 0 17 33 50 67 83 100 Defective Line Count Does it work? Findings Summary How to tell? • Problem: Line-level! • More cost-effective than logistic regression at 5% inspection budget, but not at 20%. • Cost-sensitive measures more suitable. • Cost-effectiveness similar to FindBugs and • Comparable to static analysis warnings! PMD. • Entropy-based ordering improves cost- … measured using Cost-effectiveness curve. effectiveness of PMD & FindBugs