Large-Scale API Protocol Mining for Automated Bug Detection Michael - PowerPoint PPT Presentation

State Partitioning Protocol transformation: Setup states vs. liable states l.add i.hasNext l.iterator new LinkedList → i i.hasNext → l l.add i.next ambiguous → split l.add 29

State Partitioning Protocol transformation: Setup states vs. liable states l.add i.hasNext l.iterator new LinkedList → i i.hasNext → l i.next l.add l.add l.add 29

State Partitioning Protocol transformation: Setup states vs. liable states l.add i.hasNext l.iterator new LinkedList → i i.hasNext → l i.next l.add l.add setup liable l.add 29

Overview Analysis Static Runtime checking verification 30

Overview Analysis Static Runtime checking verification 30

Dynamic Protocol Checking Runtime verification Input (JavaMOP) 31

Dynamic Protocol Checking Runtime verification Input (JavaMOP) Challenge 1: Check many different execution paths 31

Dynamic Protocol Checking Runtime verification Input (JavaMOP) Challenge 1: Challenge 2: Check many different Monitoring execution paths mined protocols 31

Randomly Generated Input Challenge 1: Check many different execution paths Input 32

Randomly Generated Input Challenge 1: Check many different execution paths Input Random test generation 32

Randomly Generated Input Challenge 1: Check many different execution paths Input Random test Call sequences generation that trigger an exception 32

Randomly Generated Input Challenge 1: Check many different execution paths Input Non-exceptional Random test Call sequences sequences generation that trigger an exception 32

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () i.hasNext new LinkedList → l i.iterator → i i.next 33

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () i.hasNext new LinkedList → l i.iterator → i l i.next 33

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () i.hasNext new LinkedList → l i.iterator → i l i.next 33

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () Violation!? i.hasNext new LinkedList → l i.iterator → i l i.next 33

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () i1 = l.iterator () i2.next() i.hasNext new LinkedList → l i.iterator → i l i.next 33

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () i1 = l.iterator () i2.next() i.hasNext new LinkedList → l i.iterator → i (l,i1) i.next 33

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () i1 = l.iterator () i2.next() i.hasNext new LinkedList → l i.iterator → i (l,i1) i2 i.next 33

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () i1 = l.iterator () Violation!? i2.next() i.hasNext new LinkedList → l i.iterator → i (l,i1) i2 i.next 33

Protocol Monitoring Challenge 2: Monitoring mined protocols l = new LinkedList () i1 = l.iterator () Naive approach gives Violation!? i2.next() too many violations i.hasNext new LinkedList → l i.iterator → i (l,i1) i2 i.next 33

Explicit Fail Transitions Fail only in liable states i.hasNext new LinkedList → l i.iterator → i i.next i.next i.hasNext F 34

Explicit Fail Transitions Fail only in liable states i.hasNext new LinkedList → l i.iterator → i i.next i.next i.hasNext Violation: F � Reach fail state � End in non-final, liable state 34

Evaluation Questions � Find relevant issues by monitoring mined protocols? � How useful is generated input? Setup: DaCapo benchmarks, 1.6 MLOC Java 35

Results Randomly generated Protocol violations Bug (exception, Program Test cases Total Relevant unexpected behavior) avrora 15,753 5 4 or batik 3,477 0 0 code smell (perfor- daytrader 32,446 0 0 eclipse 816 0 0 mance/maintainability fop 6,536 52 50 problem) h2 7,584 14 7 lucene 1,985 0 0 pmd 1,286 0 0 sunflow 4,300 1 0 tomcat 14,627 1 1 xalan 21,083 1 1 Sum 160,857 74 63 36

Results Randomly generated Protocol violations Bug (exception, Program Test cases Total Relevant unexpected behavior) avrora 15,753 5 4 or batik 3,477 0 0 code smell (perfor- daytrader 32,446 0 0 eclipse 816 0 0 mance/maintainability fop 6,536 52 50 problem) h2 7,584 14 7 lucene 1,985 0 0 pmd 1,286 0 0 sunflow 4,300 1 0 tomcat 14,627 1 1 85% true xalan 21,083 1 1 Sum 160,857 74 63 positives 36

Examples try { is = u.openStream (); r = new InputStreamReader(is, "UTF -8"); br = new BufferedReader(r); } finally { if ( is != null ){ try { is.close (); } catch ( IOException ignored ){} is = null; } if ( r != null ){ try{ r.close (); } catch ( IOException ignored ){} r = null; } if ( br == null ){ try{ br.close (); } catch ( IOException ignored ){} br = null; } 37 }

Examples try { is = u.openStream (); r = new InputStreamReader(is, "UTF -8"); br = new BufferedReader(r); } finally { if ( is != null ){ try { is.close (); } catch ( IOException ignored ){} is = null; } if ( r != null ){ try{ r.close (); } catch ( IOException ignored ){} r = null; } if ( br == null ){ try{ br.close (); } catch ( IOException ignored ){} br = null; Reader never closed } 37 }

Examples Iterator i = pinConnections.iterator (); PinLink currLink = (PinConnect.PinLink) i.next (); currLink.propagateSignals (); while (i.hasNext ()) { currLink = (PinConnect.PinLink) i.next (); currLink.propagateSignals (); } 38

Examples Iterator i = pinConnections.iterator (); PinLink currLink = (PinConnect.PinLink) i.next (); currLink.propagateSignals (); while (i.hasNext ()) { currLink = (PinConnect.PinLink) i.next (); currLink.propagateSignals (); } Incorrect iterator usage 38

Normal vs. Generated Input 39

Overview Analysis Static Runtime checking verification 40

Overview Analysis Static Runtime checking verification 40

State of the Art + specification Typestate Anomaly checking detection 41

State of the Art + specification Typestate Anomaly checking detection + Precise + Automatic - Needs specification - Imprecise 41

State of the Art + specification Typestate Anomaly checking detection + Precise + Automatic - Needs specification - Imprecise Combine both! Precise checker for mined multi-object protocols 41