the rv system tutorial
play

The RV System Tutorial Patrick Meredith and Grigore Rosu joint - PowerPoint PPT Presentation

The RV System Tutorial Patrick Meredith and Grigore Rosu joint work with Feng Chen, Dongyun Jin, Dennis Gri ffj th, Michael Ilseman Runtime Verification, Inc. University of Illinois Wednesday, November 10, 2010 The RV System


  1. MOP Monitoring Model Program Execution Observation/Abstraction Action Abstract Trace Verification … Monitors M 1 M 2 M 3 Action Monitors can be dynamically created or destroyed Parametric monitoring Wednesday, November 10, 2010

  2. Parametric Properties Needed, but hard to monitor e ffj ciently Parameters SafeEnum(Vector v, Enumeration+ e) { event create after(Vector v) returning(Enumeration e): ... event updatesource after(Vector v) : ... event next before(Enumeration e) : ... ere : create next* updatesource+ next @match { System.out.println(“Failed Enumeration!"); } } Wednesday, November 10, 2010

  3. Safe Enumeration as Parametric Property Usage pattern (using regular expressions) of three events updatesource(v) : change vector v create(v,e) : create enumeration e from vector v next(e) : use enumeration e Monitor next updatesource updatesource next create 0 1 3 2 Violation state Wednesday, November 10, 2010

  4. Monitoring Safe Enum … Main Thread: Task Thread: Vector v = //initialization; … … Enumeration e = v.elements(); … v.remove(0); … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  5. Monitoring Safe Enum … Main Thread: Task Thread: Vector v = //initialization; … … Enumeration e = v.elements(); … v.remove(0); … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  6. Monitoring Safe Enum … Main Thread: Task Thread: create Vector v = //initialization; … … Enumeration e = v.elements(); … v.remove(0); … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  7. Monitoring Safe Enum … Main Thread: Task Thread: create Vector v = //initialization; … … Enumeration e = v.elements(); … v.remove(0); … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  8. Monitoring Safe Enum … Main Thread: Task Thread: create Vector v = //initialization; … … Enumeration e = v.elements(); … v.remove(0); … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  9. Monitoring Safe Enum … Main Thread: Task Thread: create Vector v = //initialization; … updatesource … Enumeration e = v.elements(); … v.remove(0); … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  10. Monitoring Safe Enum … Main Thread: Task Thread: create Vector v = //initialization; … updatesource … Enumeration e = v.elements(); … v.remove(0); … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  11. Monitoring Safe Enum … Main Thread: Task Thread: create Vector v = //initialization; … updatesource … Enumeration e = v.elements(); … v.remove(0); next … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  12. Monitoring Safe Enum … Main Thread: Task Thread: create Vector v = //initialization; … updatesource … Enumeration e = v.elements(); … v.remove(0); next … Object obj = e.nextElement(); … next updatesource updatesource next create 0 1 3 2 Wednesday, November 10, 2010

  13. Lack of Parameters Leads to False Alarms … Main Thread: Task Thread: Vector v = //initialization; … … Enumeration e = v.elements(); … v.remove(0); … Object obj = e.nextElement(); … Wednesday, November 10, 2010

  14. Lack of Parameters Leads to False Alarms … Main Thread: Task Thread: Vector v = //initialization; … … Enumeration e = v.elements(); … v.remove(0); v2.remove(0); … Object obj = e.nextElement(); … Wednesday, November 10, 2010

  15. Lack of Parameters Leads to False Alarms … Main Thread: Task Thread: create Vector v = //initialization; … updatesource … Enumeration e = v.elements(); … v.remove(0); v2.remove(0); next … Object obj = e.nextElement(); … Wednesday, November 10, 2010

  16. Lack of Parameters Leads to False Alarms … Main Thread: Task Thread: create Vector v = //initialization; … updatesource … Enumeration e = v.elements(); … v.remove(0); v2.remove(0); next … Object obj = e.nextElement(); … Appear to be a violation but it is not; false alarm! Wednesday, November 10, 2010

  17. Adding Parameters to Events Main Thread: Task Thread: Vector v = //initialization; … … Enumeration e = v.elements(); … v2.remove(0); … Object obj = e.nextElement(); … Wednesday, November 10, 2010

  18. Adding Parameters to Events … Main Thread: Task Thread: create(v, e) Vector v = //initialization; … update(v2) … Enumeration e = v.elements(); … v2.remove(0); next(e) … … Object obj = e.nextElement(); … Wednesday, November 10, 2010

  19. Adding Parameters to Events … Main Thread: Task Thread: create(v, e) Vector v = //initialization; … update(v) update(v2) … Enumeration e = v.elements(); … v.remove(0); v2.remove(0); next(e) … … Object obj = e.nextElement(); … Wednesday, November 10, 2010

  20. Adding Parameters to Events … Main Thread: Task Thread: create(v, e) Vector v = //initialization; … update(v) update(v2) … Enumeration e = v.elements(); … v.remove(0); v2.remove(0); next(e) … … Object obj = e.nextElement(); … Parametric traces : traces containing events with parameters; Abundant in practice, especially in object-oriented programs Wednesday, November 10, 2010

  21. Checking Parametric Traces Wednesday, November 10, 2010

  22. Checking Parametric Traces parametric trace updatesource(v1) create (v1,e1) updatesource(v2) next(e1) create(v1,e2) updatesource(v1) next(e1) Wednesday, November 10, 2010

  23. Checking Parametric Traces parametric trace non-parametric monitor updatesource(v1) next create (v1,e1) create 1 0 updatesource(v2) updatesource next(e1) next 3 2 create(v1,e2) updatesource updatesource(v1) next(e1) Wednesday, November 10, 2010

  24. Checking Parametric Traces parametric trace non-parametric monitor updatesource(v1) next create (v1,e1) create 1 0 updatesource(v2) updatesource next(e1) next 3 2 create(v1,e2) updatesource updatesource(v1) next(e1) Wednesday, November 10, 2010

  25. Checking Parametric Traces parametric trace parametric monitor updatesource(v1) next create (v1,e1) create 1 0 updatesource(v2) updatesource next(e1) next 3 2 create(v1,e2) updatesource updatesource(v1) next(e1) Wednesday, November 10, 2010

  26. Parametric Monitors • Other approaches: Monolithic (centralized) monitors – Tracematches [Oxford], Program Query Language (PQL) [Stanford], Eagle [NASA], etc. – Bound to specific formalisms/checking mechanisms – Limited expressiveness, specific to application domains • Our solution: decentralized monitors – Formalism-independent, works with any formalism More expressive, adaptive to di fg erent domains • – Facilitates optimization (separation of concerns) Evaluation shows better performance • Wednesday, November 10, 2010

  27. Parametric Trace Slicing updatesource updatesource(v1) create (v1,e1) create updatesource(v2) next(e1) next create(v1,e2) updatesource(v1) updatesource next(e1) next For given parameters (v, e) Wednesday, November 10, 2010

  28. Parametric Trace Slicing v2, e2 v1, e1 v1, e2 v2, e1 updatesource updatesource(v1) create (v1,e1) create updatesource(v2) next(e1) next create(v1,e2) updatesource(v1) updatesource next(e1) next For given parameters (v, e) Wednesday, November 10, 2010

  29. Parametric Trace Slicing v1, e1 v1, e2 v2, e1 updatesource updatesource(v1) create (v1,e1) create updatesource(v2) next(e1) next create(v1,e2) updatesource(v1) updatesource next(e1) next For given parameters (v, e) Wednesday, November 10, 2010

  30. Parametric Trace Slicing v1, e1 v1, e2 v2, e1 updatesource updatesource(v1) create create (v1,e1) updatesource(v2) next(e1) next create(v1,e2) updatesource updatesource(v1) next next(e1) For given parameters (v, e) Wednesday, November 10, 2010

  31. Parametric Trace Slicing v1, e1 v1, e2 v2, e1 updatesource updatesource(v1) create create (v1,e1) trace slice updatesource(v2) next(e1) next create(v1,e2) updatesource updatesource(v1) next next(e1) For given parameters (v, e) Wednesday, November 10, 2010

  32. Parametric Trace Slicing v1, e1 v1, e2 v2, e1 updatesource updatesource updatesource(v1) create create (v1,e1) trace slice updatesource updatesource(v2) next(e1) next next create(v1,e2) create updatesource updatesource(v1) next next next(e1) For given parameters (v, e) Wednesday, November 10, 2010

  33. Naive monitoring of Parametric Traces • Every parametric trace contains multiple non- parametric trace slices, each corresponding to a particular parameter binding next updatesource updatesource next create 0 1 2 3 next updatesource updatesource next create 0 1 2 3 Wednesday, November 10, 2010

  34. Naive monitoring of Parametric Traces • Every parametric trace contains multiple non- parametric trace slices, each corresponding to a particular parameter binding next updatesource updatesource next v1, e1 create 0 1 2 3 next updatesource updatesource next v1, e2 create 0 1 2 3 Wednesday, November 10, 2010

  35. Parametric Trace Slicing - Challenges v1, e1 v1, e2 v2, e1 update update update(v1) update(v1) createEnum createEnum(v1,e1) createEnum(v1,e1) update(v2) update useEnum useEnum useEnum(e1) useEnum(e1) createEnum(v1,e2) createEnum update update(v1) update(v1) useEnum useEnum useEnum(e1) useEnum(e1) For given parameters (v, e) Wednesday, November 10, 2010

  36. Parametric Trace Slicing - Challenges v1, e1 v1, e2 v2, e1 update update update(v1) update(v1) How to do it efficiently? createEnum createEnum(v1,e1) createEnum(v1,e1) update(v2) update useEnum useEnum useEnum(e1) useEnum(e1) createEnum(v1,e2) createEnum update update(v1) update(v1) useEnum useEnum useEnum(e1) useEnum(e1) For given parameters (v, e) Wednesday, November 10, 2010

  37. Parametric Trace Slicing - Challenges v1, e1 v1, e2 v2, e1 update update update(v1) update(v1) How to do it efficiently? createEnum createEnum(v1,e1) createEnum(v1,e1) update(v2) update useEnum useEnum useEnum(e1) useEnum(e1) What if the trace is not complete? createEnum(v1,e2) createEnum update update(v1) update(v1) useEnum useEnum useEnum(e1) useEnum(e1) For given parameters (v, e) Wednesday, November 10, 2010

  38. Online Parametric Trace Slicing • Online: process events as receiving them and do not look back for the previous events • E ffj cient – Scan the trace once – Events discarded immediately after being processed • What information should be kept for the unknown future? Wednesday, November 10, 2010

  39. Overview • Monitoring • RV-Monitor Demo • RV-Monitor Techniques and Implementation – Monitor Synthesis – Parametric Monitoring – Optimizations • Prediction • RV-Predict Demo • RV-Predict Techniques and Implementation – Sliced Causality – Pipeline – Race Prediction Wednesday, November 10, 2010

  40. Slicing Example For given parameters (v, e) Wednesday, November 10, 2010

  41. Slicing Example v1 update update update(v1) For given parameters (v, e) Wednesday, November 10, 2010

  42. Slicing Example v1 v1, e1 update update update(v1) createEnum(v1,e1) For given parameters (v, e) Wednesday, November 10, 2010

  43. Slicing Example v1 v1, e1 update update update(v1) createEnum(v1,e1) For given parameters (v, e) Wednesday, November 10, 2010

  44. Slicing Example v1 v1, e1 update update update(v1) createEnum createEnum(v1,e1) For given parameters (v, e) Wednesday, November 10, 2010

  45. Slicing Example v1 v1, e1 v2 update update update(v1) createEnum createEnum(v1,e1) update(v2) update For given parameters (v, e) Wednesday, November 10, 2010

  46. Slicing Example v1 v1, e1 v2 e1 update update update(v1) createEnum createEnum(v1,e1) update(v2) update update useEnum(e1) useEnum For given parameters (v, e) Wednesday, November 10, 2010

  47. Slicing Example v1 v1, e1 v2 e1 update update update(v1) createEnum createEnum(v1,e1) update(v2) update update useEnum useEnum(e1) useEnum For given parameters (v, e) Wednesday, November 10, 2010

  48. Slicing Example v1 v1, e1 v2 e1 v2, e1 update update update(v1) createEnum createEnum(v1,e1) update(v2) update update useEnum useEnum(e1) useEnum For given parameters (v, e) Wednesday, November 10, 2010

  49. Slicing Example v1 v1, e1 v2 e1 v2, e1 update update update(v1) createEnum createEnum(v1,e1) update(v2) update update useEnum useEnum(e1) useEnum useEnum For given parameters (v, e) Wednesday, November 10, 2010

  50. Slicing Example v1 v1, e1 v2 e1 v2, e1 v1, e2 update update update update(v1) createEnum createEnum(v1,e1) update(v2) update update useEnum useEnum(e1) useEnum useEnum createEnum(v1,e2) createEnum For given parameters (v, e) Wednesday, November 10, 2010

  51. Slicing Example v1 v1, e1 v2 e1 v2, e1 v1, e2 update update update update(v1) createEnum createEnum(v1,e1) update(v2) update update useEnum useEnum(e1) useEnum useEnum createEnum(v1,e2) createEnum For given parameters (v, e) Wednesday, November 10, 2010

  52. Slicing Example v1 v1, e1 v2 e1 v2, e1 v1, e2 update update update update(v1) Optimization: based on static property analysis, generate createEnum createEnum(v1,e1) specialized slicing code for the given specification update(v2) update update useEnum useEnum(e1) useEnum useEnum createEnum(v1,e2) createEnum For given parameters (v, e) Wednesday, November 10, 2010

  53. Slicing Example v1 v1, e1 v2 v1, e2 update update update update(v1) Optimization: based on static property analysis, generate createEnum createEnum(v1,e1) specialized slicing code for the given specification update(v2) update useEnum useEnum(e1) createEnum(v1,e2) createEnum For given parameters (v, e) Wednesday, November 10, 2010

  54. RV-Monitor Performance Unsafe- Unsafe- Unsafe- All HasNext UnsafeIter MapIter SyncColl SyncMap Prop TM MOP RV TM MOP RV TM MOP RV TM MOP RV TM MOP RV RV antlr 1 4 -2 0 3 -2 3 3 1 -1 -1 -1 0 -2 0 0 bloat 2119 448 116 19194 569 251 OOM 1203 178 1359 746 212 1942 716 130 982 chart 1 0 -2 15 2 -1 1 0 -2 -2 -2 -1 -2 -2 -2 0 eclipse 1 -4 -2 1 -5 -4 0 -5 -3 -5 -4 -5 -5 -2 -3 -3 fop 2 4 -2 4 7 -1 9 7 -2 1 -2 -2 -1 -3 -1 1 hsqldb 15 0 -3 13 -1 -3 13 1 -3 9 -4 -2 7 -3 -3 -3 jython 13 0 0 11 0 1 150 18 3 11 1 1 10 0 0 4 luindex -7 1 -1 4 -2 -1 3 -1 0 -1 2 0 -1 2 0 12 lusearch 3 -1 -2 22 1 2 7 0 -7 3 0 -6 5 4 0 3 pmd 70 26 -1 207 12 5 OOM 181 56 40 13 2 58 17 -1 69 xalan 5 1 -1 16 4 0 5 5 0 7 -1 -2 7 0 -1 1 Fig. 6. Comparison of Tracematches (TM), JavaMOP (MOP), and RV : Comparison of Tracematches (TM), JavaMOP (MOP), and RV: Average percent runtime overhead 38 Wednesday, November 10, 2010

  55. Overview • Monitoring • RV-Monitor Demo • RV-Monitor Techniques and Implementation – Monitor Synthesis – Parametric Monitoring – Optimizations • Prediction • RV-Predict Demo • RV-Predict Techniques and Implementation – Sliced Causality – Pipeline – Race Prediction Wednesday, November 10, 2010

  56. Why Prediction • Concurrent programs are hard to analyze – Model checking: number of interleavings is prohibitively large – Testing: interleavings depend on environment • Combine dynamic and static methods to find bad behaviors near correct executions 40 Wednesday, November 10, 2010

  57. Our Solution • Sliced Causality – General purpose technique to predict (bad) behaviors from correct runs – Sound: No false alarms • RV-Predict – Tool implementing Sliced Causality – Allows for prediction of any property for which an algorithm exists – Better than tools specialized simply for data race or atomicity violations 41 Wednesday, November 10, 2010

  58. Prediction Example Property : “authenticate before access” Task Thread: Main Thread: s 1 : resource.authenticate(); … s 2 : flag.value = true; … … s 3 : if (! flag.value) while (! flag.value) Thread.yield() ; s 4 : resource.access(); … Observed execution: … s 1 s 2 s 3 s 4 … 42 Wednesday, November 10, 2010

  59. Prediction Example Property : “authenticate before access” Task Thread: Main Thread: s 1 : resource.authenticate(); … s 2 : flag.value = true; … … s 3 : if (! flag.value) Thread.yield() ; s 4 : resource.access(); … Observed execution: … s 1 s 2 s 3 s 4 … • Buggy S 4 can be executed before S 1 • Low possibility to hit error in testing 43 Wednesday, November 10, 2010

  60. Prediction Example Property : “authenticate before access” Task Thread: Main Thread: s 1 : resource.authenticate(); … s 2 : flag.value = true; … … s 3 : if (! flag.value) Thread.yield() ; s 4 : resource.access(); Can we predict the error even when the above … execution is observed? Yes! But not in the traditional way Observed execution: … s 1 s 2 s 3 s 4 … • Buggy S 4 can be executed before S 1 • Low possibility to hit error in testing 43 Wednesday, November 10, 2010

  61. Special Case: Data Races • Our techniques work for any behavioral property • One of the simplest properties is race detection – Two accesses to a shared variable can take place concurrently – At least one of the accesses is a write 44 Wednesday, November 10, 2010

  62. Overview • Monitoring • RV-Monitor Demo • RV-Monitor Techniques and Implementation – Monitor Synthesis – Parametric Monitoring – Optimizations • Prediction • RV-Predict Demo • RV-Predict Techniques and Implementation – Sliced Causality – Pipeline – Race Prediction Wednesday, November 10, 2010

  63. Overview • Monitoring • RV-Monitor Demo • RV-Monitor Techniques and Implementation – Monitor Synthesis – Parametric Monitoring – Optimizations • Prediction • RV-Predict Demo • RV-Predict Techniques and Implementation – Sliced Causality – Pipeline – Race Prediction Wednesday, November 10, 2010

  64. Predictive Runtime Analysis Search space 47 Wednesday, November 10, 2010

  65. Predictive Runtime Analysis Search space Observed execution 48 Wednesday, November 10, 2010

  66. Predictive Runtime Analysis Search space Observed execution Causal model 49 Wednesday, November 10, 2010

  67. Predictive Runtime Analysis Search space Observed execution Causal model Inferred executions Bug 50 Wednesday, November 10, 2010

  68. Predictive Runtime Analysis Search space Observed execution Causal model Inferred executions More relaxed causal Bug model yields more inferred executions 50 Wednesday, November 10, 2010

  69. Traditional Predictive Runtime Analysis: Happens-Before • Originally for distributed systems [Lamport-78] – Applied to shared memory systems by several authors • Causal model = non-permutable pairs of events = {intra-thread total orders} U {causal dependencies} – a – Causal dependency: if two events access the same location and one writes it, then their execution order matters • Inferred executions = extending the causal model 51 Wednesday, November 10, 2010

  70. Happens-Before Works... If Lucky Property : “authenticate before access” Task Thread: Main Thread: s 3 : if (! flag.value) Thread.yield() ; s 1 : resource.authenticate() s 2 : flag.value = true; s 4 : resource.access(); Observed execution: s 3 s 1 s 2 s 4 52 Wednesday, November 10, 2010

Recommend


More recommend