specification free monitoring CS 119 can we avoid writing specs?
specification and programming specification runtime verification program input output 2
properties can be hard to write To quote quite excellent NASA software engineer when asked what properties his system would have to satisfy: “ I have absolutely no! idea what properties this system should satisfy ”. the problem is real: the problem is real: can we avoid writing specs? can we avoid writing specs? 3
generic checking-algorithms algorithm runtime verification program input output 4
learning specifications visualization specification specification learning program input output 5
6
how to avoid writing specs • generate specs from runs (reverse dynamic engineering) – instrument program to detect certain events – collect information during one or more runs and establish database of results – this database accumulates specification of nominal behavior • error checking algorithms for specific narrow problems – data races – high level data races – atomicity violations – deadlocks 7
learning specs from runs • DAIKON, learning JML: – http://groups.csail.mit.edu/pag/daikon/ • Perecotta, learning temporal formulas: – http://www.cs.virginia.edu/perracotta • Redux : learning call graphs etc. – http://valgrind.org/docs/redux2003.ps 8
so many traces Hmm … looks pretty good to me. good run bad run 9
algorithmic “potential” analysis Shoot … some foot prints of a bug! good run turn a hard to test property into a stronger easy to test property. bad run 1 0
pros and cons - Gives false positives + Scales well (one trace) - Gives false negatives + Often finds the bugs it is supposed to find - Only special bugs 1 1
data races
what is a data race? The traditional definition: A data race occurs when two concurrent threads access a shared variable and when at least one access is a write, and the threads use no explicit mechanism to prevent the accesses from being simultaneous. 1 3
a classic Java example Let’s consider the function increase(), which is a part of a class that acts as a counter public void increase() { counter++; } Although written as a single “increment” operation, the “++” operator is actually mapped into three JVM instructions [load operand, increment, write-back] 1 4
example – continued Thread A Thread B … … Context load oper load oper Switch increment increment write-back write-back … … counter = 3 4 We shall refer to this traditional notion of data race as a low-level data race , since it focuses on a single variable 1 5
low-level data races • the standard way to avoid low-level data races on a variable is to protect the variable with a lock: all accessing threads must acquire this lock before accessing the variable, and release it again after. • there exist several algorithms for analyzing multi- threaded programs for low-level data races. – we will mention the Eraser algorithm here (Savage et al 97), developed for C – and Racer, developed for Java … 1 6
Racer an algorithm for detecting data races in Java programs shifting to Racer slides send email to havelund@gmail.com to obtain these
high-level data races work work by y Cyrille Artho yrille Artho, Klaus Havelund, and laus Havelund, and Armin Biere rmin Biere slides by slides by Cyrille Artho yrille Artho included ncluded
data race void swap () { void reset () { lx = c.x; synchronized(this){ ly = c.y; c.x = 0; c.x = ly ; } c.y = lx ; synchronized(this){ } c.y = 0; } Pair of coordinates x and y . } Two threads. Problem: thread order non-deterministic. Data corruption possible! Lock protection needed. 1 9
repairing the situation: High-Level Data Race protecting x and y in swap 5,8 void swap () { void reset () { synchronized(this){ synchronized(this){ lx = c.x; c.x = 0; 0,8 ly = c.y; } } synchronized(this){ synchronized(this){ c.y = 0; c.x = ly ; } 8,0 c.y = lx ; } 8,0 } Result is neither a swap or a reset } All field accesses synchronized: Racer reports no errors. No classical data race for these threads, but clearly undesired behavior! Problem: swap may run while reset is in progress! 2 0
the problem • the reset method releases its lock in between setting x and then setting y. • this gives the swap method the chance to interleave the two partial resets. • the swap method “has it right”: it holds its lock during operation on x and y. 2 1
the solution • this difference in views can be detected dynamically. • essentially, this approach tries to infer what the developer intended when writing the multi-threaded code, by discovering view inconsistencies. • depends on at least one thread getting it right. 2 2
Views express per thread what fields are guarded by a lock. some examples of views: Thread a Thread b Thread c synchronized (L) synchronized (L) synchronized (L) { { { access(x); access(x); access(x); access(y); } } } synchronized (L) synchronized (L) { { { {x,y} } access(x); access(y); Thread d } access(y); } synchronized (L) { {x},{y} } { {x},{x,y} } { access(x); Consistent: a and c, a and d } { {x} } Inconsistent: a and b 2 3
the algorithm 1) For each thread, for each lock, identify all fields covered by that lock (views). 2) For each thread, find the views that have no other view that contains them (maximal views). 3) For each pair of threads t1 and t2: find the intersection between t1’s maximal view and the views of t2. 4) Verify that those intersections form a chain. That is: s1 ⊆ s2 ⊆ s3 ⊆ ⋯ 2 4
low-level versus high-level data races Low-Level High-Level For each variable: a lock set For each lock: a variable set (several) L2 L1 L2 L1 L2 L1 L3 y y x x x y 2 5
applying algorithm to example void swap () { void reset () { synchronized(this){ synchronized(this){ lx = c.x; c.x = 0; ly = c.y; } } synchronized(this){ synchronized(this){ c.y = 0; c.x = ly ; } Overlaps are: c.y = lx ; } {x} and {y}. } } {x} ⊆ {y} x , y x {y} ⊆ {x} y maximal of swap 2 6
a formal view Let I be the set of object instances generated by a particular run of a Java program. Then F is the set of all fields of all instances in I A view v ∈ P( F ) is a subset of F Let l be a lock, t a thread, and B ( t , l ) the set of all synchronized blocks using lock l executed by thread t . For b ∈ B ( t , l ), a view generated by t with respect to l , is defined as the set of fields accessed in b by t The set of generated views V (t) ⊆ P(F) of a thread t is the set of all views v generated by t 2 7
maximal view A view v m generated by a thread t is a maximal view, if it is maximal with respect to set inclusion in V(t): ∀ v ∈ V(t) [( v m ⊆ v ) → ( v m = v )] Note that this definition suggests that there might be more than a single maximal view Let M(t) denote the set of all maximal views of thread t 2 8
overlapping Only two views which have fields in common can be responsible for a conflict. This observation is the motivation for the next definition: Given a set of views V(t) generated by t , and a view v’ generated by another thread, the overlapping views of t with v’ are all non-empty intersections of views in V( t ) with v’ : overlap ( t , v’ ) = { v ∩ v’ | ( v ∈ V( t )) ∧ ( v ∩ v ’ ≠ Ø)} 2 9
view vompatibility A set of views V( t ) is compatible with the maximal view v m of another thread if all overlapping views of t with v m form a chain: compatible ( t , v m ) if ∀ v 1 , v 2 ∈ overlap ( t , v m ) [( v 1 ⊆ v 2 ) ∨ ( v 2 ⊆ v 1 )] 3 0
view consistency View consistency is the mutual compatibility between all threads: A thread is only allowed to use views that are compatible with the maximal views of all other threads. ∀ t 1 ≠ t 2 , v m ∈ M( t 1 ) [compatible( t 2 , v m )] 3 1
HL data race in Remote Agent Database update( ) If( & not ok( )) issueWarning() set( ) Task Monitor Flag 3 2
neither sound nor complete False positive when one thread uses coarser locking than required due to efficiency. x x L L y y False negatives when: x x L L All threads use the same locking y y Random execution trace does not expose problem 3 3
so, what is it good for? • much higher chance of detecting an error than if one relies on actually executing the particular interleaving that leads to an error, without requiring much computational resources. • developers seem to follow the guideline of view consistency to a surprisingly large extent. 3 4
atomicity checking stale (outdated) values work work by y Cyrille Artho yrille Artho, Klaus Havelund, and laus Havelund, and Armin Biere rmin Biere slides by slides by Cyrille Artho yrille Artho included ncluded
recall the high level data race void swap () { void reset () { synchronized(this){ synchronized(this){ lx = c.x; c.x = 0; ly = c.y; } } synchronized(this){ synchronized(this){ c.y = 0; c.x = ly ; } c.y = lx ; } } } Problem: swap may run while reset is in progress! 3 6
Recommend
More recommend
