Transactional memory with data Transactional memory with data - PowerPoint PPT Presentation

Transactional memory with data Transactional memory with data invariants: or “ “putting the C back putting the C back invariants: or in ACID” ” in ACID Tim Harris, Simon Peyton Jones Tim Harris, Simon Peyton Jones

Introduction Introduction • “List XYZ remains in sorted order List XYZ remains in sorted order” ” • “ URK! But how to detect this: Find thread1 thread1 [] [10] and instrument all updates? Distinguish sorted/unsorted lists in the type system? thread2 thread2 [5,10,20] thread3 thread3 [10,20] [20,10,5] Time

Transactions to the rescue Transactions to the rescue • Write Write executable invariants executable invariants to check the to check the • properties we’ ’re interested in re interested in properties we • Invariants fit in really nicely with transactions: Invariants fit in really nicely with transactions: • – Of course, the DB folk knew this long ago :-) Of course, the DB folk knew this long ago :-) – – Every transaction must preserve every invariant Every transaction must preserve every invariant – – Transactions define updates between Transactions define updates between consistent consistent states states – – Invariants can (must!) be broken within transactions Invariants can (must!) be broken within transactions – – It doesn It doesn’ ’t matter if an update is to the spine of the list t matter if an update is to the spine of the list – or to a key that it contains (unlike much related work, or to a key that it contains (unlike much related work, e.g. Eiffel and Spec#) e.g. Eiffel and Spec#)

A sorted list using invariants A sorted list using invariants Defining a function ‘ newSortedList ’ in Haskell-ish pseudo-code newSortedList = do { = do { newSortedList Construct an ordinary mutable list as usual r <- newList newList(); (); r <- check (isSorted isSorted r); r); check ( Assert that the list is sorted now, and every update to the return r; return r; list must keep it sorted } } Return the list we created • The runtime system keeps track of the invariant The runtime system keeps track of the invariant • and ensures transactions don’ ’t violate it t violate it – – the type the type and ensures transactions don system treats it as any other list system treats it as any other list

Semantics Semantics • The invariant is an STM action The invariant is an STM action… … so [in so [in • Haskell] it’ ’s guaranteed not to do I/O s guaranteed not to do I/O Haskell] it – But what if it loops? But what if it loops? – – What if it updates or allocates transactional state? – What if it updates or allocates transactional state? – What if it calls What if it calls check check to add yet another invariant? to add yet another invariant? – – What if it uses condition synchronization and blocks What if it uses condition synchronization and blocks – the transaction? the transaction?

First design choice: overview First design choice: overview • Run invariants in [closed] nested Run invariants in [closed] nested tx tx (so they see (so they see • the tentative updates), check they succeed (don’ ’t t the tentative updates), check they succeed (don throw an exception), then roll back each nested tx tx throw an exception), then roll back each nested • All this happens atomically with the user All this happens atomically with the user tx tx • Nested tx check invariants User’s and are then aborted (so no transaction interference with user tx) Time Whole lot occurs atomically

First design choice: semantics First design choice: semantics • The invariant is an arbitrary STM action The invariant is an arbitrary STM action… … • Your program – What if it loops? – What if it loops? loops… – What if it updates trans state? What if it updates trans state? – Can use them internally, but – What if it calls What if it calls check check ? ? – updates discarded – What if it blocks? What if it blocks? – Checked at that call, but then discarded The user’s tx blocks until the invariant can complete

STM Haskell STM Haskell Transactional state is held in TVars newTVar :: a -> STM ( :: a -> STM (TVar TVar a) a) newTVar readTVar readTVar :: :: TVar TVar a -> STM a a -> STM a writeTVar writeTVar :: :: TVar TVar a -> a -> STM () a -> a -> STM () atomic :: STM a -> IO a The type system distinguishes atomic :: STM a -> IO a transactional code (types like “STM a”) from general imperative code (types like “IO a”) incT :: :: TVar TVar Int Int -> -> STM STM () () incT Sequential composition of incT r = do { v <- r = do { v <- readTVar readTVar r r incT STM actions ; writeTVar writeTVar r (v+1) } r (v+1) } ; main = do { r <- atomic ( newTVar newTVar 0) 0) main = do { r <- atomic ( ; fork (atomic (incT ; fork (atomic ( incT r)) r)) ; atomic (incT ; atomic ( incT r) r) ; ... } ; ... }

Second design choice: restrict to reads Second design choice: restrict to reads • We can distinguish between STM actions that just We can distinguish between STM actions that just • read from the heap from STM actions that perform read from the heap from STM actions that perform updates / create TVars TVars / block / add invariants / block / add invariants updates / create An STM action where “e” may be forced to data ReadOnly ReadOnly data be unified with phantom types “Pure” or “Effect”. Sequencing “STM e a” and data Effect data Effect “STM f b” defined to require a==b. readTVar can be sequentially composed with any kind of STM action type STM e a = … type STM e a = … writeTVar forces “e” to be unified with “Effect” readTVar :: :: TVar TVar a -> STM e a a -> STM e a readTVar writeTVar :: a -> writeTVar :: a -> TVar TVar a -> STM Effect a a -> STM Effect a Combinators express their constraints (if any) atomic :: STM e a -> IO a atomic :: STM e a -> IO a check :: STM ReadOnly ReadOnly a -> STM Effect () a -> STM Effect () check :: STM

Invariants over state pairs Invariants over state pairs • Exciting observation from the Spec# group Exciting observation from the Spec# group • • Suppose we want invariants over state pairs Suppose we want invariants over state pairs • rather than single states rather than single states – “ “Value of x never decreases Value of x never decreases” ” – – versus versus “ “Value of x is always positive Value of x is always positive” ” – Take an STM action and run it in the pre- transactional state old :: STM ReadOnly ReadOnly a -> STM e a a -> STM e a old :: STM check (do { cur <- readTVar check (do { cur <- readTVar x; x; prev <- old ( <- old (readTVar readTVar x); x); prev assert (prev prev <= cur); } ) <= cur); } ) assert (

Implementation Implementation • Isn Isn’ ’t it slow checking every invariant upon every t it slow checking every invariant upon every • transaction? transaction? – It would be if we actually checked them all – It would be if we actually checked them all – When invariant checking is enabled we dynamically track – When invariant checking is enabled we dynamically track dependencies from TVars dependencies from TVars to invariants that depend on them to invariants that depend on them – “Pay for play Pay for play” ” (same wake-up mechanism we use for condition (same wake-up mechanism we use for condition – “ synchronization) synchronization) – – These are the only references to the data structures used to These are the only references to the data structures used to represent invariants: the GC reclaims these structures when represent invariants: the GC reclaims these structures when they are unreachable they are unreachable – (But note that the extra links may extend the lifetime of – (But note that the extra links may extend the lifetime of individual objects – – the invariant and everything reachable the invariant and everything reachable individual objects from it will be retained while at least one TVar from it will be retained while at least one TVar it depends on is it depends on is reachable) reachable)

Implementation (2) Implementation (2) TVar’ ’s s contents contents TVar v123 v123 Next wait-q entry Next wait-q entry Invariant’ ’s closure s closure Invariant Tx record from last execution Tx record from last execution TVar’ ’s s TVar contents contents v75 v75 Next wait-q entry Next wait-q entry

Conclusions Conclusions • First system to integrate invariants with First system to integrate invariants with • transactional memory transactional memory • “ “Putting the C back in ACID Putting the C back in ACID” ” • • Many of the ideas have a long history; both Many of the ideas have a long history; both • semantically and in the implementation semantically and in the implementation • Do we want some kind of Do we want some kind of “ “trigger trigger” ”-like -like • construct too? construct too? • Workloads Workloads workloads workloads workloads workloads •