The Push/Pull Model of Transactions Eric Koskinen Matthew - PowerPoint PPT Presentation

The Push/Pull Model of Transactions Eric Koskinen Matthew Parkinson Yale University, New Haven Microsoft Research, Cambridge United States United Kingdom CertiKOS Weekly Lunch 16 October 2015 Appeared in PLDI 2015

Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm Shared Concurrent Concurrent Concurrent Memory HashMap Queue List

Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm Shared Concurrent Concurrent Concurrent Memory HashMap Queue List Adding atomic to a language . . . Version 4.7

Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm Shared Concurrent Concurrent Concurrent Memory HashMap Queue List Adding atomic to a language . . . Serializable. Opaque. …

Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm Shared Concurrent Concurrent Concurrent Memory HashMap Queue List Adding atomic to a language . . . many issues arise in the implementation. ? ? ? Serializable. Opaque. …

! m s i m i s s e P Thread atomic { x changed? x := 3; y changed? y := 9; } O p t i m i s m ! Thread atomic { x := 3; y := 9; } all ok?

! m s i m i s s e P Opacity! Interleaved execution equivalent to some serial execution and cannot observe intermediate state. ! y t i l i b a z i l a i r e S O p t i m i s m ! Interleaved execution s equivalent to some n x T serial execution. t n e d n e p e D

! m s i m i s s e P Opacity! ! y t i l i b a z i l a i r e S O p t i m i s m Memory Ops ! Thread Thread atomic { atomic { x := 3; stk.push(4); ADT Ops y := 9; ht.get('a'); } (Boosting) }

! m s i m i s s e P Opacity! Hardware TM Software TM Hybrid TM ! y t i l i b a z i l a i r e S O p t i m i s m Memory Ops ! s n x T t n e ADT Ops d n e p e (Boosting) D

Consistency ! m s i m i s s e P Opacity! Irrevocability Open Nested Hardware TM Software TM Hybrid TM ! y t i l i b a z L i l a a i z r e y S R e a O d s p t i m i s m Memory Ops ! s n x T t n e ADT Ops d Eager Writes n e p e (Boosting) D

Consistency ! m s i m i s s e P Opacity! Irrevocability Open Nested Hardware TM Software TM Hybrid TM ! y t i l i b a z L i l a a i z r e y S R e a O d s p t i m i s m Memory Ops ! s n x T ? ? ? t Serializable. Opaque. … n e ADT Ops d Eager Writes n e p e (Boosting) D

The P USH /P ULL Model ⟨ ht.map(7,2) ,_, gUC ⟩ Push Pull ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ q.enq(‘a’) ,_ ⟩

The P USH /P ULL Model ⟨ ht.map(7,2) ,_, gUC ⟩ Push Pull ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ q.enq(‘a’) ,_ ⟩ Serializable. Opaque. …

Linearizable Commute The P USH /P ULL Model ⟨ ht.map(7,2) ,_, gUC ⟩ Push Pull ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ q.enq(‘a’) ,_ ⟩ Serializable. Opaque. …

Thread Thread atomic { atomic { if(b) { while(…){ opA(…); if(x>MAX) } else … opB(…); } … }

Thread Thread atomic { atomic { if(b) { while(…){ opA(…); if(x>MAX) } else … opB(…); } … Linearizable Linearizable } Operation Operation

Thread Thread { tx c 1 , σ 1 } { tx c 2 , σ 2 } 1. Abstract away the language semantics

Thread Thread { tx c 1 , σ 1 } { tx c 2 , σ 2 } example language: 1. Abstract away the language semantics

Thread Thread G { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } ⟨ op1 , _ ⟩ ⟨ op2 , _ ⟩ 1. Abstract away the language semantics 2. Abstract away the state (with logs)

Thread Thread G { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } ⟨ op1 , _ ⟩ ⟨ op2 , _ ⟩ Closed under log prefix Parameterized by predicate allowed 1. Abstract away the language semantics 2. Abstract away the state (with logs)

Thread Thread G { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } ⟨ op1 , _ ⟩ ⟨ op2 , _ ⟩ Closed under log prefix Parameterized by predicate allowed Log Equality

Thread Thread G { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } ⟨ op1 , _ ⟩ ⟨ op2 , _ ⟩ 7 Simple Rules: Apply Push Pull Unpull Unpush Unapply Commit Log Equality

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } Two Logs ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) , σ , σ 1 , c ⟩ atomic { atomic { op1; op3; op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) , σ , σ 1 , c ⟩ atomic { atomic { ⟨ q.enq(‘a’) , σ 1 , σ ' 1 , c 1 ⟩ op1; op3; Apply op2; op4; … … } } 1 1 Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) , σ , σ 1 , c ⟩ atomic { atomic { ⟨ q.enq(‘a’) , σ 1 , σ ' 1 , c 1 ⟩ op1; op3; Apply op2; op4; … … } } A PPLY Criterion ( i ) : Possible next op 1 1 Criterion ( ii ) : L 1 allows ⟨ m, σ 1 , σ ' 1 ,id ⟩ Valid semantics of log Criterion ( iii ) : fresh ( id ) append { tx c 1 , σ 1 , L 1 }, G ➝ { tx c' 1 , σ ' 1 , L 1 ·[ op, σ 1 , σ ' 1 , c 1 ]}, G Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) , σ , σ 1 , c ⟩ atomic { atomic { Unapply ⟨ q.enq(‘a’) , σ 1 , σ ' 1 , c 1 ⟩ op1; op3; op2; op4; … … } } U NAPPLY { tx c' 1 , σ ' 1 , L 1 ·[ op, σ 1 , σ ' 1 , c 1 ]}, G 
 ➝ { tx c 1 , σ 1 , L 1 }, G Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } Two Logs ⟨ ht.map(7,2) ,_, gUC ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } Two Logs ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) ,_ ⟩ Push ⟨ ht.map(3,x) ,_, gUC ⟩ atomic { atomic { op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Application : Optimism vs. Pessimism Thread Thread Application : Out-of-order P USH ing with redo-logs { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } Two Logs ⟨ ht.map(7,2) ,_, gUC ⟩ Push ⟨ ht.map(3,x) ,_, gUC ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ Left-mover over logs op2; op4; … … } } P USH op ◀ ⎣ ` L ⎦ unpushed Criterion ( i ) : Act as if op happens next ⎣ G ⎦ gUC ∖ ⎣ L 1 ⎦ pushed ◀ op Criterion ( ii ) : No conflict w/ other uncmted Criterion ( iii ) : G allows op { tx c , σ , ` L ·[ op ]· L' }, G Uncommitted ➝ { tx c , σ , ` L ·[ op ]· L' }, G ·[ op , gUC ] Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread Application : Opacity [GK’08] and dependent transactions [RRHW’09] { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } ⟨ ht.map(7,2) ,_, gUC ⟩ Pull ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ atomic { atomic { Push ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ op2; op4; … … } } P ULL Criterion ( i ) : op ∉ L 2 Didn’t pull already L 2 allows op Criterion ( ii ) : Local log allows op op ◀ ⎣ L 2 ⎦ pushed ∪ ⎣ L 2 ⎦ unpushed Criterion ( iii ) : Can act as if op happened earlier { tx c 2 , σ 2 , L 2 }, G 1 ·[ op , g ]· G 2 ➝ { tx c 2 , σ 2 , L 2 ·[ op ]}, G 1 ·[ op , g ]· G 2 Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ Apply op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ Push ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ Pull op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ ht.get(5) , _, gUC ⟩ op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } G { tx c 2 , σ 2 , L 2 } ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ Pull ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ ht.get(5) , _, gUC ⟩ ⟨ q.enq(‘a’) , _ ⟩ op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit