Read-Copy-Update for Read-Copy-Update for HelenOS HelenOS - PowerPoint PPT Presentation

Read-Copy-Update for Read-Copy-Update for HelenOS HelenOS http://d3s.mff.cuni.cz Martjn Děcký decky@d3s.mff.cuni.cz CHARLES UNIVERSITY IN PRAGUE faculty of mathematjcs and physics faculty of mathematjcs and physics

Introductjon Introductjon HelenOS Microkernel multjserver operatjng system Relying on asynchronous IPC mechanism Major motjvatjon for scalable concurrent algorithms and data structures Martjn Děcký Researcher in computer science (operatjng systems) Not an expert on concurrent algorithms But very lucky to be able to cooperate with hugely talented people in this area Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 2

In a Nutshell In a Nutshell Asynchronous IPC = Communicatjng partjes may access the communicatjon facilitjes concurrently Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 3

In a Nutshell In a Nutshell Asynchronous IPC = Communicatjng partjes may access the communicatjon facilitjes concurrently → The state of the shared communicatjon facilitjes needs to be protected by explicit synchronizatjon means Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 4

In a Nutshell In a Nutshell Asynchronous IPC = Communicatjng partjes have to access the communicatjon facilitjes concurrently Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 5

In a Nutshell In a Nutshell Asynchronous IPC = Communicatjng partjes have to access the communicatjon facilitjes concurrently ← In order to counterweight the overhead of the communicatjon by doing other useful work while waitjng for a reply Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 6

In a Nutshell In a Nutshell Asynchronous IPC → Communicatjon facilitjes have to use concurrency-friendly (non-blocking) synchronizatjon means Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 7

In a Nutshell In a Nutshell Asynchronous IPC → Communicatjon facilitjes have to use concurrency-friendly (non-blocking) synchronizatjon means ← In order to avoid limitjng the achievable degree of concurrency Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 8

Basic Synchronizatjon Taxonomy Basic Synchronizatjon Taxonomy For accessing shared data structures Mutual exclusion synchronizatjon Temporal separatjon of scheduling entjtjes Typical means Disabling preemptjon , Dekker's algorithm , direct use of atomic test-and-set operatjons , etc. Typical mechanisms Locks , semaphores , conditjon variables , etc. [+] Relatjvely intuitjve semantjcs, well-known characteristjcs [-] Overhead, restrictjon of concurrency, deadlocks Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 9

Mutual Exclusion Synchronizatjon Mutual Exclusion Synchronizatjon Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 10

Basic Synchronizatjon Taxonomy Basic Synchronizatjon Taxonomy Non-blocking synchronizatjon Replace temporal separatjon by sophistjcated means that guarantee logical consistency Typical means Atomic writes , direct use of atomic read-modify-write operatjons , etc. Typical mechanisms Transactjonal memory, hazard pointers , Read-Copy-Update , etc. [+] Reasonable (almost no) overhead and restrictjon of concurrency in favorable cases, guarantee of progress [-] Less intuitjve semantjcs, sometjmes non-trivial characteristjcs, non-favorable cases, livelocks Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 11

Non-blocking Synchronizatjon Non-blocking Synchronizatjon Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 12

Non-blocking Synchronizatjon Non-blocking Synchronizatjon Wait-freedom Guaranteed system-wide progress and starvatjon-freedom (all operatjons are fjnitely bounded) Wait-freedom algorithms always exist [1], but the performance of general methods is usually inferior to blocking algorithms Wait-free queue by Kogan & Petrank [2] Lock-freedom Guaranteed system-wide progress, but individual threads can starve Four phases: Data operatjon, assistjng obstructjon, abortjng obstructjon, waitjng Obstructjon-freedom Guaranteed single thread progress if isolated for a bounded tjme (obstructjng threads need to be suspended) Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 13

From Means to Mechanism From Means to Mechanism Synchronizatjon Synchronizatjon means mechanism Individual instance of usage Generic reusable patuern Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 14

From Means to Mechanism From Means to Mechanism Synchronizatjon Synchronizatjon means mechanism Individual instance of usage Generic reusable patuern E.g. non-blocking list E.g. non-blocking list implementatjon using implementatjon using atomic pointer writes Read-Copy-Update Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 15

What Is Read-Copy-Update What Is Read-Copy-Update Non-blocking synchronizatjon mechanism Targetjng synchronizatjon of read-mostly pointer-based data structures with immutable values Favorable case: R/W ratjo of ~ 10:1 (but even 1:1 is achievable) Unlimited number of readers without blocking (not waitjng for other readers or writers) Litule overhead on the reader side (smaller than taking an uncontended lock) Readers have to tolerate “stale” data and late updates Readers have to observe “safe” access patuerns Synchronizatjon among writers out of scope of the mechanism Optjonal provisions for asynchronous reclamatjon Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 16

What Is Read-Copy-Update (2) What Is Read-Copy-Update (2) Read-side critjcal sectjon Delimited by r e a d _ l o c k ( ) and r e a d _ u n l o c k ( ) operatjons (non-blocking) Protected data can be referenced only inside the critjcal sectjon Safe a c c e s s ( ) methods for reading pointers Avoiding unsafe compiler optjmizatjons (reloading the pointer) Not necessary for reading values Quiescent state (a thread outside a critjcal sectjon) Grace period (all threads pass through a quiescent state) Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 17

What Is Read-Copy-Update (3) What Is Read-Copy-Update (3) Synchronous write-side update Atomically unlinking an old element Calling a s operatjon y n c h r o n i z e ( ) Blocks untjl a grace period elapses (all readers pass a quiescent state, no longer referencing the unlinked data) Possibility to reclaim or free the unlinked data Insertjng a new element using safe a operatjon s s i g n ( ) Avoiding unsafe compiler optjmizatjons and store reordering on weakly ordered architectures Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 18

Synchronous Update Example Synchronous Update Example I. head next v0 next v1 Atomic pointer update to remove the element with v0 from the list Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 19

Synchronous Update Example Synchronous Update Example I. II. head head next v0 next v0 next v1 next v1 Blocking on s y n c h r o n i z e ( ) During the grace period preexistjng readers can stjll access the “stale” element with v0 Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 20

Synchronous Update Example Synchronous Update Example I. II. III. head head head next v0 next v0 next v2 next v1 next v1 next v1 No reader can reference the element with v0 anymore – it can be reclaimed New element with v2 can be atomically inserted Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 21

What Is Read-Copy-Update (4) What Is Read-Copy-Update (4) Asynchronous write-side update Using a c a l l ( ) operatjon Non-blocking operatjon registering a callback Callback is executed afuer a grace period elapses Using a b a r r i e r ( ) operatjon Waitjng for all queued asynchronous callbacks Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 22

Grace Period Detectjon Grace Period Detectjon Cornerstone of any RCU algorithm Implicit trade-ofg between precision and overhead Any extension of a grace period is also a grace period Long (imprecise) grace periods Blocking synchronous writers for a longer tjme Increasing memory usage due to unreclaimed elements Short (precise) grace periods Increasing overhead on the reader side (need for memory barriers, atomic operatjons, other heavy-weight operatjons, etc.) Usual compromise Identjfying naturally occurring quiescent states for the given RCU algorithm Context switches, exceptjons (tjmer tjcks), etc. Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 23

The Big Picture ... The Big Picture ... Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 24

Motjvatjon for RCU in HelenOS Motjvatjon for RCU in HelenOS Foundatjon for a scalable concurrent data structure Developing a microkernel-specifjc RCU algorithm Specifjc requirements, constraints and use cases Last well-known RCU implementatjon for a microkernel in 2003 (K42) Martjn Děcký , FOSDEM 2014, February 2 nd 2014 Read-Copy-Update for HelenOS 25