shared memory consistency models a tutorial outline
play

Shared Memory Consistency Models: A Tutorial Outline Concurrent - PowerPoint PPT Presentation

Oct 24, 2023 •194 likes •1.02k views

CS533 Concepts of Operating Systems Jonathan Walpole Shared Memory Consistency Models: A Tutorial Outline Concurrent programming on a uniprocessor The effect of optimizations on a uniprocessor The effect of the same optimizations on

  1. CS533 Concepts of Operating Systems Jonathan Walpole

  2. Shared Memory Consistency Models: A Tutorial

  3. Outline • Concurrent programming on a uniprocessor • The effect of optimizations on a uniprocessor • The effect of the same optimizations on a multiprocessor • Methods for restoring sequential consistency • Conclusion

  4. Outline • Concurrent programming on a uniprocessor • The effect of optimizations on a uniprocessor • The effect of the same optimizations on a multiprocessor • Methods for restoring sequential consistency • Conclusion

  5. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 0

  6. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 0

  7. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 0

  8. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 1

  9. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 1

  10. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Critical section is protected!

  11. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 0

  12. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 1

  13. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 1

  14. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 1

  15. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Both processes can block, but the critical section is still protected!

  16. Outline • Concurrent programming on a uniprocessor • The effect of optimizations on a uniprocessor • The effect of the same optimizations on a multiprocessor • Methods for restoring sequential consistency • Conclusion

  17. Write Buffer With Bypass SpeedUp: - Write takes 100 cycles - Buffering takes 1 cycle - So Buffer and keep going! Problem: Read from a location with a buffered write pending?

  18. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag1 = 0 Flag2 = 0

  19. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag1 = 0 Flag2 = 1 Flag2 = 0

  20. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag1 = 0 Flag2 = 1 Flag2 = 0

  21. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag1 = 0 Flag2 = 1 Flag2 = 0

  22. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag1 = 0 Flag2 = 1 Flag2 = 0 Critical section is not protected!

  23. Write Buffer With Bypass Rule: � � - If a write is issued, buffer it and keep executing � � Unless: there is a read from the same location (subsequent writes don't matter), then wait for the write to complete �

  24. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag1 = 0 Flag2 = 0

  25. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag1 = 0 Flag2 = 1 Flag2 = 0

  26. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Stall! Flag1 = 1 Flag1 = 0 Flag2 = 1 Flag2 = 0

  27. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 1 Flag2 = 0

  28. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 1 Flag2 = 1

  29. Is This a General Solution ? - If each CPU has a write buffer with bypass, and follows the rules, will the algorithm still work correctly?

  30. Outline • Concurrent programming on a uniprocessor • The effect of optimizations on a uniprocessor • The effect of the same optimizations on a multiprocessor • Methods for restoring sequential consistency • Conclusion

  31. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 0 Flag2 = 0

  32. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag1 = 0 Flag1 = 1 Flag2 = 0

  33. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag2 = 1 Flag1 = 0 Flag1 = 1 Flag2 = 0

  34. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag2 = 1 Flag1 = 0 Flag1 = 1 Flag2 = 0

  35. Dekker’s Algorithm Process 1:: Process 2:: Flag1 = 1 Flag2 = 1 If (Flag2 == 0) If (Flag1 == 0) critical section critical section Flag2 = 1 Flag1 = 0 Flag1 = 1 Flag2 = 0

  36. Its Broken! How did that happen? - write buffers are processor specific - writes are not visible to other processors until they hit memory

  37. Generalization of the Problem Dekker’s algorithm has the form: WX WY RY RX - The write buffer delays the writes until after the reads! - It reorders the reads and writes - Both processes can read the value prior to the other’s write!

  38. 1 WX RY WY RX 2 WX RY RX WY 3 WX WY RY RX 4 WX RX RY WY There are 4! or 24 possible orderings. 5 WX WY RX RY 6 WX RX WY RY 7 RY WX WY RX 8 RY WX RX WY 9 WY WX RY RX 10 RX WX RY WY 11 WY WX RX RY 12 RX WX WY RY 13 RY WY WX RX If either WX<RX or WY<RY 14 RY RX WX WY Then the Critical Section is protected 15 WY RY WX RX (Correct Behavior). 16 RX RY WX WY 17 WY RX WX RY 18 RX WY WX RY 19 RY WY RX WX 20 RY RX WY WX 21 WY RY RX WX 22 RX RY WY WX 23 WY RX RY WX 24 RX WY RY WX

  39. 1 WX RY WY RX 2 WX RY RX WY 3 WX WY RY RX 4 WX RX RY WY There are 4! or 24 possible orderings. 5 WX WY RX RY 6 WX RX WY RY 7 RY WX WY RX 8 RY WX RX WY 9 WY WX RY RX 10 RX WX RY WY 11 WY WX RX RY 12 RX WX WY RY 13 RY WY WX RX If either WX<RX or WY<RY 14 RY RX WX WY Then the Critical Section is protected 15 WY RY WX RX (Correct Behavior). 16 RX RY WX WY 17 WY RX WX RY 18 RX WY WX RY 19 RY WY RX WX 20 RY RX WY WX 18 of the 24 orderings are OK. 21 WY RY RX WX But the other 6 are trouble! 22 RX RY WY WX 23 WY RX RY WX 24 RX WY RY WX

  40. Another Example What happens if reads and writes can be delayed by the interconnect? - non-uniform memory access time - cache misses - complex interconnects

  41. Non-Uniform Write Delays Process 1:: Process 2:: Data = 2000; While (Head == 0) {;} Head = 1; LocalValue = Data Memory Interconnect Data = 0 Head = 0

  42. Non-Uniform Write Delays Process 1:: Process 2:: Data = 2000; While (Head == 0) {;} Head = 1; LocalValue = Data Memory Interconnect Data = 0 Head = 0

  43. Non-Uniform Write Delays Process 1:: Process 2:: Data = 2000; While (Head == 0) {;} Head = 1; LocalValue = Data Memory Interconnect Data = 0 Head = 0

  44. Non-Uniform Write Delays Process 1:: Process 2:: Data = 2000; While (Head == 0) {;} Head = 1; LocalValue = Data Memory Interconnect Data = 0 Head = 1

  45. Non-Uniform Write Delays Process 1:: Process 2:: Data = 2000; While (Head == 0) {;} Head = 1; LocalValue = Data Memory Interconnect Data = 0 Head = 1

  46. Non-Uniform Write Delays Process 1:: Process 2:: Data = 2000; While (Head == 0) {;} Head = 1; LocalValue = Data Memory Interconnect WRONG Data = 0 Head = 1 DATA !

  47. Non-Uniform Write Delays Process 1:: Process 2:: Data = 2000; While (Head == 0) {;} Head = 1; LocalValue = Data Memory Interconnect Data = 2000 Head = 1

  48. What Went Wrong? Maybe we need to acknowledge each write before proceeding to the next?

Recommend

Shared Memory Consistency Models: A Tutorial By Sarita Adve, Kourosh Gharachorloo WRL

Shared Memory Consistency Models: A Tutorial By Sarita Adve, Kourosh Gharachorloo WRL

Shared Memory Consistency Models: A Tutorial By Sarita Adve, Kourosh Gharachorloo WRL Research Report, 1995 Presentation: Vince Schuster Contents Overview Uniprocessor Review Sequential Consistency Relaxed Memory Models

598 views • 25 slides
Todays Topics - Distributed Shared Memory The Shared Memory Abstraction, why? Approaches

Todays Topics - Distributed Shared Memory The Shared Memory Abstraction, why? Approaches

Distributed Systems Lecture 6 1 Todays Topics - Distributed Shared Memory The Shared Memory Abstraction, why? Approaches to implementation; Consistency Models: Slide 1 Strict Consistency; Sequential Consistency Release

271 views • 12 slides
Distributed Shared Memory Shared memory : difficult to realize vs . easy to program with.

Distributed Shared Memory Shared memory : difficult to realize vs . easy to program with.

CSCE 613 : Operating Systems Memory Models CSCE 613: Interlude: Distributed Shared Memory Shared Memory Systems Consistency Models Distributed Shared Memory Systems page based shared-variable based Reading (old!):

679 views • 21 slides
Lecture 27: Pot-Pourri Todays topics: Consistency Models Shared memory vs

Lecture 27: Pot-Pourri Todays topics: Consistency Models Shared memory vs

Lecture 27: Pot-Pourri Todays topics: Consistency Models Shared memory vs message-passing Simultaneous multi-threading (SMT) GPUs Accelerators Disks and reliability 1 Coherence Vs. Consistency Recall that

630 views • 31 slides
CS6354: Memory models 1 To read more This days papers: Adve and Gharachorloo, Shared

CS6354: Memory models 1 To read more This days papers: Adve and Gharachorloo, Shared

CS6354: Memory models 1 To read more This days papers: Adve and Gharachorloo, Shared Memory Consistency Models: A Tutorial Boehm and Adve, Foundations of the C++ Concurrency Memory Model, section 1 only Supplementary

859 views • 44 slides
Distributed Shared Memory Distributed Shared Memory Systems Page based

Distributed Shared Memory Distributed Shared Memory Systems Page based

Operating Systems DSM Distributed Shared Memory Distributed Shared Memory Systems Page based Shared-variable based Consistency Models Strict consistency Sequential consistency Release consistency

320 views • 11 slides
Memory Consistency Models CSE 451 James Bornholt Memory consistency models The short version:

Memory Consistency Models CSE 451 James Bornholt Memory consistency models The short version:

Memory Consistency Models CSE 451 James Bornholt Memory consistency models The short version: Multiprocessors reorder memory operations in unintuitive, scary ways This behavior is necessary for performance Application programmers

803 views • 36 slides
Memory Consistency Models Adam Wierman Daniel Neill Adve, Pai, and Ranganathan. Recent advances

Memory Consistency Models Adam Wierman Daniel Neill Adve, Pai, and Ranganathan. Recent advances

Memory Consistency Models Adam Wierman Daniel Neill Adve, Pai, and Ranganathan. Recent advances in memory consistency models for hardware shared-memory systems, 1999. Gniady, Falsafi, and Vijaykumar . Is SC+ILP=RC? , 1999. Hill. Multiprocessors

405 views • 25 slides
A Tutorial By Sarita Adve &amp; Kourosh Gharachorloo Slides by Jim Larson Outline

A Tutorial By Sarita Adve &amp; Kourosh Gharachorloo Slides by Jim Larson Outline

Shared Memory Consistency Models: A Tutorial By Sarita Adve &amp; Kourosh Gharachorloo Slides by Jim Larson Outline Concurrent programming on a uniprocessor The effect of optimizations on a uniprocessor The effect of the same

1.49k views • 100 slides
Weak Memory Models: A Tutorial Jade Alglave University College London February 3rd, 2014

Weak Memory Models: A Tutorial Jade Alglave University College London February 3rd, 2014

Weak Memory Models: A Tutorial Jade Alglave University College London February 3rd, 2014 Sequential Consistency A comfortable model for concurrent programming would be Sequential Consistency (SC), as defined by Leslie Lamport in 1979: The

2.4k views • 51 slides
CSC2/458 Parallel and Distributed Systems Parallel Memory Systems Consistency Sreepathi Pai

CSC2/458 Parallel and Distributed Systems Parallel Memory Systems Consistency Sreepathi Pai

CSC2/458 Parallel and Distributed Systems Parallel Memory Systems Consistency Sreepathi Pai February 8, 2018 URCS Outline Memory Consistency Programming on Relaxed Consistency Machines Memory Models for Languages Special Relativity

375 views • 36 slides
Replication and Consistency Setting: Concurrent threads accessing shared data Roland Meyer (TU

Replication and Consistency Setting: Concurrent threads accessing shared data Roland Meyer (TU

Replication and Consistency in and hardware general memory consistency models in particular Roland Meyer Technische Universit at Kaiserslautern Roland Meyer (TU KL) Replication and Consistency IFIP WG 2.2 09/2016 1 / 32

1.76k views • 150 slides
Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides

Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides

Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides courtesy of Alvin Lebeck) Software DSM 101 Software DSM 101 Software-based distributed shared memory (DSM) provides an illusion of shared memory on

414 views • 12 slides
Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides

Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides

Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides courtesy of Alvin Lebeck) Software DSM 101 Software DSM 101 Software-based distributed shared memory (DSM) provides an illusion of shared memory on

236 views • 23 slides
D ISTRIBUTED S YSTEMS [COMP9243] Replication and consistency of memory objects Shared address

D ISTRIBUTED S YSTEMS [COMP9243] Replication and consistency of memory objects Shared address

S HARED A DDRESS S PACE DSM consists of two components: Shared address space D ISTRIBUTED S YSTEMS [COMP9243] Replication and consistency of memory objects Shared address space: Lecture 3b: Distributed Shared Memory Node 1 Node 2 Slide

798 views • 8 slides
Memory consistency models Computer Architecture J. Daniel Garca Snchez (coordinator) David

Memory consistency models Computer Architecture J. Daniel Garca Snchez (coordinator) David

Memory consistency models Memory consistency models Computer Architecture J. Daniel Garca Snchez (coordinator) David Expsito Singh Francisco Javier Garca Blas ARCOS Group Computer Science and Engineering Department University Carlos

520 views • 50 slides
Distributed Shared Memory Presented by Humayun Arafat 1 Outline Background Shared Memory,

Distributed Shared Memory Presented by Humayun Arafat 1 Outline Background Shared Memory,

Distributed Shared Memory Presented by Humayun Arafat 1 Outline Background Shared Memory, Distributed memory systems Distributed shared memory Design Implementation TreadMarks Comparison TreadMarks with Princetons home based protocol

524 views • 31 slides
Distributed Memory and Cache Consistency (some slides courtesy of Alvin Lebeck) Software DSM 101

Distributed Memory and Cache Consistency (some slides courtesy of Alvin Lebeck) Software DSM 101

Distributed Memory and Cache Consistency (some slides courtesy of Alvin Lebeck) Software DSM 101 Software-based distributed shared memory (DSM) provides an illusion of shared memory on a cluster. remote-fork the same program on each node

335 views • 14 slides
Synthesizing Memory Models from Framework Sketches and Litmus Tests James Bornholt Emina

Synthesizing Memory Models from Framework Sketches and Litmus Tests James Bornholt Emina

Synthesizing Memory Models from Framework Sketches and Litmus Tests James Bornholt Emina Torlak University of Washington Memory consistency models define memory reordering behaviors on mul&gt;processors Memory consistency models define

1.06k views • 86 slides
Shared-Memory Programming Models Programmierung Paralleler und Verteilter Systeme (PPV) Sommer

Shared-Memory Programming Models Programmierung Paralleler und Verteilter Systeme (PPV) Sommer

Shared-Memory Programming Models Programmierung Paralleler und Verteilter Systeme (PPV) Sommer 2015 Frank Feinbube, M.Sc., Felix Eberhardt, M.Sc., Prof. Dr. Andreas Polze Shared-Memory Parallelism 2 Process model All memory is local,

928 views • 72 slides
Parallel Models Different ways to exploit parallelism Outline Shared-Variables Parallelism

Parallel Models Different ways to exploit parallelism Outline Shared-Variables Parallelism

Parallel Models Different ways to exploit parallelism Outline Shared-Variables Parallelism threads shared-memory architectures Message-Passing Parallelism processes distributed-memory architectures Practicalities

640 views • 34 slides
Weak memory models INF4140 - Models of concurrency Weak memory models Fall 2016 30. 10. 2016

Weak memory models INF4140 - Models of concurrency Weak memory models Fall 2016 30. 10. 2016

Weak memory models INF4140 - Models of concurrency Weak memory models Fall 2016 30. 10. 2016 Overview Weak memory models 1 Introduction 2 Hardware architectures Compiler optimizations Sequential consistency Weak memory models 3 TSO

1.3k views • 87 slides
Outline Cache coherence the hardware view 1 2 Synchronization and memory consistency review 3

Outline Cache coherence the hardware view 1 2 Synchronization and memory consistency review 3

Outline Cache coherence the hardware view 1 2 Synchronization and memory consistency review 3 C11 Atomics 4 Avoiding locks 1 / 43 Important memory system properties Coherence concerns accesses to a single memory location - Must obey

858 views • 48 slides
Memory Consistency Models Nima Honarmand Fall 2015 :: CSE 610 Parallel Computer Architectures

Memory Consistency Models Nima Honarmand Fall 2015 :: CSE 610 Parallel Computer Architectures

Fall 2015 :: CSE 610 Parallel Computer Architectures Memory Consistency Models Nima Honarmand Fall 2015 :: CSE 610 Parallel Computer Architectures Why Consistency Models Matter Each thread accesses two types of memory locations

859 views • 46 slides

More recommend