Architectural Support for Atomic Durability in Non-Volatile Memory Arpit Joshi , Vijay Nagarajan, Stratis Viglas, Marcelo Cintra NVMW 2018
Summary • Non-Volatile Memory (NVM) - on the memory bus • enables in-memory persistent data structures • Persistent data structures require an atomic durability primitive to ensure crash consistency • Logging is a technique to provide atomic durability • ATOM: hardware support for atomic durability by way of undo logging 2
Atomic Durability • All or nothing persists: think transactions ( A CI D ) 3
Atomic Durability • All or nothing persists: think transactions ( A CI D ) Initial State A 100 B 100 3
Atomic Durability • All or nothing persists: think transactions ( A CI D ) Initial State A 100 B 100 Atomic_Begin A = A - 50 B = B + 50 Atomic_End 3
Atomic Durability • All or nothing persists: think transactions ( A CI D ) Final State Initial State A 100 B 100 A 100 B 100 Atomic_Begin A = A - 50 B = B + 50 Atomic_End 3
Atomic Durability • All or nothing persists: think transactions ( A CI D ) Final State Initial State A 100 B 100 A 100 B 100 Final State A 50 B 150 Atomic_Begin A = A - 50 B = B + 50 Atomic_End 3
Atomic Durability • All or nothing persists: think transactions ( A CI D ) Final State Initial State A 100 B 100 A 100 B 100 Final State A 50 B 150 Atomic_Begin A = A - 50 Final State Final State B = B + 50 Atomic_End A 50 B 100 A 100 B 150 3
Atomic Durability • All or nothing persists: think transactions ( A CI D ) Final State Initial State A 100 B 100 A 100 B 100 Final State A 50 B 150 Atomic_Begin A = A - 50 Final State Final State B = B + 50 Atomic_End A 50 B 100 A 100 B 150 3
Mechanisms Shadow Paging Write-Ahead-Logging ➡ beneficial for coarse grained updates 4
Mechanisms Shadow Paging Write-Ahead-Logging ➡ beneficial for coarse grained updates REDO UNDO ➡ reads redirection ➡ fine grained log->data ordering ➡ victim cache 4
Undo Logging 1. Compute : Compute the new value (V = A - 50) 2. Log : Write old value of data to log space in persistent memory (Log [A , 100]) 3. Modify : Modify data in-place (A = V) NVM Data Log A 100 5
Undo Logging 1. Compute : Compute the new value (V = A - 50) 2. Log : Write old value of data to log space in persistent memory (Log [A , 100]) 3. Modify : Modify data in-place (A = V) NVM Data Log A 100 A 100 5
Undo Logging 1. Compute : Compute the new value (V = A - 50) 2. Log : Write old value of data to log space in persistent memory (Log [A , 100]) 3. Modify : Modify data in-place (A = V) NVM Data Log A 100 50 A 100 5
Undo Logging 1. Compute : Compute the new value (V = A - 50) 2. Log : Write old value of data to log space in persistent memory (Log [A , 100]) 3. Modify : Modify data in-place (A = V) NVM Data Log A 100 50 A 100 Log writes reach NVM before data writes. (Log —> Data ordering) 5
Undo Logging 1. Compute : Compute the new value (V = A - 50) 2. Log : Write old value of data to log space in persistent memory (Log [A , 100]) 3. Modify : Modify data in-place (A = V) NVM Data Log A 100 50 A 100 Logging is essentially a data movement task. 5
System Architecture Core Core Cache Cache DRAM NVM Secondary Storage Secondary Storage Disk Based Persistence NVM Based Persistence 6
System Architecture Core Core Cache Cache DRAM NVM Software Controlled Secondary Storage Secondary Storage Disk Based Persistence NVM Based Persistence 6
System Architecture Core Core Cache Cache Hardware Controlled DRAM NVM Software Controlled Secondary Storage Secondary Storage Disk Based Persistence NVM Based Persistence 6
Logging with Disk Compute Log Modify Flush Log Flush Data 7
Logging with Disk Volatile Phase Persistence Phase Compute Log Modify Flush Log Flush Data 7
Logging with Disk Volatile Phase Persistence Phase Compute Log Modify Flush Log Flush Data Clear separation of volatile and persistence phases. 7
Logging with NVM Compute Log Flush Log Modify Flush Data 8
Logging with NVM Compute Log Flush Log Modify Flush Data Volatile and persistence phases overlap. 8
ATOM Compute Modify Flush Data Log Flush Log In Hardware 9
ATOM Compute Modify Flush Data Log Flush Log In Hardware Goal: Move logging out of critical path. 9
Programming Model while ( ! Done ) { ATOMIC_BEGIN Write Undo Log while ( ! Done ) { Flush Log Modify Data Modify Data } } Flush Data Flush Data ATOMIC_END Software Logging ATOM 10
Baseline Hardware Logging Core • Create Undo Log • on a store, write old Cache value to log A 100 • Flush Undo Log • enforce log —> data ordering NVM Data Log A 100 11
Baseline Hardware Logging A = 50 Core • Create Undo Log • on a store, write old Cache value to log A 100 • Flush Undo Log • enforce log —> data ordering NVM Data Log A 100 11
Baseline Hardware Logging A = 50 Core • Create Undo Log • on a store, write old Cache value to log L(A) = 100 A 100 • Flush Undo Log • enforce log —> data ordering NVM Data Log A 100 A 100 11
Baseline Hardware Logging A = 50 Core • Create Undo Log • on a store, write old Log Cache value to log Done L(A) = 100 A 100 50 • Flush Undo Log • enforce log —> data ordering NVM Data Log A 100 A 100 11
ATOM Design Philosophy Where is log —> data ordering enforced? Core Store Queue Cache Memory Controller Store Buffer NVM 12
ATOM Design Philosophy Where is log —> data ordering enforced? Core Store Queue Baseline Design Cache Memory Controller Store Buffer NVM 12
ATOM Design Philosophy Where is log —> data ordering enforced? Core Store Queue Baseline Design Cache Memory Controller Store Buffer ATOM Design NVM 12
Baseline Implementation SQ Cache Mem Ctrl Memory 13
Baseline Implementation SQ ST(A) Cache Mem Ctrl Memory 13
Baseline Implementation SQ ST(A) Cache L(A) Mem Ctrl Memory 13
Baseline Implementation SQ ST(A) Cache L(A) Mem Ctrl L(A) Memory 13
Baseline Implementation SQ ST(A) Cache L(A) Mem Ctrl L(A) Memory WRITE L(A) 13
Baseline Implementation SQ ST(A) Cache L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) 13
Baseline Implementation SQ ST(A) Cache L(A) L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) 13
Baseline Implementation SQ ST(A) ST(A) Cache L(A) L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) 13
Baseline Implementation Store Completion Time SQ ST(A) ST(A) Cache L(A) L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) 13
Baseline Implementation Store Completion Time SQ ST(A) ST(A) Cache L(A) L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) Log persist operation in the critical path of retiring stores. 13
ATOM Posted Log SQ Cache Mem Ctrl Memory 14
ATOM Posted Log SQ ST(A) Cache Mem Ctrl Memory 14
ATOM Posted Log SQ ST(A) Cache L(A) Mem Ctrl Memory 14
ATOM Posted Log SQ ST(A) Cache L(A) Mem Ctrl L(A) Memory 14
ATOM Posted Log SQ ST(A) Cache L(A) Mem Ctrl L(A) Memory WRITE L(A) 14
ATOM Posted Log SQ ST(A) Cache L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) 14
ATOM Posted Log SQ ST(A) Cache L(A) L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) 14
ATOM Posted Log SQ ST(A) ST(A) Cache L(A) L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) 14
ATOM Posted Log Store Completion Time SQ ST(A) ST(A) Cache L(A) L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) 14
ATOM Posted Log Store Completion Time SQ ST(A) ST(A) Cache L(A) L(A) Mem Ctrl L(A) L(A) Memory WRITE L(A) Remove log persist operations from the critical path by enforcing ordering at memory controller. 14
ATOM Store Miss SQ Cache Mem Ctrl Memory 15
ATOM Store Miss SQ Cache Mem Ctrl Memory RD (A) 15
ATOM Store Miss SQ Cache Mem Ctrl RD(A) Memory RD (A) 15
ATOM Store Miss SQ Cache RD(A) Mem Ctrl RD(A) Memory RD (A) 15
ATOM Store Miss SQ Cache RD(A) L(A) Mem Ctrl RD(A) Memory RD (A) 15
ATOM Store Miss SQ Cache RD(A) L(A) Mem Ctrl L(A) RD(A) Memory RD (A) 15
ATOM Store Miss SQ Cache RD(A) L(A) Mem Ctrl L(A) RD(A) Memory WRITE L(A) RD (A) 15
ATOM Store Miss SQ Cache RD(A) L(A) Mem Ctrl L(A) L(A) RD(A) Memory WRITE L(A) RD (A) 15
ATOM Store Miss SQ Cache RD(A) L(A) L(A) Mem Ctrl L(A) L(A) RD(A) Memory WRITE L(A) RD (A) 15
ATOM Store Miss SQ ST(A) Cache RD(A) L(A) L(A) Mem Ctrl L(A) L(A) RD(A) Memory WRITE L(A) RD (A) 15
ATOM Store Miss Store Completion Time SQ ST(A) Cache RD(A) L(A) L(A) Mem Ctrl L(A) L(A) RD(A) Memory WRITE L(A) RD (A) 15
Recommend
More recommend
