exploiting microarchitectural flaws
play

Exploiting Microarchitectural Flaws in the Heart of the Memory - PowerPoint PPT Presentation

Exploiting Microarchitectural Flaws in the Heart of the Memory Subsystem Daniel Moghimi, Worcester Polytechnic University Feb 20, 2020 Columbia University Spoiler!! 2 CPU Memory Subsystem Allocation Queue Front End CPU Memory Subsystem


  1. MemJam – 4K Aliasing across Sibling Threads Core Thread A Thread B Store 0x12ABCDEF Load 0xFECD1 Store 0x12ABCDEF Load 0xFECD2 Execute & Time Store 0x12ABCDEF Load 0xFECD3 Store 0x12ABCDEF Load 0xFECD4 Store 0x12ABCDEF Load 0xFECD5 Store 0x12ABCDEF Load 0xFECD6 Store 0x12ABCDEF Load 0xFECD7 Store 0x12ABCDEF Load 0xFECD8 Store 0x12ABCDEF Store 0x12ABCDEF 40

  2. MemJam – 4K Aliasing across Sibling Threads Core Thread A Thread B Store 0x12ABC200 Load 0xFECD1 Store 0x12ABC200 Load 0xFECD2 Execute & Time Store 0x12ABC200 Load 0xFECD3 Store 0x12ABC200 Load 0xFECD4 Store 0x12ABC200 Load 0xFECD5 Store 0x12ABC200 Load 0xFECD6 Store 0x12ABC200 Load 0xFECD7 Store 0x12ABC200 Load 0xFECD8 Store 0x12ABC200 Store 0x12ABC200 41

  3. MemJam – 4K Aliasing across Sibling Threads Core Thread A Thread B Store 0x12ABC Load 0xFECD1 Store 0x12ABC Load 0xFECD2 Execute & Time Store 0x12ABC Load 0xFECD3 Store 0x12ABC Load 0xFECD4 Store 0x12ABC Load 0xFECD5 Store 0x12ABC Load 0xFECD6 Store 0x12ABC Load 0xFECD7 Store 0x12ABC Load 0xFECD8 Store 0x12ABC Store 0x12ABC 42

  4. MemJam – Intra Cache Line Resolution Least 12 bits (Virtual Address = Physical Address) Rest of the bits (Virtual != Physical) 43

  5. MemJam – Intra Cache Line Resolution Least 12 bits (Virtual Address = Physical Address) Rest of the bits (Virtual != Physical) L1 Cache Attacks 44

  6. MemJam – Intra Cache Line Resolution Least 12 bits (Virtual Address = Physical Address) Rest of the bits (Virtual != Physical) L1 Cache Attacks L2/L3 Cache Attacks 45

  7. MemJam – Intra Cache Line Resolution Least 12 bits (Virtual Address = Physical Address) Rest of the bits (Virtual != Physical) MemJam L1 Cache Attacks L2/L3 Cache Attacks • Conflicted intra-cache line Leakage (4-byte granularity) • Higher time → Memory accesses with the same bit 3 - 12 • 4 bits of intra-cache level leakage 46

  8. MemJam – Attacking So-Called Constant Time AES • Scatter-gather implementation of AES • Intel SGX Software Development Kit (SDK) and IPP Cryptography Library • 256 S-Box – 4 Cache Line • Cache independent access pattern 64 Bytes A LINE 2 4 Cache Lines B LINE 2 C LINE 2 D LINE 2 A B C D B S-Box Lookup 47

  9. MemJam – Attacking So-Called Constant Time AES 64 Bytes LINE 2 4 Cache Lines

  10. MemJam – AES Key Recovery

  11. Are there other Address Aliasing? Memory Subsystem L1 Store Buffer DATA PFN [8:0] VFN Offset DTLB DATA PFN [8:0] VFN Offset … …. … … DATA Offset PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN Offset DATA PFN VFN Offset DATA PFN [8:0] VFN Offset DATA PFN VFN Offset DATA PFN [8:0] VFN Offset … …. … … VFN Offset DATA PFN [8:0] VFN Offset DATA PFN

  12. Spoiler: Finding Undocumented Aliasing … Virtual Pages Memory Subsystem L1 Store Buffer DATA PFN [8:0] VFN Offset DTLB DATA PFN [8:0] VFN Offset … …. … … DATA Offset PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN Offset DATA PFN VFN Offset DATA PFN [8:0] VFN Offset DATA PFN VFN Offset DATA PFN [8:0] VFN Offset … …. … … VFN Offset DATA PFN [8:0] VFN Offset DATA PFN

  13. Spoiler: Finding Undocumented Aliasing … Virtual Pages Memory Subsystem 64 pages L1 Store Buffer DATA PFN [8:0] VFN Offset DTLB DATA PFN [8:0] VFN Offset … …. … … DATA Offset PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN Offset DATA PFN VFN Offset DATA PFN [8:0] VFN Offset DATA PFN VFN Offset DATA PFN [8:0] VFN Offset … …. … … VFN Offset DATA PFN [8:0] VFN Offset DATA PFN

  14. Spoiler: Finding Undocumented Aliasing … Virtual Pages Memory Subsystem 64 pages L1 Store Buffer 0 x 4 0 0 F E 1 0 C 0 DATA PFN [8:0] VFN 0C0 0 x 4 0 0 F E 2 0 C 0 Stores DTLB DATA PFN [8:0] VFN 0C0 … … … …. … … 0 x 4 0 1020 0 C 0 DATA 0C0 PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN 0C0 DATA PFN VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN VFN Offset DATA PFN [8:0] VFN 0C0 … …. … … VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN

  15. Spoiler: Finding Undocumented Aliasing … Virtual Pages Memory Subsystem 64 pages L1 Store Buffer 0 x 4 0 0 F E 1 0 C 0 DATA PFN [8:0] VFN 0C0 0 x 4 0 0 F E 2 0 C 0 Stores DTLB DATA PFN [8:0] VFN 0C0 … … … …. … … 0 x 4 0 1 0 2 0 0 C 0 DATA 0C0 PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN 0C0 DATA PFN VFN Offset DATA PFN [8:0] VFN 0C0 0 x 4 F 1 2 3 4 0 C 0 DATA PFN VFN 0C0 Load DATA PFN [8:0] VFN 0C0 … …. … … VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN

  16. Spoiler: Finding Undocumented Aliasing … Virtual Pages Memory Subsystem L1 Store Buffer 0 x 4 0 0 F E 2 0 C 0 DATA PFN [8:0] VFN 0C0 0 x 4 0 0 F E 3 0 C 0 Stores DTLB DATA PFN [8:0] VFN 0C0 … … … …. … … 0 x 4 0 1 0 2 1 0 C 0 DATA 0C0 PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN 0C0 DATA PFN VFN Offset DATA PFN [8:0] VFN 0C0 0 x 4 F 1 2 3 4 0 C 0 DATA PFN VFN 0C0 Load DATA PFN [8:0] VFN 0C0 … …. … … VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN

  17. Spoiler: Finding Undocumented Aliasing … Virtual Pages Memory Subsystem L1 Store Buffer 0 x 4 0 0 F E 3 0 C 0 DATA PFN [8:0] VFN 0C0 0 x 4 0 0 F E 4 0 C 0 Stores DTLB DATA PFN [8:0] VFN 0C0 … … … …. … … 0 x 4 0 1 0 2 2 0 C 0 DATA 0C0 PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN 0C0 DATA PFN VFN Offset DATA PFN [8:0] VFN 0C0 0 x 4 F 1 2 3 4 0 C 0 DATA PFN VFN 0C0 Load DATA PFN [8:0] VFN 0C0 … …. … … VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN

  18. Spoiler: Finding Undocumented Aliasing … Virtual Pages Memory Subsystem L1 Store Buffer 0 x 4 0 0 F E 4 0 C 0 DATA PFN [8:0] VFN 0C0 0 x 4 0 0 F E 5 0 C 0 DTLB DATA PFN [8:0] VFN 0C0 … … … …. … … 0 x 4 0 1 0 2 3 0 C 0 DATA 0C0 PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN 0C0 DATA PFN VFN Offset DATA PFN [8:0] VFN 0C0 0 x 4 F 1 2 3 4 0 C 0 DATA PFN VFN 0C0 DATA PFN [8:0] VFN 0C0 … …. … … VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN

  19. Spoiler: Finding Undocumented Aliasings … Virtual Pages Memory Subsystem L1 Store Buffer 0 x 4 0 0 F E 4 0 C 0 Physical Addresses DATA PFN [8:0] VFN 0C0 0 x 4 0 0 F E 5 0 C 0 DTLB DATA PFN [8:0] VFN 0C0 … … 0 x 6 5 F 3 2 X X 0 C 0 … …. … … 0 x 4 0 1 0 2 3 0 C 0 DATA 0C0 PFN [8:0] VFN Load Buffer 0 x 3 2 A C 2 X X 0 C 0 DATA PFN [8:0] VFN 0C0 DATA PFN VFN Offset DATA PFN [8:0] VFN 0C0 0 x 4 F 1 2 3 4 0 C 0 DATA PFN VFN 0C0 DATA PFN [8:0] VFN 0C0 … …. … … VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN

  20. Spoiler: Finding Undocumented Aliasing … Virtual Pages Memory Subsystem L1 Store Buffer DATA PFN [8:0] VFN 0C0 DTLB DATA PFN [8:0] VFN 0C0 … …. … … DATA 0C0 PFN [8:0] VFN Load Buffer DATA PFN [8:0] VFN 0C0 DATA PFN VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN VFN 0C0 DATA PFN [8:0] VFN 0C0 … …. … … VFN Offset DATA PFN [8:0] VFN 0C0 DATA PFN

  21. Spoiler: Finding Undocumented Aliasing … Virtual Pages 60

  22. Spoiler: Learning on Physical Address Bits Least 12 bits (Virtual Address = Physical Address) Rest of the bits (Virtual != Physical) MemJam L1 Cache Attacks L2/L3 Cache Attacks 61

  23. Spoiler: Learning on Physical Address Bits Least 12 bits (Virtual Address = Physical Address) VFN MemJam L1 Cache Attacks PFN MemJam 62 L2/L3 Cache Attacks

  24. Spoiler: Learning on Physical Address Bits Least 12 bits (Virtual Address = Physical Address) VFN MemJam L1 Cache Attacks Pime+Probe on Cache, Eviction Sets, Rowhammer PFN MemJam 63 L2/L3 Cache Attacks

  25. Spoiler: Learning on Physical Address Bits Least 12 bits (Virtual Address = Physical Address) VFN MemJam L1 Cache Attacks Pime+Probe on Cache, Eviction Sets, Rowhammer PFN Spoiler MemJam 64 L2/L3 Cache Attacks

  26. 65 Spoiler – JavaScript Eviction Sets

  27. 66 Spoiler - • Row Buffer Conflict Rowhammer • Single-sided Rowhammer

  28. 67 Spoiler - • Detecting Contiguous Memory • Double-sided Rowhammer Rowhammer

  29. 2018: Meltdown Attack? 68

  30. 2018: Meltdown Attack? Virtual Address Space User Space CPU Registers Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line 69

  31. 2018: Meltdown Attack? Virtual Address Space Oracle User Space CPU Registers Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line

  32. 2018: Meltdown Attack? Virtual Address Space Fault Oracle User Space CPU Registers Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line

  33. 2018: Meltdown Attack? Virtual Address Space Fault Oracle User Space P CPU Registers Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line

  34. 2018: Meltdown Attack? Virtual Address Space Fault Oracle User Space P CPU Registers Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line

  35. 2018: Meltdown Attack? Virtual Address Space Oracle User Space CPU Registers Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line

  36. 2018: Meltdown Attack? Virtual Address Space Oracle User Space CPU Registers F+R Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line

  37. 2018: Meltdown Attack? Virtual Address Space Oracle User Space CPU Registers F+R Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line

  38. 2018: Meltdown Attack? Virtual Address Space Oracle User Space CPU Registers F+R Kernel Space 0xf…81a0123 P A S S W O R D 256 different CPU Cache Line

  39. 2018: Meltdown Attack? Virtual Address Space Oracle User Space CPU Registers Kernel Space ‘ P ’ = 0x50 P A S S W O R D 256 different CPU Cache Line

  40. Microarchitecture Data Sampling (MDS) • Meltdown is fixed but you can still leak on the fix hardware. • Which part of the CPU leak the data?! • Why does it leak? 79

  41. CPU Memory Subsystem – Challenges? Memory Subsystem EUs Store Buffer ROB DATA PFN [8:0] VFN Offset Store DATA PFN [8:0] VFN Offset Allocation … …. … … Queue DATA PFN [8:0] VFN Offset Load L1 Fill Buffer stor $$, (add_A) Load Buffer Load stor ##, (add_B) Scheduler DATA PFN VFN Offset load (add_C), CX ALU DATA PFN VFN Offset DTLB … …. … … add CX, BX DATA PFN VFN Offset ALU L2 L3 Front End Back End DRAM 80 80

  42. Memory Canonical Access #GP Virtual Address VFN Offset 81

  43. Y Memory Canonical TLB Access PMH #GP Virtual Address VFN Offset PTE P Physical Page Number RW US … A … … 82

  44. Y Y Memory Canonical TLB Perm. Access PMH #GP Virtual Address VFN Offset PTE P Physical Page Number RW US … A … … 83

  45. Y Y Y Memory Canonical TLB Perm. Present Access PMH #GP #PF Virtual Address VFN Offset PTE Physical Page Number P RW US … A … … 84

  46. Y Y Y Y Memory Canonical TLB Perm. Present Accessed Access Set A PMH #GP #PF Bit Virtual Address VFN Offset PTE Physical Page Number P RW US … A … … 85

  47. Y Y Y Y Memory Canonical TLB Perm. Present Accessed Access Set A PMH Y #GP #PF Bit Aligned Vector #GP Virtual Address VFN Offset PTE Physical Page Number P RW US … A … … 86

  48. Y Y Y Y Memory Canonical TLB Perm. Present Accessed Access Set A PMH Y #GP #PF Bit Y Cache Aligned Aligned Vector Split #GP Cache Virtual Address VFN Offset PTE Physical Page Number P RW US … A … … 87

  49. Y Y Y Y Memory Canonical TLB Perm. Present Accessed Access Set A PMH Y #GP #PF Bit Y Y Cache Aligned Cached Aligned Vector Cache Miss Split #GP Handler Cache Virtual Address VFN Offset PTE Physical Page Number P RW US … A … … 88

  50. Y Y Y Y Memory Canonical TLB Perm. Present Accessed Access Set A PMH Y #GP #PF Bit Y Y Y False Cache Aligned Cached Store Dep. Aligned Vector Hazard Cache Miss Split #GP Handler Cache Recovery Virtual Address VFN Offset PTE Physical Page Number P RW US … A … … 89

  51. Y Y Y Y Memory Canonical TLB Perm. Present Accessed Access Set A PMH Y #GP #PF Bit Y Y Y Y TSX False Cache Aligned Cached Failure Store Dep. Aligned Vector #RTM Hazard Cache Miss Split #GP Handler Cache Recovery Virtual Address VFN Offset PTE Physical Page Number P RW US … A … … 90

  52. Fault VS. Assist Dilemma • Microcode Assists: The CPU executes an internal event handler to service complex instructions/operations • Fault (#GP , #PF , #RTM): An assist that run a software-based callback

  53. CPU Memory Subsystem – Hazard Recovery Memory Subsystem EUs Store Buffer ROB DATA PFN [8:0] VFN Offset Store DATA PFN [8:0] VFN Offset Allocation … …. … … Queue DATA PFN [8:0] VFN Offset Load L1 Fill Buffer stor $$, (addr_B) Load Buffer Load load (addr_A), AX Scheduler DATA PFN VFN Offset ALU DATA PFN VFN Offset DTLB … …. … … DATA PFN VFN Offset ALU L2 L3 Front End Back End DRAM 92 92

  54. CPU Memory Subsystem – Hazard Recovery Memory Subsystem EUs Store Buffer ROB DATA PFN [8:0] VFN Offset Store DATA PFN [8:0] VFN Offset Allocation … …. … … Queue DATA PFN [8:0] VFN Offset Load L1 Fill Buffer stor $$, (addr_B) Load Buffer Load load (addr_A), AX Scheduler DATA PFN VFN Offset ALU DATA PFN VFN Offset DTLB … …. … … DATA PFN VFN Offset ALU L2 L3 Front End Back End DRAM 93 93

  55. CPU Memory Subsystem – Hazard Recovery Memory Subsystem EUs Store Buffer ROB DATA PFN [8:0] VFN Offset Store DATA PFN [8:0] VFN Offset Allocation … …. … … Queue DATA PFN [8:0] VFN Offset Load L1 Fill Buffer stor $$, (addr_B) Load Buffer Load load (addr_A), AX Scheduler DATA PFN VFN Offset ALU DATA PFN VFN Offset DTLB … …. … … DATA PFN VFN Offset ALU L2 L3 Front End Back End DRAM 94 94

  56. CPU Memory Subsystem – Hazard Recovery Memory Subsystem EUs Store Buffer ROB DATA PFN [8:0] VFN Offset Store DATA PFN [8:0] VFN Offset Allocation … …. … … Queue DATA PFN [8:0] VFN Offset Load L1 Fill Buffer stor $$, (addr_B) Load Buffer Load load (addr_A), AX Scheduler DATA PFN VFN Offset ALU DATA PFN VFN Offset DTLB … …. … … DATA PFN VFN Offset ALU L2 L3 Front End Back End DRAM 95 95

  57. MDS Attacks (ZombieLoad , RIDL, Fallout, …) • The CPU must flush the pipeline before executing an assist. • Upon an Exception/Fault/Assist on a Load, Intel CPUs: • Execute the load until the last stage. • Flush the pipeline at the retirement stage (Cheap Recovery Logic). • Continue the load with some data to reach the retirement stage. • Which data? 96

  58. CPU Memory Subsystem – Leaky Buffers Fallout Memory Subsystem Store Buffer DATA PFN [8:0] VFN Offset DATA PFN [8:0] VFN Offset … …. … … DATA PFN [8:0] VFN Offset L1 Fill Buffer Load Buffer DATA PFN VFN Offset DATA PFN VFN Offset DTLB … …. … … DATA PFN VFN Offset ZombieLoad L2 L3 DRAM 97 97

  59. CPU Memory Subsystem – Leaky Buffers Fallout Memory Subsystem Store Buffer DATA PFN [8:0] VFN Offset DATA PFN [8:0] VFN Offset … …. … … DATA PFN [8:0] VFN Offset L1 Fill Buffer Load Buffer DATA PFN VFN Offset DATA PFN VFN Offset DTLB … …. … … DATA PFN VFN Offset ZombieLoad L2 L3 DRAM 98 98

  60. MDS Attacks (ZombieLoad , RIDL, Fallout, …) • The CPU must flush the pipeline before executing an assist. • Upon an Exception/Fault/Assist on a Load, Intel CPUs: • Execute the load until the last stage. • Flush the pipeline at the retirement stage (Cheap Recovery Logic). • Continue the load with some data to reach the retirement stage. • Which data? (Fill buffer, Store Buffer, Load Buffer) • Which one will be leaked first? 99

  61. ZombieLoad Attack Core L1D Cache L2 L3 DRAM 100

Recommend


More recommend