HeteroOS - Heterogeneous Memory Management in Datacenter Sudarsun - PowerPoint PPT Presentation

HeteroOS - Heterogeneous Memory Management in Datacenter Sudarsun Kannan (University of Wisconsin-Madison), Ada Gavrilovska (Georgia Tech), Vishal Gupta (VMWare), Karsten Schwan (Gerogia Tech)

Systems and OS for Decades Application • OSes had to make simple binary data placement malloc() Cache write()/read() OS Virtual File Memory System Memory Controller SATA • Keep volatile hot data in memory • Move cold or persistent data to storage DRAM Hard Drive 1 Introduction Motivation Analysis Design Conclusion 1

Data Explosion Across Environments 2 Introduction Motivation Analysis Design Conclusion 2

Changing Memory Hierarchy Application More CPUs and accelerators Cache OS Virtual Memory 3D-DRAM Controller Memory heterogeneity NVM DRAM 3 Introduction Motivation Analysis Design Conclusion 3

Changing Memory Hierarchy 3D-DRAM NVM DRAM 1 x Bandwidth 2x 0.25x 0.75 x 1 x Latency 4x Capacity Limited Very low High Persistence? No No Yes Programmable No No No $/GB 1 x ~4x 0.3x Significant difference in latency, bandwidth, and capacity across devices 4 Introduction Motivation Analysis Design Conclusion

OS for Heterogeneous Memory Systems To truly embrace memory heterogeneity: 1. OSes must seamlessly scale heap capacity across heterogeneous memory Our prior work – pVM (persistent virtual memory) EuroSys ‘16 • Virtual OS Memory Simply add more memory like NVM and 3D-DRAM applications will automatically scale DRAM NVM 5 Introduction Motivation Analysis Design Conclusion 5

OS for Heterogeneous Memory Systems To truly embrace memory heterogeneity: 1. OSes must seamlessly scale heap capacity across heterogeneous memory Our prior work – pVM (persistent virtual memory) EuroSys ’16 • 2. Efficiently place data across memories with different characteristics HeteroOS • Virtual OS Memory Virtual memory must provide optimal memory 3D-DRAM placement DRAM NVM 6 Introduction Motivation Analysis Design Conclusion 6

Complex Datacenter System Stack APP 1 APP 2 APP 1 Complex multi-layered Guest OS Guest OS software stack Hypervisors do not expose Hypervisor even NUMA info to guest OS Heterogeneity increases complexity Where to manage heterogeneity - APP, the OS, or the hypervisor (VMM)? 7 Introduction Motivation Analysis Design Conclusion

State-of-the-art Systems • Application-level data placement across memory [X-mem, EuroSys ‘16] ü Works well for heap-intensive applications with exclusive memory access ü Lacks holistic view of system and in-effective for non-exclusive access ü Cannot place OS-level pages OS or Hypervisor-level hotness tracking and page migration [HeteroVisor, VEE ‘15] • ü Application-agnostic data placement ü Reactive technique – significant data tracking and movement overhead 8 Introduction Motivation Analysis Design Conclusion

HeteroOS Key Idea • Designed for virtualized datacenter systems Expose heterogeneity to guest-OS • Capture demand for different memory page types at the guest-OS • Page types include heap, IO cache, network buffers ü Directly allocate/place pages on faster memory based on demand for page type • Migration only if direct OS allocation fails • 9 Introduction Motivation Analysis Design Conclusion

Talk Outline Our study • ü Analysis of memory heterogeneity impact on applications ü Page migration cost analysis • HeteroOS guest-OS management OS design for direct memory placement and management ü Coordinated management • Coordinate management between guest-OS with hypervisor ü • Conclusion 10

Applications Compute + memory + IO CPU + Memory Metis X-Stream Network Memory + Network 11 Introduction Motivation Analysis Design Conclusion

Emulating Heterogeneous Memory • Prior techniques used simulators or delaying all memory instruction - Infeasible for long running application • Two memory sockets used to represent fast (FastMem) and slow (SlowMem) memory • SlowMem node thermal throttled to reduce bandwidth reduced by 9x Processors Thermal throttled memory Node 0 (FastMem) Node 1 (SlowMem) 12 Introduction Motivation Analysis Design Conclusion

Memory Latency and Bandwidth Impact? 4 Graphchi Slowdown relative to using FastMem-only Bandwidth sensitivity Xstream 3 Redis Taller the bars, more app. Nginx slowdown Latency 2 sensitivity 1 L2:B2 L3:B2 L5:B5 L5:B9 ü X-axis - factor by which latency is increased and bandwidth is reduced E.g., L2:B2 indicates 2x increase in latency, 2x reduction in bandwidth • ü Y-axis - application slowdown relative to using only FastMem 13 13 Introduction Motivation Analysis Design Conclusion

Memory Latency and Bandwidth Impact? 4 Graphchi Slowdown relative to using FastMem-only Bandwidth sensitivity Xstream 3 Redis Taller the bars, more app. Nginx slowdown Latency 2 sensitivity 1 L2:B2 L3:B2 L5:B5 L5:B9 Heterogeneity impact significant on applications Current OS-level NUMA mechanisms not sufficient 14 14 Introduction Motivation Analysis Design Conclusion

Migration-based Techniques Memory heterogeneity is hidden from guest [Intel Memory Drive Technology] • • Initially, all pages are placed in slower memory [HeteroVisor:Gupta:VEE15] Hypervisor identifies hot and cold pages passively • Moves hot pages to faster memory and cold pages to slower memory • Pages Cold pages Hot pages FastMem SlowMem • Management dependent on page migration across memories Page migrations cause significant performance overhead 15 Introduction Motivation Analysis Design Conclusion

Understanding Page Migration Overheads Page migration —> hotness tracking + page movement Hot page scan in software requires • Traversing page table and setting page reference bit • Invalidating TLB to force CPU to access page table • Clearing bits and traversing page table to count references • Repeating until hotness threshold reached Page movement requires Allocating memory pages at destination • Copying pages and invalidating old TLB entries • Releasing old memory pages • 16 Introduction Motivation Analysis Design Conclusion 16

Understanding Page Migration Overheads Graphchi 75 Rutime overhead (%) Migration Hotness tracking 60 45 30 15 0 100ms, 32K 300ms, 32K 500ms, 32K Sampling frequency, pages sampled Hotness tracking more expensive than migration! Expensive data migrations can impact seamless capacity scaling benefits 17 17 Introduction Motivation Analysis Design Conclusion 17

HeteroOS Design goals Application transparent heterogeneity management • Reduce page migrations and maximize application performance • Design steps Step 1 (HeteroOS-guest) • ü Directly allocate and manage heterogeneity at the guest-OS Step 2 (HeteroOS-coordinated) • ü Guest-OS coordinates with hypervisor for hotness-tracking and migration 18 18 Introduction Motivation Analysis Design Conclusion

Guest-OS Management Expose heterogeneity to guest-OS via. NUMA abstraction • Perform direct allocation to FastMem and reduce page migration • Unmodified applications Guest OS FastMem Manager SlowMem Manager Analysis Find inactive FastMem pages Alloc FastMem Failed Alloc SlowMem Hypervisor On-Demand Alloc Driver BackEnd 19 19 Introduction Motivation Analysis Design Conclusion

How Applications Use Memory? • Applications and OS allocate different types of pages Heap I/O cache NW-buff Slab 100% 75% #. Of Pages 50% 25% 0% Graphchi Metis Redis Xstream • Placement of OS-level pages critical in addition to application’s heap pages Current OSes always prioritize heap pages to faster memory 20 Introduction Motivation Analysis Design Conclusion 20

Heterogeneity-aware OS Placement Traditional OS virtual memory • Heap pages always prioritized even when demand for I/O pages is high Heap page I/O page I/O page I/O page Heap page I/O page Move to SlowMem Heap page I/O page I/O page I/O page Heap page even when I/O page demand is high FastMem (DRAM) Heterogeneity-aware OS virtual memory • All page types prioritized based on the demand Heap page Heap page I/O page I/O page I/O page I/O page I/O page Move to SlowMem memory Heap page I/O page I/O page I/O page I/O page 21 FastMem (DRAM) Introduction Motivation Analysis Design Conclusion 21

Heterogeneity-aware OS Placement • Principle: OS knows how applications and subsystems use memory • OS knows demand for heap, IO cache, network buffer pages - Demand represents #. of pages of a page type requested in an epoch • Directly allocate to ”right memory” based on current demand - Use migration selectively when direct placement is not possible 22 22 Introduction Motivation Analysis Design Conclusion 22

Guest-OS Management Impact Gains(%) relative to 250 Migration-only HeteroOS-guest FastMem-only 200 SlowMem-only Taller bars show 150 better perf. 100 50 0 1/4 1/8 1/4 1/8 1/4 1/8 1/4 1/8 Graphchi X-Stream Redis Metis FastMem to SlowMem capacity ratio X-axis shows FastMem to SlowMem capacity ratio • ü 1/4 -> FastMem 8GB, SlowMem 32GB, 1/8 -> FastMem 4GB, SlowMem 32GB Y-axis show the gains (%) relative to using only SlowMem • 23 23 Introduction Motivation Analysis Design Conclusion