PageSeer : Using Page Walks to Trigger Page Swaps in Hybrid Memory - PowerPoint PPT Presentation

PageSeer : Using Page Walks to Trigger Page Swaps in Hybrid Memory Systems Apostolos Kokolis , Dimitrios Skarlatos and Josep Torrellas University of Illinois at Urbana-Champaign http://iacoma.cs.uiuc.edu HPCA 2019, Feb 2019

Introduction • Data intensive applications demand memory capacity • DRAM can no longer provide the capacity needed • Non-volatile memory (NVM) technologies + higher density Cannot replace DRAM entirely - slower • Solution: Hybrid Memory Systems MC DRAM NVM PageSeer: Using Page Walks to Trigger Page Swaps in 2 Hybrid Memory Systems

Managing a Hybrid Memory System 1. DRAM is a cache for the NVM MC 2. DRAM and NVM share a flat address space Capacity loss DRAM More capacity & BW NVM Need to swap memory between NVM and DRAM PageSeer: Using Page Walks to Trigger Page Swaps in 3 Hybrid Memory Systems

Challenges of HW Managed Hybrid Schemes Memory Request for Page D HW needs to 1. Swap pages • Decide which pages to swap MC 2. Track page activity D 3 • Accurately identify “hot” pages to swap A B C D F 3. Record the page remappings DRAM NVM PageSeer: Using Page Walks to Trigger Page Swaps in 4 Hybrid Memory Systems

Challenges of HW Managed Hybrid Schemes Memory Request for Page C HW needs to 1. Swap pages • Decide which pages to swap MC A C 2. Track page activity • Accurately identify “hot” pages to swap C B A D F 3. Record the page remappings DRAM NVM PageSeer: Using Page Walks to Trigger Page Swaps in 5 Hybrid Memory Systems

Challenges of HW Managed Hybrid Schemes HW needs to 1. Swap pages • Decide which pages to swap MC 2. Track page activity • Accurately identify “hot” pages to swap C B A D F 3. Record the page remappings DRAM 4. Store meta-data information NVM • Remappings • Page activity PageSeer: Using Page Walks to Trigger Page Swaps in 6 Hybrid Memory Systems

Prior Work • Focused on identifying “hot” pages to swap: 1. Use access counters per memory segment v Conservative swapping à Miss opportunities 2. Start a swap on a first access to a memory segment v Aggressive swapping à Unnecessary traffic 3. Predict future memory accesses v Hard to predict NVM DRAM Segment ID Counter Read ≥ Threshold A 5 Write Swap Buffer Write B 4 Read Swap Buffer C 2 PageSeer: Using Page Walks to Trigger Page Swaps in 7 Hybrid Memory Systems

PageSeer Motivation • Swapping is a costly operation and takes time • Need accuracy in predicting future memory accesses and swapping • Need to start the swaps as early as possible v How to predict future memory accesses? NVM DRAM Write Swap Buffer Read Read Write Swap Buffer PageSeer: Using Page Walks to Trigger Page Swaps in 8 Hybrid Memory Systems

PageSeer Motivation • Memory requests require two steps • Translation from VA to PA • Data access Fill Fill CPU TLB PW Memory PTE L2 LLC Caches TLB Request Translation Timeline Replay CPU TLB L2 LLC L1 Memory Request Data Access Timeline PageSeer: Using Page Walks to Trigger Page Swaps in 9 Hybrid Memory Systems

PageSeer Motivation • Memory intensive applications cause TLB misses • TLB miss results in a page walk • If the page is cold • Page translation Miss in caches • Actual page data Fill Fill CPU Page TLB L2 LLC Memory Caches TLB Request Walk (PW) Translation Timeline Replay TLB L1 L2 LLC Memory Request Data Access Timeline PageSeer: Using Page Walks to Trigger Page Swaps in 10 Hybrid Memory Systems

PageSeer Motivation Insight: Page Walks give information about future accesses to a page earlier At this point we know the page that will be accessed Fill Fill CPU Page TLB L2 LLC Memory Caches TLB Request Walk (PW) Translation Timeline Replay TLB L1 L2 LLC Memory Request Data Access Timeline PageSeer: Using Page Walks to Trigger Page Swaps in 11 Hybrid Memory Systems

Contribution: PageSeer • Uses pages walks to trigger page swaps in HW-only hybrid memory systems • Uses a page-correlation mechanism to prefetch page swaps • Includes HW structures to recognize and move “hot” pages to DRAM • Improves performance by 19% and reduces Average Main Memory Access Time (AMMAT) by 29% over state-of-the-art PageSeer: Using Page Walks to Trigger Page Swaps in 12 Hybrid Memory Systems

Background – Page Walk Virtual Address 47 - 39 38 - 30 29 - 21 20 - 12 11 - 0 c + PTE A 4-KB Page A c c PMD A Physical + + Address c + PUD A c + PGD A CR3 PageSeer: Using Page Walks to Trigger Page Swaps in 13 Hybrid Memory Systems

PageSeer Design Processor ¡Chip ¡ • Communication link between the MMU and the Hybrid Core ¡ Core ¡ Memory Controller L1 ¡TLB ¡ L1 ¡TLB ¡ L1 ¡ L1 ¡ L2 ¡TLB ¡ L2 ¡ L2 ¡TLB ¡ L2 ¡ • When a Page Walk reaches the MMU ¡ MMU ¡ PTE level Shared ¡L3 ¡ • Conventional: MMU sends a memory request to the Hybrid ¡Memory ¡ caches Controller ¡(HMC) ¡ • PageSeer: in addition MMU Path ¡to ¡ Memories ¡ sends it to the HMC Crossbar ¡interconnect ¡ Logic ¡Layer ¡ ¡ Logic ¡Layer ¡ ¡ v Inform the HMC about DRAM ¡ NVM ¡ forthcoming memory accesses PageSeer: Using Page Walks to Trigger Page Swaps in 14 Hybrid Memory Systems

PageSeer Overview TLB Goal: give time to the DRAM L2 HMC for swaps and to prepare its HW structs MMU Shared L3 NVM HMC Translation Timeline Data Access Timeline PageSeer: Using Page Walks to Trigger Page Swaps in 15 Hybrid Memory Systems

PageSeer Overview 1. MMU signals the HMC TLB DRAM L2 2 2. HMC finds physical A 3 PTE 1 page number MMU 3. HMC prepares its HW 4 structures C 4. Starts swapping Shared L3 NVM HMC Fill Fill CPU TLB PW PTE L2 LLC HMC Memory Caches TLB Request Translation Timeline Replay TLB L1 L2 LLC HMC Memory Request Data Access Timeline PageSeer: Using Page Walks to Trigger Page Swaps in 16 Hybrid Memory Systems

PageSeer Overview 1. MMU signals the HMC TLB DRAM L2 2 2. HMC finds physical C 3 PTE 1 page number MMU 3. HMC prepares its HW 4 structures A 4. Starts swapping Shared L3 NVM HMC Fill Fill CPU TLB PW PTE L2 LLC HMC Memory Caches TLB Request Benefits: • If the MMU request reaches HMC à PTE is already prefetched Translation Timeline • When the request is replayed à page swap to DRAM has already started Replay TLB L1 L2 LLC HMC Memory Request Data Access Timeline PageSeer: Using Page Walks to Trigger Page Swaps in 17 Hybrid Memory Systems

Hybrid Memory Controller Hybrid ¡Memory ¡Controller ¡ Page Remapping Table (PRT) (HMC) ¡ MMU ¡ Memory ¡request ¡ signal ¡ • Records remappings between DRAM-NVM pages • On the critical path PRT ¡ • Every memory request checks the PRT Path ¡to ¡ Memories ¡ PageSeer: Using Page Walks to Trigger Page Swaps in 18 Hybrid Memory Systems

Hybrid Memory Controller Hybrid ¡Memory ¡Controller ¡ Page Remapping Table (PRT) (HMC) ¡ MMU ¡ Memory ¡request ¡ signal ¡ • Hit rate and lookup time is important => Cache some entries • Swap at page granularity PRT ¡ NVM DRAM NVM DRAM NVM DRAM Page Pages PPN PPN PPN PPN s Set 0 Set 1 Set 2 Set 3 Path ¡to ¡ Memories ¡ PageSeer: Using Page Walks to Trigger Page Swaps in 19 Hybrid Memory Systems

Hybrid Memory Controller Hybrid ¡Memory ¡Controller ¡ Hot Page Tables (HPTs) (HMC) ¡ MMU ¡ Memory ¡request ¡ signal ¡ • 1 for DRAM – 1 for NVM • Track hot pages • DRAM HPT PRT ¡ • Pages that should remain in DRAM HPTs ¡ • NVM HPT • Candidate pages to swap to DRAM Page Number Counter Path ¡to ¡ Memories ¡ PageSeer: Using Page Walks to Trigger Page Swaps in 20 Hybrid Memory Systems

Hybrid Memory Controller Hybrid ¡Memory ¡Controller ¡ Page Correlation Table (PCT) (HMC) ¡ MMU ¡ Memory ¡request ¡ signal ¡ • Keeps historic data for page accesses • Captures the correlation PRT ¡ PCT ¡ between pages à prefetch HPTs ¡ PCT entry PPN Counter Next PPN Next counter Path ¡to ¡ Memories ¡ PageSeer: Using Page Walks to Trigger Page Swaps in 21 Hybrid Memory Systems

Hybrid Memory Controller Hybrid ¡Memory ¡Controller ¡ MMU Driver (HMC) ¡ MMU ¡ Memory ¡request ¡ signal ¡ • Receives the MMU signal and checks for prefetch swaps • Receives requests for PTEs PRT ¡ PCT ¡ • Caches recently fetched PTEs HPTs ¡ MMU ¡Driver ¡ ¡ Path ¡to ¡ Memories ¡ PageSeer: Using Page Walks to Trigger Page Swaps in 22 Hybrid Memory Systems

Hybrid Memory Controller Hybrid ¡Memory ¡Controller ¡ Swap Driver (HMC) ¡ MMU ¡ Memory ¡request ¡ signal ¡ • Initiates page swaps triggered from • NVM HPT • PCT PRT ¡ PCT ¡ • Checks if an access is for a page that is being swapped HPTs ¡ MMU ¡Driver ¡ ¡ Swap ¡Driver ¡ Path ¡to ¡ Memories ¡ PageSeer: Using Page Walks to Trigger Page Swaps in 23 Hybrid Memory Systems