Memory Management Marco Serafini COMPSCI 532 Lecture 12 - PowerPoint PPT Presentation

Memory Management Marco Serafini COMPSCI 532 Lecture 12

Announcements • Project 2 published • Info on website • GitHub repo on Piazza • Deadline: November 7 • Next class: hands-on session • Threads, processes, sockets, … • Bring your laptop with Java SDK installed! • AWS credits: invitation published on Piazza 2

Virtual Memory

Virtual Memory in Everyday Life • Tabs store webpages in main memory (RAM) • Many tabs à Data written to disk • Switch to old tabs à Disk reads • The whole system becomes slow 4 4

Programmer’s View of Memory 0000…0000 Global vars x in stack Stack void foo ( int i) { int x = i; int * z = new int ; *z in heap free space *z = i; } Heap FFFF….FFFF 5

Many Abstractions… 0000…0000 • Our code has dedicated memory Global vars • As much memory as addressable • 2 64 -1 = 1.8 * 10 19 addresses Stack • One address, one 64-bit word • ~ 1.47 * 10 20 bytes • > 10 5 petabytes (or 100 exabytes) free space • Neither assumption holds! Virtual memory implements these Heap abstractions FFFF….FFFF 6

Memory Hierarchy and Caching General pattern Fast memory (small, expensive) Cache recently Evict inactive data read data Slow memory (large, cheap) 7

Memory Hierarchy and Caching General pattern Virtual Memory Fast memory Fast (small, expensive) Physical Memory (RAM) Cache recently Evict inactive data read data Slow memory Slow (large, cheap) Disk 8

Virtual Memory 3 Virtual Address Space access (one separate per process) physical address 0 actual data 2 Address 1 translation virtual à OR … Each byte physical has a 64 bit 3 … 1 VIRTUAL CPU executes access address. … … store/load actual data disk operation location , … … on virtual cache in address main 2 64 -1 memory, return Data layout as seen by the program ( virtual ) 9

Extended Memory Hierarchy • Hierarchy is getting deeper • Non-volatile memory • Solid State Drives (SSD) • We will consider a simplified Caches view, as most OSs do Main memory • Swap partition on disk •File or separated logical drive Swap Disk 10

Advantages of Virtual Memory • For programmers • Can use more space than main memory… • … with limited performance overhead • ... without having to explicitly code disk spilling • For the Operating System • Isolation: Each process has its own Virtual Address Space • Easier to enforce memory protection 11

Virtual Memory v1 Basic functionalities: Address translation + caching

Granularity • Question: Which level of granularity is better? 1. Group contiguous virtual addresses in a block… • … and move the block to and from disk at once 2. Store each virtual address separately… • … and move single bytes back and forth from disk • Correct Answer: 1 • A virtual memory block is called page • Block of contiguous memory addresses • Treated as an “atom” by virtual memory • Usually a few KB (e.g., 4KB in my laptop) 13

Address Translation 52 bit 12 bit virtual page address Memory page P offset ( O ) P O address A V … 0 page number page offset … … 1 2 Page table A+O byte for V address + Page table (P * size(row)) … (one separate per process) Page address Control bits 2 12 -1 Page table … 2 12 = 4096 bytes = 4 KB address … A dirty? disk? .. 2 main memory accesses (if page in main memory) 14

Granularity: Advantages of Paging • A byte-granularity translation table would be huge • The page table is smaller • Sequential disk I/O is faster than random access • We read one full page at a time • Access locality • A process accessing an address is likely to access a nearby address soon thereafter 15

Caching: Fetching Pages from Disk • “Residency” control bit says if page is in memory • If page is not in memory: page fault • How virtual memory deals with page faults: 1. Get the disk position from the page table 2. Invoke the OS to fetch page from disk (swap file) 3. Store the page in memory, update page table 4. Return memory address 16

Caching: Evicting Pages to Disk • If main memory is full, remove some page (eviction) • Update the residency bit and address in page table • “Modified” control bit: if modified, write evicted page to disk • Which pages to evict? Least Recently Used (LRU) • A simple implementation 1. Keep an “accessed” control bit 2. Periodically reset it 3. Pages that were not accessed can be removed • Actually more complex than that (requires hw support) 17

Questions • Address translation occurs infrequently • True / false? • Address translation is better done in software • True / false? • The page table should be stored inside the CPU • True / false? 18

Virtual Memory v2 Speeding up translation

Hardware Support • Software implementation (OS only) is too slow • Solution: some VM logic implemented in hardware • Memory Management Unit (MMU) • Translation Lookaside Buffer (TLB) Disk controller Main memory CPU CPU package Page table CPU accesses virtual addresses cache TLB MMU MMU implements address translation MMU accesses physical addresses (directly) or disk addresses (with the OS) Bus 20

Translation Lookaside Buffer • Cache of page table entries • Entries: <Page number, Page address, Control bits> • Question: Can we access it using offsets like the page table? • TLB is a fully associative cache • Need to read all elements to find the page table • This is done in parallel in hardware 21

Question: Page Table in Memory? 52 bit 12 bit virtual page address Memory page P offset ( O ) P O address A V … 0 page number page offset … … 1 2 Page table A+O byte for V address + Page table (P * size(row)) … (one separate per process) Page address Control bits 2 12 -1 Page table … 2 12 = 4096 bytes = 4 KB address … A dirty? disk? .. 22

Virtual Memory v3 Dealing with large page tables

Page Table in Memory • Page tables can be very large • 32 bit architecture: A few (~ 4) megabytes • 64 bit architecture: (2 52 – 1 = 4.5 * 10 15 entries) * a few bytes • How to avoid keeping everything in memory • Hierarchical page tables • Indirection with intermediate page tables • Partition page tables in chunks • Not all page tables chunks in memory • Inverted page tables • Hash map • Need additional data structures to track pages on disk 24

Hierarchical (2-Level) Page Table 52 bit 12 bit virtual Page table chunk P1 P2 O address (separate per process) V page number page offset Page address Control bits Page directory A dirty? disk? .. (separate per process) 1 Page table address Page directory 2 3 PC + (P2 * address + A + O size(row)) (P1 * size(row)) PC (page could be (chunk could on memory or be on disk ) disk) 25

Inverted Page Table 52 bit 12 bit virtual Memory page P offset (Y) P O address V 0 page number page offset … … 2 A+O Byte for V Inverted page table (separate per process) … 1 Page table Page address Control bits 2 12 -1 address + (hash (P) mod N) 2 12 = 4096 bytes = 4 KB A dirty? disk? .. must consider hash collisions keeps only N elements needs additional data structures to track pages on disk (not depicted) 26

Dynamic Page Sizing • Some architectures support multiple sizes • Examples: 4 KB, 2 MB, 1 GB, … • Linux: Huge Pages • Windows: Large Pages • FreeBSD: SuperPages 27

Virtual Memory v4 Memory Protection

Memory Protection • Page table / TLB store access rights • Read-Write, Read-only, Execute only • Which processes can access page (for sharing) • Only OS can change tables • I.e. only kernel-mode operations 29

Memory-Mapped Files

Memory-Mapped Files ( mmap ) • Goal: Use VM to cache your application file to memory • OS lets you map a file to a virtual memory space • Same VM mechanisms, but on your file (not swap file) • You access the file as if it was in memory • OS will transfer data from file to memory and back • Problem: you never know when data is written to disk… • Reads by multiple threads à excessive disk I/O (potentially) • Upon a crash, data might be lost (if page not yet written to disk) • ... unless you flush a page, which forces a write to disk • So use with care! 31

Languages with Managed Memory 32

Traditional Languages (C/C++) • Basic heap management • Must manually allocate/deallocate regions ( malloc / free ) • Must manage pointers • Problem: difficult to tell if a region is still in use • Memory leaks: memory is not deallocated • Incorrectly reclaim memory for objects that are still used • Hard to debug: memory access may return “junk” • Advantage: control 33 33

Memory-Managed Languages (Java) • “No pointers”: object variables are pointers • Garbage collection deallocates memory for you • Advantage: never read “junk” • Disadvantage: performance bottlenecks • Overhead can be mitigated by understanding how GC works! • Goals of GC • Maximize memory utilization • Minimize memory fragmentation • Minimize GC overhead 34 34