Operating Systems WT 2019/20 Memory Management Shared Memory - PowerPoint PPT Presentation

Operating Systems WT 2019/20 Memory Management

Shared Memory Process 1 virtual memory ● most modern OS’s provide a way for processes to share Physical memory memory compiler ● e.g. code pages for image executables Process 2 virtual memory ● Processes can create shared memory sections manually Operating Systems 2

Shared Memory Note: the shared Process A Process B ● region may be 00000000 mapped at difgerent addresses in the User difgerent processes User accessible accessible Pointers! ● v.a.s. v.a.s. 7FFFFFFF Physical Operating Systems 3 Memory

Shared Memory HANDLE OpenFileMapping (HANDLE hFile, Open an existing mapping ● DWORD dwDesiredAccess, object BOOL bInheritHandle, LPCTSTR lpName ); Name comes from ● previous CreateFileMapping() call First process creates mapping, subsequent processes open mapping ● dwDesiredAccess: same as fdwProtect ● lpName: name created with CreateFileMapping() ● CloseHandle() destroys mapping handles ● Operating Systems 4

UNIX – Shared Memory (recap) ● Processes can share memory explicitly int shm_open( const char *name, int ofmag, mode_t mode) ● Shared memory segments are a named resource ● Represented within process resources through fjle descriptor ● Can be mapped to process address space using mmap ● Access has to be synchronized (critical sections) Operating Systems 5

Prototype Page Table Entries (PTEs) Software structure to manage page table redundancy for potentially shared ● pages Array of prototype PTEs is created as part of shared memory object ● Provide reference count for shared pages ● Shared page valid: ● process & prototype PTE point to physical page ● Page invalidated: ● process PTE points to prototype PTE ● Prototype PTE describes 5 states for shared page: ● Active/valid, Transition, Demand zero, Page fjle, Mapped fjle ● Layer between page table and page frame number database ● Operating Systems 6

Prototype PTEs – the bigger picture Two virtual pages in a mapped view ● First page is valid; 2nd page is invalid and in page fjle ● Prototype PTE contains exact location ● Process PTE points to prototype PTE ● PFN n Segment PFN Valid PFN n structure Invalid - points to prototype Valid PFN n PFN n PTE Invalid – in PTE address page file Page directory Share count=1 Page table Prototype page tables Physical PFN entry Operating Systems 7 memory

Memory-Mapped Files Process fjle data without manual, bufgered fjle I/O (read/write) ● Appears in memory as if read into a bufger in its entirety ● OS does the heavy lifting effjciently and reliably ● Data structures will be saved verbatim – be careful with pointers ● Convenient & effjcient in-memory algorithms: ● Can process data much larger than physical memory ● Improved performance for fjle processing (prefetching) ● No need to manage bufgers and fjle data ● No need to consume space in paging fjle ● Multiple processes can share memory ● Operating Systems 8

File Mapping Object HANDLE CreateFileMapping (HANDLE hFile, Parameters: LPSECURITY_ATTRIBUTES lpsa, hFile: DWORD fdwProtect, ● DWORD dwMaximumSizeHigh, hFile: handle to open fjle with ● compatible access rights (fdwProtect) DWORD dwMaximumSizeLow, LPCTSTR lpszMapName ); hFile == 0xFFFFFFFF: paging fjle, ● no need to create separate fjle fdwProtect: ● PAGE_READONLY, PAGE_READWRITE, PAGE_WRITECOPY ● dwMaximumSizeHigh, dwMaximumSizeLow: ● Zero: current fjle size is used ● lpszMapName: ● Name of mapping object for sharing between processes or NULL ● Operating Systems 9

Shared Memory File Mapping Process A Process B 00000000 User User accessible accessible v.a.s. v.a.s. 7FFFFFFF Physical Operating Systems 10 Memory

Copy-On-Write Pages ● Pages are originally set up as shared, read-only ● Access violation on write attempt – pager makes a copy of the page and allocates it privately to the process doing the write ● So, only need unique copies for the pages in the shared region that are actually written (example of “lazy evaluation”) ● Original values of data are still shared – e.g. writeable data initialized with C initializers ● Data copied to new physical page, but virtual addresses unaltered Operating Systems 11

Copy-On-Write Pages Orig. Data Page 1 Orig. Data Page 2 Page 3 Process Process Physical Address Address memory Space Space Operating Systems 12

Copy-On-Write Pages Orig. Data Page 1 Mod’d. Data Page 2 Page 3 Copy of page 2 Process Process Physical Address Address memory Space Space Operating Systems 13

Conclusions Intel logical selector offset Address descriptor s table limit base + Physical Address 31 22 21 12 11 0 Intel Linear 10 10 12 Address operand 4 Kb page PTE 4Kb PDE 4KiB frame 22 bit Page table operand offset 1024 entries 4Mb PDE 4 Mb page 4MiB frame Page directory Physical Memory cr 3 1024x4byte entries Operating Systems 14 (one per process) Physical address

Conclusions demand zero page read from page faults disk or kernel allocations Standby Page List Process Free zero Zero modified “soft” Bad Working page Page page Page page Page thread Sets writer List List faults List Modified Page List working set replacement Private pages at process exit Operating Systems 15

Working Set Replacement to standby or modified page list Process “WorkingSet” ● When working set max reached (or working set trim occurs), process must give up pages to make room for new pages ● a single process cannot take over all of physical memory unless other processes aren’t using it ● Question : Which pages to remove? Operating Systems 16

Balance Set Manager Balance set = sum of all working sets ● Balance Set Manager is a system thread ● Wakes up every second. If paging activity high or memory needed: ● trims working sets of processes – if thread in a long user-mode wait, marks kernel stack pages as pageable – if process has no nonpageable kernel stacks, “outswaps” process – triggers a separate thread to do the “outswap” by gradually reducing target – process’s working set limit to zero Evidence: Look for threads in “Transition” state in PerfMon ● Means that kernel stack has been paged out, and thread is waiting for memory to ● be allocated so it can be paged back in Operating Systems 17

Page Replacement: First In First Out (FIFO) ● Access to Memory Pages, in order: 3, 2, 1, 0, 3, 2, 4, 3, 2, 1, 0, 4 ● If a page needs to be replaced, evict the oldest page 3 2 1 0 3 2 4 3 2 1 0 4 3 3 3 0 0 0 4 4 4 4 4 4 - 2 2 2 3 3 3 3 3 1 1 1 - - 1 1 1 2 2 2 2 2 0 0 Operating Systems 18

Bélády's anomaly – FIFO anomaly ● increasing the number of page frames results in an increase in the number of page faults for a given memory access pattern 3 3 2 2 1 1 0 0 3 3 2 2 4 4 3 3 2 2 1 1 0 0 4 4 3 3 3 0 0 0 4 4 4 4 4 4 3 frames - - 2 2 2 3 3 3 3 3 1 1 1 - - - - 1 1 1 2 2 2 2 2 0 0 3 2 1 0 3 2 4 3 2 1 0 4 3 3 3 3 3 3 4 4 4 4 0 0 4 frames - 2 2 2 2 2 2 3 3 3 3 4 - - 1 1 1 1 1 1 2 2 2 2 - - - Operating Systems 0 0 0 0 0 0 1 1 1 19

Page Replacement: Least Recently Used (LRU) ● If a page needs to be replaced, evict the longest unused page ● expensive: every read access requires write to timestamp ● expensive: fjnding page to replace needs lots of comparisons 3 2 1 0 3 2 4 3 2 1 0 4 3 3 3 0 0 0 4 4 4 1 1 1 - 2 2 2 3 3 3 3 3 3 0 0 - - 1 1 1 2 2 2 2 2 2 4 Operating Systems 20

Page Replacement: Second Chance ● Extension to FIFO, favoring frequently accessed pages ● set “accessed” bit to 1 on page access (not page load) ● If a page needs to be replaced, evict the oldest page with unset access bit, then unset all access bits 3 2 1 0 3 2 4 3 2 1 0 4 3 3 3 0 0 0 4 4 4 1 1 1 - 2 2 2 3 3 3 3 3 3 0 0 - - 1 1 1 2 2 2 2 2 2 4 Operating Systems 21

System Working Set Just as processes have working sets, Windows’ pageable system-space code and data lives in the ● “system working set” Made up of 4 components: ● Paged pool ● Pageable code and data in the executive ● Pageable code and data in kernel-mode drivers, Win32K.Sys, graphics drivers, etc. ● Global fjle system data cache ● To get physical (resident) size of these with PerfMon, look at: ● Memory | Pool Paged Resident Bytes ● Memory | System Code Resident Bytes ● Memory | System Driver Resident Bytes ● Memory | System Cache Resident Bytes ● Memory | Cache bytes counter is total of these four “resident” (physical) counters (not just ● the cache; in NT4, same as “File Cache” on Task Manager / Performance tab) Operating Systems 22