Memory Management Memory Management 5A. Memory Management and - PDF document

4/14/2018 Memory Management Memory Management 5A. Memory Management and Address Spaces 1. allocate/assign physical memory to processes 5B. Memory Allocation – explicit requests: malloc (sbrk) – implicit: program loading, stack extension 5C. Dynamic Allocation Algorithms 2. manage the virtual address space 5D. Advanced Allocation Techniques – instantiate virtual address space on context switch 5G. Common Dynamic Memory Errors – extend or reduce it on demand 5F. Garbage Collection 3. manage migration to/from secondary storage – optimize use of main storage – minimize overhead (waste, migrations) Memory management 1 Memory management 2 Memory Management Goals Linux Process – virtual address space 1. transparency – process sees only its own virtual address space 0x0100000 0x00000000 0x0110000 – process is unaware memory is being shared shared code private data DLL 1 DLL 2 2. efficiency – high effective memory utilization DLL 3 private stack – low run-time cost for allocation/relocation 0x0120000 0xFFFFFFFF 3. protection and isolation All of these segments appear to be present in memory – private data will not be corrupted whenever the process runs. – private data cannot be seen by other processes Memory management 3 Memory management 4 (code segments) Physical Memory Allocation • program code process 1 shared lib – allocated when program loaded X data and stack OS kernel – initialized with contents of load module shared code process 3 segment A data and stack • shared and Dynamically Loadable Libraries process 2 data and stack – mapped in at exec time or when needed shared code • all are read-only and fixed size segment B – somehow shared by multiple processes Physical memory is divided between the OS kernel, – shared code must be read only process private data, and shared code segments. Memory management 5 Memory management 6 1

4/14/2018 (implementing: code segments) (data/stack segments) • program loader • they are process-private, read/write – ask for memory (size and virtual location) • initialized data – copy code from load module into memory – allocated when program loaded • run-time loader – initialized from load module – request DLL be mapped (location and size) • data segment expansion/contraction – edit PLT pointers from program to DLL – requested via system calls (e.g. sbrk) • memory manager – only added/truncated part is affected – allocates memory, maps into process • process stack – allocated and grown automatically on demand Memory management 7 Memory management 8 (implementing: data/stack) Fixed Partition Memory Allocation • pre-allocate partitions for n processes • program loader – reserving space for largest possible process – ask for memory (location and size) • very easy to implement – copy data from load module into memory – common in old batch processing systems – zero the uninitialized data • well suited to well-known job mix • memory manager – must reconfigure system for larger processes – invoked for allocations and stack extensions • likely to use memory inefficiently – allocates and deallocates memory – large internal fragmentation losses – adjusts process address space accordingly – swapping results in convoys on partitions Memory management 9 Memory management 10 Fragmentation Internal Fragmentation partn #1 partn #2 partn #3 • The curse of spatially partitioned resources 8MB 4MB 4MB – wasted or unusable free space waste 2MB • internal fragmentation – not needed by its owner • external fragmentation – uselessly small pieces – all such resources, not merely memory process • inheritance of farm land waste 1MB A waste 2MB • calendar appointment packing (6 MB) process process B • Choose your poison C (3 MB) (2 MB) – static allocation … constant internal waste Total waste = 2MB + 1MB + 2MB = 5/16MB = 31% – dynamic allocation … progressive external loss Memory management 12 2

4/14/2018 (Internal Fragmentation) Variable Partition Allocation • wasted space in fixed sized blocks • start with one large "heap" of memory • caused by a mis-match between • when a process requests more memory – the chosen sizes of a fixed-sized blocks – find a large enough chunk of memory – the actual sizes that programs request – carve off a piece of the requested size – put the remainder back on the free list • average waste: 50% of each block • when a process frees memory • overall waste reduced by multiple sizes – put it back on the free list – suppose blocks come in sizes S1 and S2 • eliminates internal fragmentation losses – average waste = ((S1/2) + (S2 - S1)/2)/2 Memory management 13 Memory management 14 (External/Global Fragmentation) External Fragmentation • each allocation creates left-over fragments P F P A P A – over time these become smaller and smaller P D P D P B • tiny left-over fragments are useless – they are too small to satisfy any request P C P C P C – but their combined size may be significant • there are three obvious approaches: P E P E – try to avoid creating tiny fragments – try to recombine adjacent fragments – re-pack the allocated space more densely Fixed vs Variable Partition Stack vs Heap Allocation • Fixed partition allocation • stack allocation – allocation and free lists are trivial – compiler manages space (locals, call info) – internal fragmentation is inevitable – data is valid until stack frame is popped • average 50% (unless we have multiple sizes) – OS automatically extends/shrinks stack segment • Variable partition allocation • heap allocation – allocation is complex and expensive – explicitly allocated by application (malloc/new) • long searches of complex free lists – data is valid until free/delete (or G.C.) – eliminates internal fragmentation – heap space managed by user-mode library – external fragmentation is inevitable – data segment size adjusted by system call • can be managed by (complex) coalescing Memory management 17 Memory management 18 3

4/14/2018 sbrk(2) vs. malloc(3) managing process private data • sbrk(2) … managing size of data segment initialized 000000000000 data 000000000000 – each address space has a private data segment – process can request that it be grown/shrunk – sbrk(2) specifies desired ending address 1. loader allocates space for, and copies initialized data from load module. – this is a coarse and expensive operation 2. loader allocates space for, and zeroes uninitialized data from load module. • malloc(3) … dynamic heap allocation 3. after it starts, program uses sbrk to extend the process data segment – sbrk(2) is called to extend/shrink the heap and then puts the newly created chunk on the free list – malloc(3) is called to carve off small pieces 4. Free space “heap” is eventually consumed – mfree(3) is called to return them to the heap program uses sbrk to further extend the process data segment and then puts the newly created chunk on the free list Memory management 19 Memory management 20 (Free lists: keeping track of it all) Variable Partition Free List • fixed sized blocks are easy to track F N L head R E E E X N E T – a bit map indicating which blocks are free U N L S E • variable chunks require more information E E X N D T List might – a linked list of descriptors, one per chunk contain all F N L R E E memory E X N E T – each lists size of chunk, whether it is free fragments U N L – each has pointer to next chunk on list S E E …or only E X N D T B 1 fragments – descriptors often at front of each chunk F N that are free. L R E E E X N • allocated memory may have descriptors too E T … Memory management 21 Memory management 22 Free Chunk Carving Avoid Creating Small Fragments • Choose carefully which piece to carve U N L S E E E X N D T 1. find large enough free chunk • “There Ain’t No Such Thing As A Free Lunch” – careful choices mean longer searches U S 2. reduce len to requested size E D – optimization means more complex data structures F N F N L L R E R E E E E X E X – cost of reduced fragmentation is more cycles N N 3. new header for residual chunk E T E T • A few obvious choices for “smart” choices … 4. insert new chunk into list – Best fit U N L S E E – Worst fit E X N 5. mark carved piece as in use D T … – First fit – Next fit … Memory management 23 4