Computer Systems Lab Matteo Corti Informatikdienste, ETH Z urich - PowerPoint PPT Presentation

Computer Systems Lab Matteo Corti Informatikdienste, ETH Z¨ urich 2007-03-19 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 1 / 24

Introduction Course schedule • lecture: Monday 4:15 – 5:00 p.m. (RZ F21) • assignments: in teams, time to be arranged individually Contact Matteo Corti Mathias Payer SOW E16 RZ H7 matteo.corti@id.ethz.ch mathias.payer@inf.ethz.ch Resources • Homepage: http://www.lst.inf.ethz.ch/teaching/lectures/ss07/2100/index.html Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 2 / 24

Structure of the course Projects • Topics related to operating systems • Practical focus Organization • programming assignments (labs) • teams (max 2 students) are allowed • final grade based on the labs Environment • Unix/C Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 3 / 24

Course schedule (tentative) Date Lecture Project 2007-03-19 Memory allocation 2007-03-26 malloc lab 2007-04-02 Scheduling 2007-04-09 Easter Monday Sechsel¨ 2007-04-16 auten scheduler lab 2007-04-23 File systems 2007-04-30 filesystem lab 2007-05-07 Networking 2007-05-14 proxy lab 2007-05-21 Distributed systems 2007-05-28 Whit Monday 2007-06-04 Distributed systems distributed lab 2007-06-11 Optional labs 2007-06-18 TBD Optional labs Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 4 / 24

Suggested literature R. E. Bryant and D. O’Hallaron Computer Systems — A Programmer’s Perspective Prentice Hall, Upper Saddle River, NJ, 2003. Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 5 / 24

Project 1: Memory allocation Summary Implement a memory allocator: malloc , free , realloc Goals • Learn how to manage storage allocation. • Good exercise to master pointers Environment • Unix like system • C Resources • Computer Systems — A Programmer’s Perspective, Chapter 10.9 (Dynamic Memory Allocation), pages 730–755. Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 6 / 24

The Heap Area of a process’s (virtual) memory User stack where which is dynamically allocated. Example (Unix) Memory mapped region • grows upwards for shared libraries • the brk pointer marks the top of the heap top of the heap (brk) Heap Uninitialized data (.bss) Initialized data (.data) Program text (.text) Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 7 / 24

Memory Management The memory manager (user space) maintains a process’s heap : a collection of blocks (contiguous chunks of memory) which are either allocated or free . Allocation is explicit (e.g., malloc in C or new in Java). Deallocation can be explict (e.g., free in C) or implicit (garbage collection in Java). Issues The memory manager needs to: • distinguish block boundaries • distinguish between free and allocated blocks Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 8 / 24

Memory allocation: Strategies 32 8 16 8 16 32 8 8 32 16 8 8 32 16 16 64 8 16 8 32 • first-fit : use the first free block which is big enough • best-fit : take smallest fitting block • worst-fit : take biggest available block • quick-fit : best-fit but multiple free-lists (one per block size) Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 9 / 24

Memory allocation: Bitmaps The simplest (but often slowest) way to manage memory blocks is to mark in a bitmap if a block is free or allocated. 0 0 0 0 1 1 1 0 32 8 16 8 0 0 1 1 1 1 0 1 1 1 1 1 0 0 0 1 16 32 8 8 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 32 16 8 8 0 1 1 0 1 1 1 1 32 16 16 64 8 16 8 32 Space For a 4GB memory space and 4KB words a 128MB table is necessary to store the free/allocated information. Speed Slow scan to find a free block Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 10 / 24

Memory allocation: Dynamic data structures Free blocks are stored in a dynamic data structure (ordered list , ordered tree , . . . ). free_list 32 8 16 8 16 32 8 8 32 16 8 8 32 16 16 64 8 16 8 32 Lists are usually ordered by allocation address to allow a faster merging of free blocks. 8 16 8 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 11 / 24

Memory allocation: Dynamic data structures Free blocks are stored in a dynamic data structure (ordered list , ordered tree , . . . ). free_list 32 8 16 8 16 32 8 8 32 16 8 8 32 16 16 64 8 16 8 32 Lists are usually ordered by allocation address to allow a faster merging of free blocks. 8 16 8 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 11 / 24

Memory allocation: Dynamic data structures Free blocks are stored in a dynamic data structure (ordered list , ordered tree , . . . ). free_list 32 8 16 8 16 32 8 8 32 16 8 8 32 16 16 64 8 16 8 32 Lists are usually ordered by allocation address to allow a faster merging of free blocks. 16 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 11 / 24

Memory allocation: Lists Space Linking information as well as some flags (i.e., free or allocated) has to be stored in the block. Speed Depending on the chosen structure operations can be implemented efficiently. Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 12 / 24

Memory allocation: Space utilization External fragmentation Unused space among the blocks . Depends on the allocation strategy. Internal fragmentation Unused space inside the blocks . Depends on the possible block sizes (granularity). Data structures Space used to maintain data structures cannot be allocated (bitmaps, pointers, flags, . . . ) Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 13 / 24

Memory allocation: Buddy System An example of allocation strategy: • Each block has a size of 2 k • The system maintains a list for each size • Block allocation: • Find a block which fits • If necessary split • Put the remaining part in the right list • Free: • Mark the block as free • If is possible merge with a free neighbor The buddy system is used in several operating systems (Windows, Linux, Oberon, . . . ). Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 14 / 24

Memory allocation: Buddy System (example) Free 8 8 16 32 8 8 16 16 32 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 15 / 24

Memory allocation: Buddy System (example) Free 8 8 16 32 8 8 8 8 8 16 32 16 16 32 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 15 / 24

Memory allocation: Buddy System (example) Free 8 8 16 32 8 8 8 16 32 16 16 16 16 16 32 32 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 15 / 24

Memory allocation: Buddy System (example) Free 8 8 16 32 8 8 8 16 32 16 16 16 32 32 32 32 32 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 15 / 24

Memory allocation: Buddy System (example) Free 8 8 16 32 8 8 8 16 32 16 16 16 32 32 32 32 32 Allocate (8) 32 32 8 16 32 32 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 15 / 24

Memory allocation: Buddy System (example) Free 8 8 16 32 8 8 8 16 32 16 16 16 32 32 32 32 32 Allocate (8) 32 32 8 32 16 16 16 16 16 32 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 15 / 24

Memory allocation: Buddy System (example) Free 8 8 16 32 8 8 8 16 32 16 16 16 32 32 32 32 32 Allocate (8) 32 32 8 8 8 32 16 16 16 16 32 8 8 16 32 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 15 / 24

Memory allocation: Buddy System (example) Free 8 8 16 32 8 8 8 16 32 16 16 16 32 32 32 32 32 Allocate (8) 32 32 8 8 32 16 16 16 16 32 8 8 16 32 32 8 8 16 Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 15 / 24

Memory allocation: Slabs Disadvantages of the buddy system: • internal fragmentation (max 50%) • bad distribution of the block addresses (bad caches performance) Solaris (J. Bonwick 1994) introduced the concept of a slab allocator : • based on the idea that processes are likely to request a lot of objects of the same size: these objects can be kept and reused • slabs are collections of objects of the same size • sizes (and addresses) are not geometrically distributed Used by AmigaOS (4), Linux ( ≥ 2.2) and Solaris ( ≥ 2.4) J. Bonwick The Slab Allocator: An Object-Caching Kernel Memory Allocator Proc. 1994 USENIX Annual Tech. Conf pp. 87–98, June 1994, Boston, MA. Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 16 / 24

Lab: Task Write a dynamic memory allocator, i.e., implement: • malloc • free • realloc Goals of the lab • understand how a real memory allocator works • evaluate different design strategies • a nice exercise for low level C programming Matteo Corti (Informatikdienste, ETH Z¨ urich) Computer Systems Lab 2007-03-19 17 / 24