CS4513 Process capacity restricted to vmem size When process - PDF document

Motivation • Processes store, retrieve information CS4513 • Process capacity restricted to vmem size • When process terminates, memory lost Distributed Computer • Multiple processes share information Systems • Requirements: – large File Systems Solution? File – persistent – concurrent access System! Outline File Systems • Abstraction to disk (convenience) • Files ← – “The only thing friendly about a disk is that • Directories it has persistent storage.” • Disk space management – Devices may be different: tape, IDE/SCSI, • Misc NFS • Users – don’t care about detail – care about interface • OS – cares about implementation (efficiency) File System Concepts Files: The User’s Point of View • Files - store the data • Naming: how do I refer to it? • Directories - organize files – blah, BLAH, Blah • Partitions - separate collections of – file.c, file.com • Structure: what’s inside? directories (also called “volumes”) – all directory information kept in partition – Sequence of bytes (most modern OSes) – mount file system to access – Records - some internal structure • Protection - allow/restrict access for files, – Tree - organized records directories, partitions 1

Files: The User’s Point of View File Operations • Type: • Create • Seek - for random access • Delete • Get attributes – ascii - human readable • Truncate • Set attributes – binary - computer only readable – “magic number” or extension (executable, c-file …) • Open • Access Method: • Read – sequential (for character files, an abstraction of • Write I/O of serial device such as a modem) • Append – random (for block files, an abstraction of I/O to block device such as a disk) • Attributes: – time, protection, owner, hidden, lock, size ... Example: Unix open() Unix open() - Under the Hood int fid = open(“blah”, flags); int open(char *path, int flags [, int mode]) read(fid, …); User Space • path is name of file System Space • flags is bitmap to set switch – O_RDONLY, O_WRONLY… 0 stdin 1 stdou – O_CREATE then use mode for perms 2 t • success, returns index stder ... 3 r File File Structure ... Descriptor ... (where blocks are) (index) (attributes) (Per process) (Per device) File System Implementation Example: WinNT/2k CreateFile() • Returns file object handle: Process Open File File Descriptor Disk Control Block Table Table HANDLE CreateFile ( lpFileName , // name of file File sys info dwDesiredAccess , // read-write File Copy fd dwShareMode , // shared or not descriptors Open to mem File lpSecurity , // permissions Pointer ... Directories Array ) • File objects used for all: files, (in memory (per process) Data directories, disk drives, ports, pipes, copy, sockets and console one per device) Next up: file descriptors! 2

Contiguous Allocation File System Implementation • Store file as contiguous block – ex: w/ 1K block, 50K file has 50 conseq • Which blocks with which file? blocks • File descriptor implementations: File A: start 0, length 2 File B: start 14, length 3 • Good: – Contiguous – Linked List – Easy: remember location with 1 number – Linked List with Index – Fast: read entire file in 1 operation (length) • Bad: – I-nodes – Static: need to know file size at creation File • or tough to grow! Descriptor – Fragmentation: remember why we had paging? Linked List Allocation Linked List Allocation with Index • Keep a linked list with disk blocks Physical null null Block • Table in memory File File File File File 0 Block Block Block Block Block – faster random access 1 0 1 2 0 1 2 – can be large! null Physical 4 7 2 6 3 • 1k blocks, 500K disk Block 3 null • Good: • = 2MB! 4 7 – Easy: remember 1 number (location) – MS-DOS FAT, Win98 5 – Efficient: no space lost in fragmentation VFAT 6 3 • Bad: 7 2 – Slow: random access bad I-nodes Outline single i-node • Fast for small indirect block • Files attributes (done) files • Can hold big files • Directories ← • Size? Disk blocks • Disk space management – 4 kbyte block • Misc double indirect block triple indirect block 3

Directories Directories • Just like files, only have special bit set so you cannot modify them ( what?! ) • Before reading file, must be opened – data in directory is information / links to • Directory entry provides information to files – modify through system call get blocks – (See ls.c ) – disk location (block, address) • Organized for: – i-node number • Map ascii name to the file descriptor – efficiency - locating file quickly – convenience - user patterns • groups (.c, .exe), same names • Tree structure directory the most flexible – aliases allow files to appear at more than one location Hierarchical Directory (MS-DOS) Simple Directory • No hierarchy (all “root”) • Tree • Entry • Entry: – name – name - date – block count – type (extension) - block number (w/FAT) – block numbers – time name block count name type attrib time date block size block numbers Hierarchical Directory (Unix) Unix Directory Example Root Directory Block 132 Block 406 • Tree 1 . I-node 6 6 . I-node 26 • Entry: 26 . 1 .. 1 .. 6 .. 4 bin 26 bob – name 12 grants 7 dev 17 jeff 81 books – inode number (try “ls –I” or “ls –iad .”) 14 lib 14 sue 132 406 • example: 60 mbox 9 etc 51 sam 17 Linux 6 usr 29 mark /usr/bob/mbox Aha! 8 tmp Looking up I-node 60 Looking up bob gives has contents Relevant usr gives I-node 26 /usr/bob is of mbox data (/usr) I-node 6 in block 406 inode name is in block 132 4

Storing Files Problems • a) Directory entry contains disk blocks? No longer a tree: – contents (blocks) may change • b) Directory entry points to file descriptor? C � Directed Acyclic B Graph – if removed, refers to non-existent file – must keep count, remove only if 0 B B B ? “alias” – hard link – Similar if delete file in use (show example) • • c) Have new type of file “link”? How to manage aliases? Possibilities: a) Directory entry contains disk blocks? – contains alternate name for file – overhead, must parse tree second time b) Directory entry points to attributes – soft link structure? – often have max link count in case loop (show example) c) Have new type of file “link”? Outline Disk Space Management • n bytes • Files (done) – contiguous • Directories (done) – blocks • Similarities with memory management • Disk space management ← – contiguous is like variable-sized partitions • Misc • but moving on disk very slow! • so use blocks – blocks are like paging • how to choose block size? • (Note, disk block size typically 512 bytes, but file system logical block size chosen when formatting) Choosing Block Size Keeping Track of Free Blocks • Two methods • Large blocks (note, these are stored on the disk) – linked list of disk blocks – faster throughput, less seek time • one per block or many per block – wasted space (internal fragmentation) • Small blocks – bitmap of disk blocks • Linked List of Free Blocks (many per block) – less wasted space – 1K block, 16 bit disk block number – more seek time since more blocks = 511 free blocks/block • 200 MB disk needs 400 free blocks = 400k Disk Space • Bit Map Utilization • 200 MB disk needs 20 Mbits • 30 blocks = 30k Data Rate • 1 bit vs. 16 bits Block size 5

File System Performance Tradeoffs • Disk access 100,000x slower than memory – reduce number of disk accesses needed! • Only if the disk is nearly full does linked • Block/buffer cache list scheme require fewer blocks – cache to memory • If enough RAM, bitmap method preferred • Full cache? FIFO, LRU, 2nd chance … • If only 1 “block” of RAM, and disk is full, – exact LRU can be done (why?) • LRU inappropriate sometimes bitmap method may be inefficient since have to load multiple blocks – crash w/i-node can lead to inconsistent state – linked list can take first in line – some rarely referenced (double indirect block) Outline Modified LRU • Files (done) • Is the block likely to be needed soon? • Directories (done) • Disk space management – if no, put at beginning of list (done) • Is the block essential for consistency of • Misc ← file system? – partitions ( fdisk, mount) – write immediately • Occasionally write out all – maintenance – quotas – sync • Linux and WinNT/2000 Partitions Partitions: fdisk • mount, unmount – load “super-block” from disk • Partition is large group of sectors allocated for a – pick “access point” in file- specific purpose system / (root) – IDE disks limited to 4 physical partitions • Super-block – logical (extended) partition inside physical partition • Specify number of cylinders to use – file system type usr home tmp • Specify type – block size – magic number recognized by OS – free blocks – free i-nodes (Hey, show example) 6