File Systems CS 4410, Opera4ng Systems Fall 2016 Cornell - PowerPoint PPT Presentation

File Systems CS 4410, Opera4ng Systems Fall 2016 Cornell University Rachit Agarwal Anne Bracy See: Ch 13 in OSPP textbook The slides are the product of many rounds of teaching CS 4410 by Professors Sirer, Bracy, Agarwal, George, and Van Renesse.

File Systems 101 Long-term InformaLon Storage Needs • large amounts of informa4on • informa4on must survive processes • need concurrent access to mul4ple processes SoluLon • Store informa4on on disks in units called files • persistent, only owner can delete • managed by the OS File Systems: How the OS manages files! 2

Challenges for File System Designers • Performance: despite limita4ons of disks ‣ leverage spa4al locality • Flexibility: need jacks-of-all-trades, not just FS for X • Persistence: maintain/update user data + internal data structures on persistent storage devices • Reliability: must store data for long periods of 4me, despite crashes or malfunc4ons 3

First things first: Name the File! 1. Files are abstracted unit of informa4on 2. Don’t care exactly where on disk the file is ➜ Files have human readable names • file given name upon crea4on • use the name to access the file 4

Name + Extension Naming Conven:ons • OS dependent Windows not case sensi4ve, UNIX is • Usually ok up to 255 characters File Extensions • Also OS dependent Windows: aZaches meaning to extensions associates applica4ons to extensions UNIX: extensions not enforced by OS - Some applica4ons might insist upon them (e.g., .c, .h, .o, .s, etc. for C compiler) 5

Directory Maps human readable name to file number File File directory index Storage Name Number structure Block music 320 “foo.txt” 871 work 219 foo.txt 871 6

Path Names • Absolute: path of file from the root directory e.g., /home/pat/projects/test.c • Rela4ve: path from the current working directory (current work dir stored in process’ PCB) 2 special entries in each UNIX directory: “.” current dir “..” for parent To access a file: • Go to the folder where file resides —OR— • Specify the path where the file is 7

Paths in acLon! all files File 2 bin 737 ˝ / ˝ usr 924 home 158 File 158 mike 682 ˝ /home ˝ ada 818 tom 830 File 830 music 320 ˝ /home/tom ˝ work 219 foo.txt 871 Example: File 871 The quick /home/tom/foo.txt ˝ /home/tom/foo.txt ˝ brown fox jumped over the lazy dog. 8

ImplemenLng Directories When a file is opened, OS uses path name to find dir • Directory has informa4on about the file’s disk blocks • Directory also has aZributes of each file Directory: map ASCII file name to file aZributes & loca4on 2 op4ons: entries have all aZributes, or point to file I-nod e 9

File System Layout File System is stored on disks • disk is divided into 1 or more parLLons • Sector 0 of disk called Master Boot Record • end of MBR: par44on table (par44ons’ start & end addrs) First block of each par44on has boot block • loaded by MBR and executed on boot enLre disk PARTITION #1 PARTITION #2 PARTITION #3 PARTITION #4 PARTITION MBR TABLE BOOT BLOCK SUPERBLOCK Free Space Mgmt I-Nodes Root Dir Files & Directories 10

Storing Files Files can be allocated in different ways: • Con4guous alloca4on All bytes together, in order • Linked Structure Each block points to the next block • Indexed Structure Index block points to many other blocks Which is best? For sequen4al access? Random access? Large files? Small files? Mixed? 11

ConLguous AllocaLon All bytes together, in order + Simple: state required per file: start block & size + Performance: en4re file can be read with one seek – Fragmenta4on external is bigger problem – Usability: user needs to know size of file file1 file2 file3 file4 file5 Used in CD-ROMs, DVDs 12

Case Study #1: File Alloca6on Table (FAT) MicrosoY File AllocaLon Table [late 70’s] • originally: MS-DOS, early version of Windows • today: s4ll widely used (e.g., CD-ROMs, thumb drives, camera cards) File table: • Linear map of all blocks on disk • Each file a linked list of blocks FAT BLOCKS decoupled data data data physically next next next 32 bit entries 13

FAT File System FAT Data Blocks 0 File 9 1 • 1 entry per block File 12 2 3 File 9 Block 3 • EOF for last block 4 5 • 0 indicates free block 6 0 • usually uses a simple 7 8 alloca4on strategy (e.g. 9 File 9 Block 0 } 10 File 9 Block 1 next-fit) 11 File 9 Block 2 12 File 12 Block 0 • directory entry maps 13 0 14 name to FAT index 15 EOF 16 File 12 Block 1 EOF Directory 17 File 9 Block 4 18 bart 9 19 0 maggie 12 20 14

FAT Directory Structure music 320 work 219 Folder: a file with 32-byte entries foo.txt 871 Each Entry: • 8 byte name + 3 byte extension (ASCII) • crea4on date and 4me • last modifica4on date and 4me • first block in the file (index into FAT) • size of the file • Long and Unicode file names take up mul4ple entries 15

How Good is FAT? + Simple - Poor locality • state required per file: - Many file seeks unless start block only en4re FAT in memory + Widely supported - Poor random access + No external - Limited metadata fragmenta4on - Limited access control + all of block used for data - No support for hard links - Limita4ons on volume and file size - No support for reliability techniques 16

Case Study #2: Fast File System (FSS) [early 80’s] UNIX Fast File System Inode Array Inode • inode table File Metadata - Analogous to FAT table • inode Direct Pointer DP DP - Metadata DP DP - 12 data pointers DP DP - 3 indirect pointers DP DP DP DP Direct Pointer Indirect Pointer Dbl. Indirect Ptr. Tripl. Indirect Ptr. 17

FFS Superblock Iden4fies file system’s key parameters: • type • block size • inode array loca4on and size (or analogous structure for other FSs) block number 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 blocks: super i-node Remaining blocks block blocks 18

FFS: Index Structures Inode Array Triple Double Indirect Indirect Indirect Data Inode Blocks Blocks Blocks Blocks File Metadata Direct Pointer DP DP DP DP DP DP DP DP DP DP Direct Pointer Indirect Pointer Dbl. Indirect Ptr. Tripl. Indirect Ptr. 19

What else is in an Inode? Inode AKA file control block (FCB) • Type File - ordinary file File Metadata Metadata - directory - symbolic link Direct Pointer DP - special device DP DP • Size of the file (in #bytes) DP • #links to the i-node DP DP • Owner (user id and group id) DP • Protec4on bits DP DP • Times DP - crea4on, last accessed, last Direct Pointer Indirect Pointer modified Dbl. Indirect Ptr. Tripl. Indirect Ptr. 20

FFS: Index Structures Inode Array Triple Double Indirect Indirect Indirect Data Inode Blocks Blocks Blocks Blocks 12x4K=48K directly reachable from the inode File Metadata Direct Pointer DP DP n=1: 4MB 1K DP 2 (nx10) x4K = DP 12 DP with n levels of indirecLon DP DP 1K 1K DP n=2: 4GB DP DP 1K Direct Pointer Indirect Pointer 1K 1K Dbl. Indirect Ptr. 1K 1K Tripl. Indirect Ptr. Assume blocks are 4K & n=3: 4TB 1K block references 4 bytes 1K 21

4 CharacterisLcs of FFS 1. Tree Structure • efficiently find any block of a file 2. High Degree (or fan out) • minimizes number of seeks • supports sequen4al reads & writes 3. Fixed Structure • implementa4on simplicity 4. Asymmetric • not all data blocks are at the same level • supports large • small files don’t pay large overheads 22

Small Files in FFS Inode Array Data Inode Blocks all blocks File Metadata reached via Direct Pointer direct pointers DP DP Direct Pointer NIL NIL What if fixed 3 levels instead? NIL NIL • 4 KB file consumes ~16 KB (4 KB data + NIL NIL 3 levels of 4KB indirect blocks + inode) NIL NIL NIL • reading file would require reading 5 NIL NIL blocks to traverse tree 23

Sparse Files in FFS 2 x 4 KB bocks: 1 @ offset 0 1 @ offset 2 30 Inode Triple Double Indirect Indirect Indirect Data File Metadata Blocks Blocks Blocks Blocks Direct Pointer NIL File size (ls -lgGh): 1.1 GB NIL Space consumed (du -hs): 16 KB NIL NIL NIL Read from hole: 0 -filled buffer created NIL NIL Write to hole: storage blocks for data + NIL NIL required indirect blocks allocated NIL NIL NIL Dbl. Indirect Ptr. NIL 24

FFS Directory Structure music 320 work 219 Originally: array of 16 byte entries foo.txt 871 • 14 byte file name • 2 byte i-node number Now: linked lists. Each entry contains: • 4-byte inode number • Length of name • Name (UTF8 or some other Unicode encoding) First entry is “.”, points to self Second entry is “..”, points to parent inode 25