File Systems and NFS File Systems and NFS Representing Files On - PowerPoint PPT Presentation

File Systems and NFS File Systems and NFS

Representing Files On Disk: Nachos Representing Files On Disk: Nachos An OpenFile represents a file in OpenFile(sector) active use, with a seek pointer and OpenFile Seek(offset) read/write primitives for arbitrary Read(char* data, bytes) byte ranges. Write(char* data, bytes) once upo logical n a time block 0 /nin a l A file header describes an on-disk file as an ordered sequence of and far logical far away sectors with a length, mapped by block 1 FileHdr ,/nlived t a logical-to-physical block map. he wise logical and sage block 2 wizard. bytes sectors Allocate(...,filesize) OpenFile* ofd = filesys->Open(“tale”); length = FileLength() ofd->Read(data, 10) gives ‘once upon ‘ sector = ByteToSector(offset) ofd->Read(data, 10) gives ‘a time/nin ‘

File Metadata File Metadata On disk, each file is represented by a FileHdr structure. The FileHdr object is an in-memory copy of this structure. The FileHdr is a file system “bookeeping” structure that supplements the file data itself: these kinds of file attributes : may include owner, structures are called filesystem metadata . access control, time of create/modify/access, etc. bytes A Nachos FileHdr occupies sectors etc. exactly one disk sector. logical-physical block map (like a translation table) To operate on the file (e.g., to open it), the FileHdr must be read into memory. physical block pointers in the Any changes to the attributes block map are sector IDs or block map must be written FileHdr* hdr = new FileHdr(); back to the disk to make them permanent. hdr->FetchFrom(sector) hdr->WriteBack(sector)

Representing Large Files Representing Large Files inode The Nachos FileHdr occupies exactly one disk sector, limiting the maximum file size. direct sector size = 128 bytes 120 bytes of block map = 30 entries block each entry maps a 128-byte sector map max file size = 3840 bytes (12 entries) indirect In Unix, the FileHdr (called an index- block node or inode ) represents large files using a hierarchical block map. Each file system block is a clump of sectors (4KB, 8KB, 16KB). Inodes are 128 bytes, packed into blocks. Each inode has 68 bytes of attributes and 15 block map entries. double suppose block size = 8KB indirect 12 direct block map entries in the inode can map 96KB of data. block One indirect block (referenced by the inode) can map 16MB of data. One double indirect block pointer in inode maps 2K indirect blocks. maximum file size is 96KB + 16MB + (2K*16MB) + ...

Representing Small Files Representing Small Files Internal fragmentation in the file system blocks can waste significant space for small files. E.g., 1KB files waste 87% of disk space (and bandwidth) in a naive file system with an 8KB block size. Most files are small: one study [Irlam93] shows a median of 22KB. FFS solution: optimize small files for space efficiency. • Subdivide blocks into 2/4/8 fragments (or just frags ). • Free block maps contain one bit for each fragment. To determine if a block is free, examine bits for all its fragments. • The last block of a small file is stored on fragment(s). If multiple fragments they must be contiguous.

Basics of Directories Basics of Directories A directory is a set of file names, supporting lookup by symbolic name. In Nachos, each directory is a file containing a set of mappings from name-> FileHdr . Directory(entries) wind: 18 directory sector = Find(name) fileHdr 0 Add(name, sector) snow: 62 Remove(name) 0 Each directory entry is a fixed-size rain: 32 slot with space for a FileNameMaxLen byte name. hail: 48 Entries or slots are found by a linear scan. A directory entry may hold a pointer to another directory, sector 32 forming a hierarchical name space.

A Nachos Filesystem Filesystem On Disk On Disk A Nachos An allocation bitmap file maintains A directory maintains the free/allocated state of each physical name->FileHdr mappings for block; its FileHdr is always stored in all existing files; its FileHdr is sector 0. always stored in sector 1. sector 0 sector 1 allocation bitmap file wind: 18 directory 11100010 file 0 00101101 snow: 62 10111101 0 once upo rain: 32 n a time 10011010 hail: 48 /n in a l 00110001 00010101 Every box in this diagram 00101110 and far represents a disk sector. 00011001 far away 01000100 , lived th

A Typical Unix File Tree A Typical Unix File Tree Each volume is a set of directories and files; a host’s file tree is the set of directories and files visible to processes on a given host. / File trees are built by grafting volumes from different volumes or from network servers. bin etc tmp usr vmunix In Unix, the graft operation is the privileged mount system call, ls sh project users and each volume is a filesystem . packages mount point mount (coveredDir, volume) (volume root) coveredDir: directory pathname volume : device specifier or network volume volume root contents become visible at pathname coveredDir tex emacs

Filesystems Filesystems Each file volume ( filesystem ) has a type , determined by its disk layout or the network protocol used to access it. ufs (ffs), lfs, nfs, rfs, cdfs , etc. Filesystems are administered independently. Modern systems also include “logical” pseudo-filesystems in the naming tree, accessible through the file syscalls. procfs : the /proc filesystem allows access to process internals. mfs : the memory file system is a memory-based scratch store. Processes access filesystems through common system calls.

VFS: the Filesystem Filesystem Switch Switch VFS: the Sun Microsystems introduced the virtual file system interface in 1985 to accommodate diverse filesystem types cleanly. VFS allows diverse specific file systems to coexist in a file tree, isolating all FS-dependencies in pluggable filesystem modules. user space VFS was an internal kernel restructuring with no effect on the syscall interface. syscall layer (file, uio, etc.) network Virtual File System (VFS) Incorporates object-oriented concepts: protocol a generic procedural interface with stack NFS FFS LFS *FS etc. etc. (TCP/IP) multiple implementations. device drivers Based on abstract objects with dynamic Other abstract interfaces in the kernel: device drivers, method binding by type...in C. file objects, executable files, memory objects.

Vnodes Vnodes In the VFS framework, every file or directory in active use is represented by a vnode object in kernel memory. free vnodes syscall layer Each vnode has a standard file attributes struct. Generic vnode points at filesystem-specific struct (e.g., inode, rnode ), seen only by the filesystem. Each specific file system maintains a cache of its Vnode operations are resident vnodes. macros that vector to filesystem-specific NFS UFS procedures.

Vnode Operations and Attributes Operations and Attributes Vnode vnode attributes ( vattr ) directories only type (VREG, VDIR, VLNK, etc.) vop_lookup (OUT vpp, name) mode (9+ bits of permissions) vop_create (OUT vpp, name, vattr) nlink (hard link count) vop_remove (vp, name) owner user ID vop_link (vp, name) owner group ID vop_rename (vp, name, tdvp, tvp, name) filesystem ID vop_mkdir (OUT vpp, name, vattr) unique file ID vop_rmdir (vp, name) file size (bytes and blocks) vop_symlink (OUT vpp, name, vattr, contents) access time vop_readdir (uio, cookie) modify time vop_readlink (uio) generation number files only generic operations vop_getpages (page**, count, offset) vop_getattr (vattr) vop_putpages (page**, count, sync, offset) vop_setattr (vattr) vop_fsync () vhold() vholdrele()

V/Inode Inode Cache Cache V/ VFS free list head HASH( fsid, fileid ) Active vnodes are reference- counted by the structures that hold pointers to them. - system open file table - process current directory - file system mount points - etc. Each specific file system maintains its own hash of vnodes (BSD). - specific FS handles initialization - free list is maintained by VFS vget(vp): reclaim cached inactive vnode from VFS free list vref(vp): increment reference count on an active vnode vrele(vp): release reference count on a vnode vgone(vp): vnode is no longer valid (file is removed)

Network File System (NFS) Network File System (NFS) server client syscall layer user programs VFS syscall layer NFS VFS server UFS NFS UFS client network

NFS Protocol NFS Protocol NFS is a network protocol layered above TCP/IP. • Original implementations (and most today) use UDP datagram transport for low overhead. Maximum IP datagram size was increased to match FS block size, to allow send/receive of entire file blocks. Some newer implementations use TCP as a transport. • The NFS protocol is a set of message formats and types. Client issues a request message for a service operation. Server performs requested operation and returns a reply message with status and (perhaps) requested data.

NFS Vnodes Vnodes NFS The NFS protocol has an operation type for (almost) every vnode operation, with similar arguments/results. struct nfsnode* np = VTONFS(vp); syscall layer VFS nfs_vnodeops NFS NFS client stubs server nfsnode RPC/UDP UFS network The nfsnode holds information needed to interact with the server to operate on the file.