Revisiting Virtual File System for Metadata Optimized Non-Volatile - PowerPoint PPT Presentation

���� ���� ������������� Revisiting Virtual File System for Metadata Optimized Non-Volatile Main Memory File System Ying Wang , Dejun Jiang, Jin Xiong Institute of Computing Technology, CAS University of Chinese Academy of Sciences

Outline • Background & Motivation • Design – Cachelet for metadata caching – Global hash based metadata index – Metadata scalability • Evaluation • Summary 2

Background • Non-Volatile Main Memories(NVMMs) provide low latency , high bandwidth , byte-addressable and persistent storage – PCM, MRAM, RRAM, 3D Xpoint[1] • Intel releases Optane DC Persistent Memory (Optane PMM) [6] R lat. W lat. R BW. W BW. DRAM 60ns 69ns 20 GB/s ~15 GB/s Optane PMM 305ns 81ns ~6GB/s ~2GB/s NVMe SSD 120us 30us 2GB/s 500MB/s HDD 10ms 10ms 0.1GB/s 0.1GB/s [1] What is Intel Optane DC Persistent Memory. Intel. [2] Condit, SIGOPS 2009 [3] Wu, SC 2011 [4] Dulloor, EuroSys 2014 [5] Haris, EuroSys 2014 [6] The data from our evaluation and the paper of “Basic Performance Measurements of the Intel Optane DC Persistent 3 Memory Module”

Background • Non-Volatile Main Memories(NVMMs) provide low latency , high bandwidth , byte-addressable and persistent storage – PCM, MRAM, RRAM, 3D Xpoint[1] • Intel releases Optane DC Persistent Memory (Optane PMM) • File system can be directly built on memory – Software has become the main factor affecting the file system performance and scalability [2,3,4,5] [6] R lat. W lat. R BW. W BW. DRAM 60ns 69ns 20 GB/s ~15 GB/s File system Optane PMM 305ns 81ns ~6GB/s ~2GB/s I/O Memory bus NVMe SSD 120us 30us 2GB/s 500MB/s HDD 10ms 10ms 0.1GB/s 0.1GB/s CPU NVMM [1] What is Intel Optane DC Persistent Memory. Intel. [2] Condit, SIGOPS 2009 [3] Wu, SC 2011 [4] Dulloor, EuroSys 2014 [5] Haris, EuroSys 2014 [6] The data from our evaluation and the paper of “Basic Performance Measurements of the Intel Optane DC Persistent 3 Memory Module”

Background • Existing kernel-level NVMM file systems ,�� �32�0�� ������ ���� ,�� ��� �� ���� ,�� ��11334� ��43��� ���� � ,�� ��40�� ��43��� ����

Background • Existing kernel-level NVMM file systems – Remove page cache, generic block layer and I/O scheduler layer ,�� �32�0�� ������ ���� ,�� ��� �� ���� ,�� ��11334� ��43��� ���� � ,�� ��40�� ��43��� ����

Background • Existing kernel-level NVMM file systems – Remove page cache, generic block layer Application and I/O scheduler layer USER – Retain virtual file system(VFS) [1,2,3,4] Kernel VFS NVMM FS NVMM ,�� �32�0�� ������ ���� ,�� ��� �� ���� ,�� ��11334� ��43��� ���� � ,�� ��40�� ��43��� ����

Background • Existing kernel-level NVMM file systems – Remove page cache, generic block layer Application and I/O scheduler layer USER – Retain virtual file system(VFS) [1,2,3,4] Kernel VFS • dentry -> dcache – Speed up path lookup and maintain a unified namespace NVMM FS • inode -> icache – Speed up file metadata access NVMM ,�� �32�0�� ������ ���� ,�� ��� �� ���� ,�� ��11334� ��43��� ���� � ,�� ��40�� ��43��� ����

Background • File metadata operation type – Lookup – Update �

Background • File metadata operation type – Lookup • VFS warm cache (Cache hit) • VFS cold cache (Cache miss) – Update �

Background • File metadata operation type – Lookup • VFS warm cache (Cache hit) Only lookup in VFS • VFS cold cache (Cache miss) – Update �

Background • File metadata operation type – Lookup • VFS warm cache (Cache hit) Only lookup in VFS Lookup in both VFS and physical • VFS cold cache (Cache miss) FS, and builds VFS cache – Update �

Background • File metadata operation type – Lookup • VFS warm cache (Cache hit) Only lookup in VFS Lookup in both VFS and physical • VFS cold cache (Cache miss) FS, and builds VFS cache Update both VFS and PFS – Update �

Background • File metadata operation type – Lookup • VFS warm cache (Cache hit) Only lookup in VFS Lookup in both VFS and physical • VFS cold cache (Cache miss) FS, and builds VFS cache Update both VFS and PFS – Update Types Syscalls Example Lookup 20 open(lookup), stat, access Update 29 open(create), remove, rename, chown �

Motivation • The latency of NVMM is close to DRAM and supports high concurrent access �

Motivation • The latency of NVMM is close to DRAM and supports high concurrent access – The metadata performance of physical file system is close to VFS �

Motivation • The latency of NVMM is close to DRAM and supports high concurrent access – The metadata performance of physical file system is close to VFS – File system requires high concurrent software support �

Motivation • The latency of NVMM is close to DRAM and supports high concurrent access – The metadata performance of physical file system is close to VFS – File system requires high concurrent software support • Traditional metadata management are not suitable for NVMM file system �

Motivation • The latency of NVMM is close to DRAM and supports high concurrent access – The metadata performance of physical file system is close to VFS – File system requires high concurrent software support • Traditional metadata management are not suitable for NVMM file system – Two-layer metadata lookup and maintenance overhead – Low-scalability metadata operations �

Motivation • Tow layer metadata lookup and maintenance – The latency of NVMM is closed to DRAM, VFS and physical file system maintain a copy of metadata respectively VFS PFS 70% Metadata % total execution time 60% 50% 40% 30% 20% 10% 0% 4KB 16KB 4KB 16KB 4KB 16KB 4KB 16KB � read_ext4-dax read_NOVA write_ext4-dax write_NOVA

Motivation • Tow layer metadata lookup and maintenance – The latency of NVMM is closed to DRAM, VFS and physical file system maintain a copy of metadata respectively • In ext4-dax, metadata overhead accounts for 49.1%, which VFS lookup overhead accounts for 21.2% VFS PFS 70% Metadata % total execution time 60% 50% 40% 30% 20% 10% 0% 4KB 16KB 4KB 16KB 4KB 16KB 4KB 16KB � read_ext4-dax read_NOVA write_ext4-dax write_NOVA

Motivation • Low-scalability metadata operations 20 create file delete file Throughput (M ops/s) lookup file Cold lookup file Warm 15 10 5 0 1 4 8 12 16 20 24 �

Motivation • Low-scalability metadata operations – All metadata operations that need to be in the physical file system are locked in the parent directory 20 create file delete file Throughput (M ops/s) lookup file Cold lookup file Warm 15 10 5 0 1 4 8 12 16 20 24 �

Motivation • Low-scalability metadata operations – All metadata operations that need to be in the physical file system are locked in the parent directory • Limit the scalability of file system metadata operations • Create file, delete file 20 create file delete file Throughput (M ops/s) lookup file Cold lookup file Warm 15 10 5 0 1 4 8 12 16 20 24 �

Motivation • Low-scalability metadata operations – All metadata operations that need to be in the physical file system are locked in the parent directory • Limit the scalability of file system metadata operations • Create file, delete file – When metadata is added/deleted in VFS, the VFS lock limits scalability 20 create file delete file Throughput (M ops/s) lookup file Cold lookup file Warm 15 10 5 0 1 4 8 12 16 20 24 �

Motivation • VFS results in two copy metadata overhead and limits metadata scalability on NVMM file system 14 VFS PFS 12 10 Time (us) 8 6 4 2 0 open stat remove open stat remove � Cold cache Warm cache

Motivation • VFS results in two copy metadata overhead and limits metadata scalability on NVMM file system – Can we directly delete the metadata cache in VFS? 14 VFS PFS 12 10 Time (us) 8 6 4 2 0 open stat remove open stat remove � Cold cache Warm cache

Motivation • VFS results in two copy metadata overhead and limits metadata scalability on NVMM file system – Can we directly delete the metadata cache in VFS? • Compared to VFS having cached metadata, removing VFS cache has low performance 14 VFS PFS 12 10 Time (us) 8 6 4 2 0 open stat remove open stat remove � Cold cache Warm cache

Motivation • VFS results in two copy metadata overhead and limits metadata scalability on NVMM file system – Can we directly delete the metadata cache in VFS? • Compared to VFS having cached metadata, removing VFS cache has low performance 14 VFS PFS 12 10 Time (us) 8 6 4 2 0 open stat remove open stat remove � Cold cache Warm cache

Contribution • DirectFS ��