Dan Williams & Tiffany Kasanicky /dev/pmem0 /dev/ndblk0.0s - PowerPoint PPT Presentation

Linux Foundation Vault 2016 Dan Williams & Tiffany Kasanicky

/dev/pmem0

/dev/ndblk0.0s

Managing Persistent Memory Application IXPDIMM Applications (PMEM Aware) (Management Stack) (Traditional) NDCTL LIBNDCTL EXT4, XFS FS FS SYSFS USERSPACE IOCTL DAX LIBNVDIMM LIBNVDIMM PMEM0 PMEM1S BLK1S REGION BUS BTT(PMEM) BTT(BLK) PMEM E820 ACPI.NFIT PMEM SYSTEM PMEM SYSTEM BLK REGION ADDRESS REGION MEMORY REGION SPACE PERSISTENT MEMORY (PMEM)

Namespaces # ndctl list --namespaces --type=pmem { "dev":"namespace6.0", "mode":"raw", "size":33554432, "uuid":"70a6adce-722e-4ab8-b698-35eaea9750b3", "blockdev":"pmem6" }

Namespaces # ndctl list --namespaces --type=pmem { "dev":"namespace6.0", "mode":"raw", "size":33554432, "uuid":"70a6adce-722e-4ab8-b698-35eaea9750b3", "blockdev":"pmem6" }

Namespaces # ndctl list --namespaces --type=pmem { "dev":"namespace6.0", "mode":"raw", "size":33554432, "uuid":"70a6adce-722e-4ab8-b698-35eaea9750b3", "blockdev":"pmem6" “Namespace”: Persistent memory capacity } accessed through a PMEM or BLK disk device

Namespaces # ndctl list --namespaces --type=blk { "dev":"namespace0.0", "mode":"sector", "uuid":"5ce6c34a-88b0-469a-86f5-ea8f462a68ca", "sector_size":4096, "blockdev":"ndblk0.0s" }

Why BLK?

Namespace Modes SECTOR MEMORY RAW

Namespace Modes SECTOR MEMORY RAW • Byte-addressable • Limited DAX

Namespace Modes SECTOR MEMORY RAW • Software atomic • Byte-addressable sector update • Limited DAX semantics • Configurable sector size. • Applicable to PMEM and BLK namespaces

Namespace Modes SECTOR MEMORY RAW • Software atomic • Enables full DAX • Byte-addressable sector update (DMA/RDMA/Direct- • Limited DAX semantics I/O) • Configurable sector • Only applicable to size. PMEM namespaces • Applicable to PMEM and BLK namespaces

“Memory” Mode DAX: Direct I/O Application DAX ‘ buf ’ Disk file ‘ fd ’ pmem0 sda

“Memory” Mode DAX: Direct I/O Application Core Kernel buf struct page DAX ‘ buf ’ bio Disk file ‘ fd ’ fd (bdev, sector)

“Memory” Mode DAX: Direct I/O Application Core Kernel Block Layer + Disk Driver buf struct page DAX ‘ buf ’ bio sgl bio Disk file ‘ fd ’ fd (bdev, sector) disk driver DMA pmem0 sda

“Memory” Mode DAX: Direct I/O Application Core Kernel Block Layer + Disk Driver X buf struct page DAX ‘ buf ’ bio sgl bio Disk file ‘ fd ’ fd (bdev, sector) disk driver DMA pmem0 sda

“Memory” Mode DAX: Considerations • struct page array is 64-bytes per 4K page (16GB per 1TB) # ndctl create-namespace --reconfig=namespace9.0 --mode=memory -- map=dev --force

ndctl/libndctl < IXPDIMM ndctl/libndctl: low level generic primitives IXPDIMM: Coherent / comprehensive management stack

Tiffany Kasanicky tiffany.j.kasanicky@intel.com

Persistent Memory Management Management Tools Application Application Application User Standard Raw Standard Standard Load/Store Device Access File API File API pmem-aware MMU File System FS (DAX) Mappings Kernel NFIT/NVDIMM Driver Block I/O BIOS ACPI (NFIT) _DSM SMBIOS NVDIMM FW Persistent Memory 23

Components Enterprise Tools Basic Management • Pegasus/SFCB monitor cim cli End-user provisioning and Syslog  management via CLI User core Enabling • api db SFCB/Pegasus CIM provider for  remote access and 3rd party integration libndctl C library for programmatic access  and abstraction Kernel NFIT NVDIMM Driver Monitoring • Daemon for health monitoring  BIOS ACPI (NFIT) _DSM SMBIOS 24 3

NVDIMM Management NVDIMM NVDIMM NVDIMM Sensors & Settings: Discovery : Slot 0 Thermal, wear, spare, DRAM + NVDIMM topology, power, errors identifying information, capabilities DRAM DRAM DRAM Slot 1 CH 0 CH 1 CH 2 iMC Performance: Bytes read/written, host Memory Configuration: CPU and block read/writes Volatile and persistent iMC partitions, interleave settings CH 3 CH 4 CH 5 DRAM DRAM DRAM Slot 1 Instrumentation : FW update, SW versioning, Namespaces : data-at-rest security, FW Create/delete/inventory settings and policies namespaces NVDIMM NVDIMM NVDIMM Slot 0 DRAM DRAM 25

Discovery DRAM/NVDIMM Topology • SMBIOS Type 17 (memory device) data  NVDIMM socket, memory controller, channel population  Aggregated Memory Resources • Capabilities • Platform BIOS, NVDIMM, FW, SW  NVDIMM Information • Identifying - serial number, model number, device ID  Status – manageability, health, security  Provisioning – partitioning, attributes, state  26

Memory Provisioning 2 1. Create memory allocation goal User 1 5 2. Reboot Management Tools 3. BIOS writes NFIT 4. Driver reads NFIT Kernel NFIT/NVDIMM Driver 5. Create namespace 4 6. Mount file system 3 BIOS _DSM MRC NFIT NVDIMM NVDIMM FW Metadata PM NVDIMM 27

Diagnostics Quick Health Check • Platform Configuration Check • Security Check • FW Consistency and Settings Check • Persistent Memory Metadata Check • Address Range Scrub Results • 28

Packages Compone Package Repository nt cli ixpdimm-cli https://github.com/01org/IXPDIMMSW cim libixpdimm-cim https://github.com/01org/IXPDIMMSW core libixpdimm-core https://github.com/01org/IXPDIMMSW api libixpdimm-api https://github.com/01org/IXPDIMMSW libixpdimm-api-devel monitor ixpdimm-monitor https://github.com/01org/IXPDIMMSW cli framework libintelnvm-cli https://github.com/01org/intelnvmclilibrary libintelnvm-cli-devel i18n framework libintelnvm-i18n https://github.com/01org/intelnvmi18nlibrary libintelnvm-i18n-devel cim framework libintelnvm-cim https://github.com/01org/intelnvmcimlibrary libintelnvm-cim-devel 29

Distribution Plan Open source 3-clause BSD license • Hosted on 01.org/github – Intel maintainers • https://01.org/ixpdimm-sw  https://01.org/intel-nvm-cim-library  https://01.org/intel-nvm-cli-library  https://01.org/intel-nvm-i18n-library  Targeted OS Distributions: • RHEL/Fedora  SLES/OpenSuSE  30

tiffany.j.kasanicky@intel.com