Virtual machine images as structured data The Mirage image library - PowerPoint PPT Presentation

Virtual machine images as structured data The Mirage image library Glenn Ammons, Vasanth Bala, Todd Mummert, Darrell Reimer, Xiaolan Zhang IBM Research

Why image libraries? ● VM image puts app config in one place; image library puts enterprise config in one place ● Improve maintenance: scans, patches ● Permit analyses: search, mine, compare – What DBMSs do we use? Why did our webapp break? ● Image libraries: go beyond deploy/capture ● Provenance ● Version and access control ● Efficient, offline maintenance and analyses June 2011 IBM Research 2

VM images as structured data Image manifest Checkin Disk manifest Metadata VM image File contents FS manifest (in CAS) Checkout /etc/passwd /usr/bin/ls ... Unstructured Structured (hypervisor) (Mirage) June 2011 IBM Research 3

Experience: RC2 ● Problem: convert all images from Xen to KVM ● No downtime, low resource use, user transparency ● Must install kernels, kernel modules, change config. ● Iterative: fail → find bug → fix → try again ● Version control useful – Rollback – Comparisons for debugging ● Used virtual mount to speed conversion June 2011 IBM Research 4

Virtual mount % import kvm-kernel Image manifest % vmount IMAGE /mnt Disk manifest Metadata File contents FS manifest (in CAS) /boot/kernel /etc/network/foo ... Structured (Mirage) June 2011 IBM Research 5

Virtual mount % import kvm-kernel Image manifest % vmount IMAGE /mnt % replace-content \ /boot/kernel Disk manifest Metadata File contents FS manifest (in CAS) /boot/kernel /etc/network/foo ... Structured (Mirage) June 2011 IBM Research 6

Virtual mount % import kvm-kernel Image manifest % vmount IMAGE /mnt % replace-content \ /boot/kernel Disk manifest Metadata % vi /etc/network/foo File contents FS manifest (in CAS) /boot/kernel /etc/network/foo ... Structured (Mirage) June 2011 IBM Research 7

Experience: IBM Workload Deployer OS team MW team App team clone A.0 B.0 Install MW clone B.1 C.0 Install app C.1 June 2011 IBM Research 8

Experience: IBM Workload Deployer OS team MW team App team clone A.0 B.0 Update Install MW OS clone B.1 C.0 A.1 Install app C.1 June 2011 IBM Research 9

Experience: IBM Workload Deployer OS team MW team App team clone A.0 B.0 Update Install MW OS clone B.1 C.0 A.1 Patch w/ Install app diff(A.0, A.1) B.2 C.1 June 2011 IBM Research 10

Backup slides June 2011 IBM Research 11

Mirage architecture Clients Catalog Library Image manager services indexer Content-addressed store Database Storage June 2011 IBM Research 12

Mirage architecture Clients Deduplication Deduplication Garbage collection Garbage collection Catalog Library Image Structure-aware Structure-aware manager services indexer Optimized for images Optimized for images Content-addressed store Database Storage June 2011 IBM Research 13

Mirage architecture Clients Converts to and from Mirage format Converts to and from Mirage format Catalog Library Image Flexible plugin architecture Flexible plugin architecture manager services indexer Content-addressed store Database Storage June 2011 IBM Research 14

Mirage architecture Clients Maintains metadata Maintains metadata State (active/deleted) State (active/deleted) Provenance tree Provenance tree Catalog Library Image Access control Access control manager services indexer Version control Version control Content-addressed store Database Storage June 2011 IBM Research 15

Mirage architecture Clients Checkin/checkout Checkin/checkout Virtual mount Virtual mount Describe, list, Catalog Library Image Describe, list, compare, search manager services indexer compare, search Admin functions Admin functions Content-addressed store Database Storage June 2011 IBM Research 16

Reducing translation costs ● Runtime translation costs reduced by ● Structure-aware CAS (faster lookups) ● Virtual mount (avoids translation) ● Delta deployment (exploits sharing) ● Hybrid indexing reduces dev. Costs ● Offloads grotty details to backup/restore tools ● See paper for details June 2011 IBM Research 17