Impact of Advanced Virtualization S. Freitag Technologies on Grid - PowerPoint PPT Presentation

Virtualization in Grid Computing Centers Impact of Advanced Virtualization S. Freitag Technologies on Grid Computing Centers Virtualization International Symposium on Grid Computing 2009 Impact on Grid Computing Conclusion Stefan Freitag Robotics Research Institute Dortmund University of Technology 23. April 2009

Overview Virtualization in Grid Computing Centers S. Freitag 1 Virtualization Virtualization Impact on Grid Computing Conclusion 2 Impact on Grid Computing 3 Conclusion

Introduction Virtualization in Grid Computing Centers S. Freitag Virtualization Virtualization Impact on Resource Application Platform Grid Computing Conclusion Full Paravirtualized OS Level Network Storage Input/Output Figure: Types of Virtualization

Network VPN (Virtual Private Network) Virtualization in Grid Computing disjunct network partitions Centers gateway service for tunneling S. Freitag WAN for interconnect Virtualization Impact on Grid GW Computing GW LAN 2 Conclusion LAN 1 FW WAN FW GW LAN 3 FW

Network VLAN (Virtual Local Area Network) Virtualization in Grid Layer 2 (data link layer) construct, IEEE 802.1Q standard Computing Centers Devices on different physical LAN segments S. Freitag Communication as if on same physical LAN segment Virtualization (bcast domain) Impact on Configuration of VLAN through software Grid Computing Traffic shaping/ QoS Conclusion . . . S3 VLAN2 . . . . . . S2 S4 . . . LAN S1 VLAN1

Storage Virtualization in Grid decreasing storage costs ( GByte / $ ) Computing Centers increasing management complexity S. Freitag full SAN bandwidth for I/O requests Virtualization separation of data and metadata into different places Impact on Grid control unit: appliances or SAN switches Computing Conclusion 1 Control 2 1 4 3 1 6 4 1a Virtualization Virtualization Virtualization Control 1b 2 4 3 5 3 2

Platform Virtualization Full virtualized Environment in Grid Computing guest unaware of virtualization, run native OS Centers reduced performance because of hardware emulation S. Freitag Para virtualized Environment Virtualization management module (hypervisor or virtual machine Impact on monitor) operates with modified operating system Grid guest OS has much closer control of the underlying Computing hardware (security, influence on other VMs) Conclusion VM Management Application Application Guest OS Guest OS Application Application Virtual Virtual VM Hardware Hardware Management Guest OS Guest OS (modified) (modified) Virtualization Layer Host OS Host OS Virtualization Layer X86 Hardware X86 Hardware

Input/ Output (IOV) Virtualization Typical configuration: 4 to 6 I/O cards per server in Grid Computing Ethernet Centers Infiniband S. Freitag Fiber Channel Virtualization add I/O virtualization capability to PCIe Impact on disaggreation Grid Computing consolidation Conclusion virtualization Server NICs LAN /WAN SAN PCIe HBAs IOV Switch

Input/ Output (IOV) I/O consolidation Virtualization in Grid Computing 100% of server connected to Ethernet, only 20% to FC Centers FiberChannel over Ethernet (FCoE), pre-standard S. Freitag converged Network Adapter (C-NIC/ CNA) Virtualization benefit for compute centers Impact on Grid fewer adapters Computing power savings Conclusion cable management Server FCoE LAN /WAN Switch Ethernet Switch C−NIC SAN

Present situation Virtualization in Grid Computing Centers Platform and storage virtualization in use S. Freitag server consolidation Virtualization improved utilization of existing servers Impact on increased number of services per m 2 Grid Computing reducing TCO (maintenance,...) in the long-run Conclusion high availability run legacy applications managing mass-storage backends migration of virtual machines is bound to restrictions storage network reconfiguration

Platform Virtualization @ LRMS level Virtualization in Grid Computing Centers Job = Virtual Machine S. Freitag share same operations (start, stop, suspend, checkpointing) Scenarios: Virtualization Impact on user submits self-prepared virtual machine Grid on-demand creation of virtual machine by LRMS Computing Conclusion VM description as part of job specification LRMS schedules jobs and/ or virtual machines Not so good: black-box execution performance loss (MPI) compared to non-virtualized solution

IOV @ LRMS level Virtualization in Grid Computing Centers S. Freitag vNIC (virtual Network Interface Card) Virtualization vHBA (virtual Host Bus Adapter) Impact on Grid FC WWN used to identify to segregate devices for access Computing control Conclusion Virtualized environments: adapter’s WWN represents all DomUs (Xen) DomU migrates to new server, but WWN does not zoning and LUN masking (access control) ineffective vHBA gets own WWN which migrates with VM

Platform Virtualization @ Grid Level Virtualization in Grid Computing Centers S. Freitag Benefits Virtualization provide uniform environment to Grid users Impact on Grid Computing rapid on-demand provisioning of Grid nodes Conclusion non-interfering execution of multiple Grid middlewares decoupling of knowledge (grid operator/ grid user) job exchange, e.g. inter cluster/ Grid resources

Platform Virtualization @ Grid Level Virtualization in Grid Computing Centers S. Freitag Virtualization Drawback Impact on Grid resources run different virtualization software Computing Conclusion Xen, KVM, VMware, . . . creation of user environment for each resource

Virtual Appliances Virtualization in Grid Computing encapsulate user environment in a virtual appliance Centers S. Freitag virtual appliance = virtual machine + operating system Virtualization + application level software + MD5 Impact on Grid Computing Job = virtual machine virtual appliance Conclusion virtual appliance is already technology-dependent :-( better approach step 1: create operating system + application level software + MD5 step 2: make it technology-dependent

KIWI Virtualization in Grid idea: prepare once, run anywhere Computing Centers distribution independent, currently only SuSE supported S. Freitag centralized image description based on XML Virtualization output formats: xen, vmware, Amazon EC 2 , iso, . . . Impact on Grid adoption to Scientific Linux ongoing in Dortmund Computing Conclusion access via chroot access via loop additional software Unpacked Packed packages ready 2 serve Image Image Image description Figure: from: KIWI Cookbook

Conclusion & Outlook Virtualization in Grid Computing Centers Conclusion S. Freitag platform and storage virtualization widely spread Virtualization increased flexibility with I/O virtualization/ FCoE Impact on Grid Grids profit at LRMS and Grid layer from virtualization Computing Conclusion on-demand provisioning of virtual appliances Outlook Grid resource provides bare metal + virtualization layer provision of resources to more than one Grid infrastructure/ VO