Machine Virtualization: Efficient Hypervisors, Stealthy Malware Muli Ben-Yehuda Technion & Hypervisor Technologies and Consulting Ltd Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 1 / 21
Background: x86 machine virtualization Running multiple different unmodified operating systems Each in an isolated virtual machine Simultaneously On the x86 architecture Many uses: live migration, record & replay, testing, . . . , security Foundation of IaaS cloud computing Used nearly everywhere Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 2 / 21
x86 virtualization primer How does it work? Popek and Goldberg’s virtualization model [Popek74]: Trap and emulate Privileged instructions trap to the hypervisor Hypervisor emulates their behavior Without hardware support With hardware support Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 3 / 21
What is a rootkit? First you take control. How? Then you hide to avoid detection and maintain control. How? Usual methods are ugly and intrusive: easy to detect! Can rootkit authors do better? Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 4 / 21
Hypervisor-level rootkits Hypervisors have full control over the hardware Hypervisors can trap any operating system event Code can enter hypervisor-mode at any time Bluepill: run the rootkit as the hypervisor Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 5 / 21
Bluepill: a hypervisor level rootkit [Rutkowska06] Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 6 / 21
Recursive Bluepill Bluepill installs itself on the fly Bluepill is now the hypervisor Reminder: x86 only supports one hypervisor in hardware So how can you bluepill bluepill? Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 7 / 21
The Turtles project: Nested x86 Virtualization Efficient nested virtualization for Intel x86 based on KVM Runs multiple guest hypervisors and VMs “The Turtles Project: Design and Implementation of Nested Virtualization”, [Ben-Yehuda10] Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 8 / 21
What is the Turtles project? (cont’) Nested VMX virtualization for nested CPU virtualization Multi-dimensional paging for nested MMU virtualization Multi-level device assignment for nested I/O virtualization Micro-optimizations to make it go fast + + = Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 9 / 21
Theory of nested CPU virtualization Trap and emulate[PopekGoldberg74] ⇒ it’s all about the traps Single-level (x86) vs. multi-level (e.g., z/VM) Single level ⇒ one hypervisor, many guests Turtles approach: L 0 multiplexes the hardware between L 1 and L 2 , running both as guests of L 0 —without either being aware of it (Scheme generalized for n levels; Our focus is n=2) Guest Guest L2 L2 L2 Guest Guest Guest Guest Hypervisor Guest Hypervisor Guest L1 L1 L2 L2 L0 Host Hypervisor L0 Host Hypervisor Hardware Hardware Multiple logical levels Multiplexed on a single level Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 10 / 21
Detecting hypervisor-based rootkits Bluepill authors claim “undetectable” “Compatibility is Not Transparency: VMM Detection Myths and Realities” [Garfinkel07] Hardware discrepancies Resource-sharing attacks Timing attacks: PCI register access, page-faults on MMIO access, cpuid timing vs. nops Can you trust time? Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 11 / 21
The Dual Role of a Hypervisor Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 12 / 21
Background: interrupts IDTR IDT IDT Entry Vector 1 Address Vector 2 IDT Entry … Limit IDT Entry Vector n IDT I nterrupt Interrupt handlers R egister D escriptor T able I/O devices raise interrupts CPU temporarily stops the currently executing code CPU jumps to a pre-specified interrupt handler Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 13 / 21
Interrupts as an Attack Vector Follow the White Rabbit [Rutkowska11] Tell the device to generate “interesting” interrupts Attack: fool the CPU into SIPI Attack: syscall/hypercall injection In interrupt-based attacks an untrusted guest generates malicious interrupts which are handled in host mode Protect: handle interrupts in guest—not host—mode Serve: bare-metal performance! Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 14 / 21
ELI: Exitless Interrupts guest (a) Baseline Physical Interrupt hypervisor Interrupt Interrupt Completion Injection guest ELI (b) delivery Interrupt hypervisor Completion guest ELI delivery & (c) completion hypervisor (d) bare-metal (time) ELI: direct interrupts for unmodified, untrusted guests “ELI: Bare-Metal Performance for I/O Virtualization”, Gordon12 Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 15 / 21
ELI: delivery Shadow IDT Guest Interrupt IDT Handler Assigned #NP Interrupt P=0 IDT Entry IDTR Limit Shadow Handler P=1 IDT IDT Entry VM Non-assigned … Interrupt (#NP/#GP exit) ELI Delivery Hypervisor #NP P=0 IDT Entry Physical Interrupt #GP IDT Entry All interrupts are delivered directly to the guest Host and other guests’ interrupts are bounced back to the host . . . without the guest being aware of it Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 16 / 21
ELI: signaling completion Guests signal interrupt completions by writing to the Local Advance Programmable Interrupt Controller (LAPIC) End-of-Interrupt (EOI) register Old LAPIC: hypervisor traps load/stores to LAPIC page x2APIC: hypervisor can trap specific registers Signaling completion without trapping requires x2APIC ELI gives the guest direct access only to the EOI register Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 17 / 21
ELI: threat model Threats: malicious guests might try to: keep interrupts disabled signal invalid completions consume other guests or host interrupts Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 18 / 21
ELI: protection VMX preemption timer to force exits instead of timer interrupts Ignore spurious EOIs Protect critical interrupts by: Delivering them to a non-ELI core if available Redirecting them as NMIs → unconditional exit Use IDTR limit to force #GP exits on critical interrupts Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 19 / 21
Conclusions Machine virtualization be used for good, or evil How do you protect and serve? Happy hacking! Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 20 / 21
Questions? muli@cs.technion.ac.il mulix@hypervisorconsulting.com Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 21 / 21
Recommend
More recommend
