MTS: Bringing Multi-Tenancy to Virtual Networking Kashyap - PowerPoint PPT Presentation

MTS: Bringing Multi-Tenancy to Virtual Networking Kashyap Thimmaraju, Saad Hermak, Gábor Rétvári and Stefan Schmid USENIX Annual Technical Conference 2019 July 11, Renton, Washington, USA

Virtual Networks Using Virtual Switches VM VM VM VM VM VM $_ $_ $_ $_ $_ $_ Host OS Host OS 2

Virtual Networks Using Virtual Switches VM VM VM VM VM VM $_ $_ $_ $_ $_ $_ Virtual Switch Host OS Host OS 3

Virtual Networks Using Virtual Switches VM VM VM VM VM VM $_ $_ $_ $_ $_ $_ Host OS Host OS 4

Virtual Networks Using Virtual Switches VM VM VM VM VM VM $_ $_ $_ $_ $_ $_ Broadcast | Multicast | Unicast | Tunnel 1. Red 2. Blue 3. Green Host OS Host OS 5

More Than 20 Virtual Switches Most emphasis has been on performance and flexibility 6

Security Weaknesses of Virtual Switches 7

Processes VM VM VM Untrusted Data $_ $_ $_ Host OS Host OS A malicious VM can send arbitrary packets to the virtual switch 8

Privileged Packet VM VM VM VM Processing $_ $_ $_ $_ User Kernel Host OS Host OS Oftentimes runs in the kernel for performance 9

Single Point of Failure Virtual network configurations are complex Screenshot from Karim Elatov’s blog: https://elatov.github.io/2018/01/openstack-ansible-and-kolla-on-ubuntu-1604/#5-packet-goes-from-ovs-inte 10 gration-bridge-br-int-to-ovs-tunnel-bridge-br-tun

Single Point of VM VM VM Failure $_ $_ $_ Host OS Host OS Mis-configurations could lead to security issues 11

Co-Located with VM VM VM VM the Host OS $_ $_ $_ $_ Host OS Host OS The consequence of a compromise can be severe, e.g., break out of VM isolation 12

Exploiting Virtual VM VM VM Switches in the $_ $_ $_ Cloud Host OS Host OS SOSR’18: Remote-Code Exection OvS Con’19: Cross Tenant DoS 13

Outline Motivation ● MTS ● Evaluation ● Scalability ● Pros and Cons ● Conclusion ● 14

MTS: Multi-Tenant Switch 15

Least Privilege VM Host VM Virtual Switch $_ $_ $_ 1. Processes untrusted data 2. Privileged packet processing 3. Single point of failure 4. Co-located with the Host OS 16

Least Common VM Host VM Mechanism $_ $_ $_ 1. Processes untrusted data 2. Privileged packet processing 3. Single point of failure 4. Co-located with the Host OS 17

Extra Security VM Host VM Boundary $_ $_ $_ 1. Processes untrusted data 2. Privileged packet processing 3. Single point of failure 4. Co-located with the Host OS 18

Complete VM Host VM Mediation $_ $_ $_ T T In/Out Gw In/Out Gw PF VF VF VF VF VF VF 1. Processes untrusted data 2. Privileged packet processing L2 Switch in NIC 3. Single point of failure 4. Co-located with the Host OS SR-IOV NIC 19

Evaluation 20

Experimental Setup Resources ● Traffic Patterns ● & Factors Mellanox ConnectX4, Open vSwitch, DPDK, QEMU, KVM More details in the paper 21

Host OS Shared Resources Host OS pinned to 1 core ● CPU All vswitch-VMs pinned to 1 ● core Each Tenant VM got ● dedicated cores (not shown here) 22

Traffic Patterns VM VM VM In Out In Out In Out NIC NIC NIC p2p p2v v2v 23

B 1 4 2 Baseline vs MTS A S V V V Packet Processing E S S S Throughput L - - - I V V V Comparison N M M M E 64 byte UDP packets Roughly the same in p2p MTS is ~2x Baseline in p2v and v2v 24

Baseline vs MTS Packet Processing Throughput Comparison 64 byte UDP packets Roughly the same in p2p MTS is ~2x Baseline in p2v and v2v 25

Baseline vs MTS Network Application Throughput MTS beats Baseline in Apache and Memcached 26

1+ Physical Core 4x Network Isolation 1.5-2x Throughput 27

Scaling MTS 28

Containers in VMs Work in progress ● The packets per second throughput is ● the same as running it in a VM for 4 containers Can run 12 vswitches spread across 4 ● VMs Faced an issue with libvirt when ● adding 40 VFs to 16 vswitches spread Real cloud systems can host more across 4 VMs. The interfaces do not than just 4 tenants on a server appear in the VM although the configuration is present. 29

Pros and Cons 30

Limitations PCIe bus could become a bottleneck ● which our evaluation did not reveal The number of VFs on the NIC ● No clean solution for live migration of ● VMs with VFs 31

Pricing State-of-the-art MTS Broadcast | Broadcast | Broadcast | Multicast | Multicast | Multicast | Unicast Unicast Unicast 1. Red $ $ Charge for CPU cycles used by the 2. Blue $ tenant-specific virtual switch 32

Tenant Specific State-of-the-art MTS Virtual Switch Software Broadcast | Broadcast | Broadcast | Multicast | Multicast | Multicast | Unicast Unicast Unicast 1. Red 1. Reduce parsing logic 2. Blue 2. Support tenant-specific features 33

Conclusion 34

Key Takeaways 1. Many virtual switches can be exploited to compromise Host and Network isolation 2. MTS is based on secure design principles that addresses security weakness of existing designs 3. MTS with SR-IOV offers security and performance for modest resources Our scripts and data are on github www.github.com/securedataplane High High Mid Security Performance Resource

Backup 36

Protocol Growth for OvS 37

Complex & Manual Protocol Parsers Virtual switches have to support an increasing number of protocols over time 38

Vswitch Table Analysis 39

So Many Virtual Switches More than 20 40

So Many Virtual Switches More than 20 41

So Many Virtual Switches More than 20 42

Ingress Traffic Flow Example 43

VM VM HOST $_ $_ $_ IN/ IN/ GW T GW T PF OUT OUT VF VF VF VF VF VF L2 Switch in NIC 44

VM VM HOST $_ $_ $_ IN/ IN/ GW T GW T PF OUT OUT VF VF VF VF VF VF L2 Switch in NIC Packet destined to VM $_ 45

VM VM HOST $_ $_ $_ IN/ IN/ GW T GW T PF OUT OUT VF VF VF VF VF VF L2 Switch in NIC MAC address of the vswitch VF IP address of VM $_ 46

VM VM HOST $_ $_ $_ IN/ IN/ GW T GW T PF OUT OUT VF VF VF VF VF VF L2 Switch in NIC 47

VM VM HOST $_ $_ $_ IN/ IN/ GW T GW T PF OUT OUT VF VF VF VF VF VF L2 Switch in NIC 48

VM VM HOST $_ $_ $_ IN/ IN/ GW T GW T PF OUT OUT VF VF VF VF VF VF L2 Switch in NIC 49

VM VM HOST $_ $_ $_ IN/ IN/ GW T GW T PF OUT OUT VF VF VF VF VF VF L2 Switch in NIC 50

VM HOST $_ $_ IN/ IN/ GW T GW T PF OUT OUT VF VF VF VF VF VF L2 Switch in NIC 51

Pricing 52

How it Helps Pricing Can charge for compute and memory used by the vswitch 53

Latency 54

Baseline vs MTS Latency Comparison 64 byte UDP packets Baseline is faster than MTS in p2p MTS is faster than Baseline in p2v and v2v 55

Baseline vs MTS Latency Comparison Baseline is faster than MTS in p2p MTS is faster than Baseline in p2v and v2v 56

Baseline vs MTS Latency Comparison Baseline is faster than MTS in p2p MTS is faster than Baseline in p2v and v2v 57

Baseline vs MTS Latency Comparison Baseline is faster than MTS in p2p MTS is faster than Baseline in p2v and v2v 58