Prism: A Proxy Architecture for Datacenter Networks Yutaro Hayakawa - PowerPoint PPT Presentation

Prism: A Proxy Architecture for Datacenter Networks Yutaro Hayakawa (Keio University) Lars Eggert (NetApp) Michio Honda (NEC Laboratories Europe) Douglas Santry (NetApp) 1

Background • TCP proxy plays important roles in modern cloud services • Web cache • Application level firewall • Application level load balancer 2

Problem • Aggregated bandwidth of backends are always constrained by proxy bandwidth 3

Problem • Aggregated bandwidth of backends are always constrained by proxy bandwidth Backend1 Backend2 Backend3 4

Problem • Aggregated bandwidth of backends are always constrained by proxy bandwidth Backend1 Proxy Backend2 Backend3 5

Problem • Aggregated bandwidth of backends are always constrained by proxy bandwidth Backend1 Proxy 10G Backend2 10G 10G Backend3 6

Problem • Aggregated bandwidth of backends are always constrained by proxy bandwidth Backend1 Proxy 10G Backend2 10G 10G 10G Backend3 7

Problem • Aggregated bandwidth of backends are always constrained by proxy bandwidth Backend1 Proxy 10G Backend2 10G 10G 10G Backend3 Total Bandwidth for Backends == 10G 8

Why does it matter? • This problem is significant especially at the edge of today’s datacenter topology 9

Example : HTTP/1.1 • Simple web system • One frontend proxy and multiple backend server 40Gbps Links 10Gbps Links Frontend ・・・ ・・・ Backend Backend Backend Proxy 10

Example : HTTP/1.1 Client’s HTTP GET 40Gbps Links 10Gbps Links Frontend Frontend ・・・ Backend Backend Backend ・・・ Proxy Proxy 11

Example : HTTP/1.1 40Gbps Links Forward 10Gbps Links To backend Frontend Frontend ・・・ ・・・ Backend Backend Backend Proxy Proxy 12

Example : HTTP/1.1 40Gbps Links Return 10Gbps Links HTTP OK Frontend Frontend ・・・ ・・・ Backend Backend Backend Proxy Proxy 13

Example : HTTP/1.1 40Gbps Links 10Gbps Links Forward it to client Frontend Frontend ・・・ ・・・ Backend Backend Backend Proxy Proxy 14

Example : HTTP/1.1 40Gbps Links 10Gbps Links Frontend ・・・ ・・・ Backend Backend Backend Proxy 15

Example : HTTP/1.1 Problem: All incoming/outgoing traffic go through here 40Gbps Links 10Gbps Links Frontend ・・・ ・・・ Backend Backend Backend Proxy 16

How can we solve this problem? Outgoing traffic from backend bypass frontend proxy 40Gbps Links 10Gbps Links Frontend Frontend ・・・ ・・・ Backend Backend Backend Proxy Proxy 17

Related work • L4 load balancer • Maglev[1], Ananta[2], Duet[3] Client Talk directly Back Back Back L4 LB end end end Balance TCP connect request [1] D. E. Eisenbud et al. Maglev: A Fast and Reliable Software [4] M. Alizadeh et al. CONGA: distributed Network Load Balancer. Proc. usenix nsdi , 2016. congestion-aware load balancing for datacenters. [2] P. Patel et al. Ananta: Cloud Scale Load Balancing. Proc. acm sigcomm , 2013. Proc. Acm sigcomm. Chicago, IL, USA, 2014, pp. 503–514. [3] R. Gandhi et al. Duet: Cloud Scale Load Balancing with 18 Hardware and Software. Proc. acm sigcomm , 2014.

Prism Architecture Legacy Proxy Client 40Gbps Links L2 Switch 10Gbps Links Back Back Back Proxy end end end 19

Prism Architecture Legacy Proxy Prism Client Client Programmable Switch(e.g.P4) 40Gbps Links 40Gbps Links L2 Switch 10Gbps Links 10Gbps Links Back Back Back Switch Proxy Custom Custom Custom end end end Controller Backend Backend Backend (Proxy) 20

Prism Architecture Basic idea: Offloading forwarding of application protocol payload to a switch Legacy Proxy Prism Client Client Programmable Switch 40Gbps Links 40Gbps Links L2 Switch 10Gbps Links 10Gbps Links Back Back Back Switch Proxy Custom Custom Custom end end end Controller Backend Backend Backend (Proxy) 21

Prism Architecture Basic idea: Offloading application protocol payload forwarding to a switch => the Switch Controller (Proxy) only handles header exchange Legacy Proxy Prism Client Client Programmable Switch 40Gbps Links 40Gbps Links L2 Switch 10Gbps Links 10Gbps Links Back Back Back Switch Proxy Custom Custom Custom end end end Controller Backend Backend Backend (Proxy) 22

How does it work? Example Behavior Client HTTP/1.1 40Gbps Links Programmable Switch 10Gbps Links Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 23

How does it work? Client and Switch Controller establish Client TCP connection 40Gbps Links Programmable Switch 10Gbps Links Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 24

How does it work? Client sends request to Client switch controller GET /example/index.html 40Gbps Links Programmable Switch 10Gbps Links Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 25

How does it work? Switch controller selects Client backend 40Gbps Links Programmable Switch 10Gbps Links Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 26

How does it work? Switch controller injects a rule to the switch Client which rewrites fields of packets from backend 40Gbps Links Programmable Switch 10Gbps Links Rule Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 27

How does it work? Switch controller sends client request to Client backend with some TCP metadata 40Gbps Links Programmable Rule Switch Sequence Number ACK Number 10Gbps Links Client’s request TCP options… Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 28

How does it work? Backend migrate TCP connection from Client information given by Switch controller 40Gbps Links Programmable Rule Switch Sequence Number 10Gbps Links ACK Number Client’s request TCP options… Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 29

How does it work? Backend and client communicate directly Client via migrated TCP connection 40Gbps Links Programmable Rule Switch HTTP OK 10Gbps Links and Payload data Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 30

How does it work? Backend send number of bytes sent to client to Client switch controller 40Gbps Links Programmable Rule Switch 10Gbps Links I sent N bytes Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 31

How does it work? Switch controller removes the rule from Client the switch Rule 40Gbps Links Programmable Switch 10Gbps Links Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 32

How does it work? Switch controller waits for next request or FIN Client packet 40Gbps Links Programmable Switch Waiting for FIN or next 10Gbps Links request Switch Custom Custom Custom ・・・ ・・・ Controller Backend Backend Backend (Proxy) 33

Challenge • Why do we need to use programmable switch like P4? • How can we stop pipelined HTTP request from client during TCP connection is migrated in backend? • What happen when the client resets the connection? • More details are in paper! 34

Evaluation • We setup two experiments 1. End-to-End throughput comparison between legacy proxy and Prism for HTTP/1.1 2. Packet transformation performance measurement for Prism Switch 35

Evaluation: End-to-End Throughput • 2 clients and 2 servers Proxy HTTP HTTP Benchmarking Server App L2 Switch (Software) HTTP HTTP Benchmarking Server App 36

Evaluation: End-to-End Throughput • 2 clients and 2 servers Prism Switch Controller HTTP Prism Benchmarking HTTP App Backend Prism Switch HTTP Prism (Software) Benchmarking HTTP App Backend 37

Evaluation: End-to-End Throughput • Total throughput of proxy’s clients are limited to about 9Gbps in Prism 15 Proxy maximum Throughput [Gbps] 10 • Total throughput of Prism’s clients exceeds 10Gbps in 2MB data transfer 5 • Finally reached to about 18.7Gbps in maximum 0 1 10 100 1000 2000 5000 10000 50000 75000 100000 Object Size [KB] Prism can use bandwidth of uplinks which proxy’s clients can’t use 38

Evaluation: End-to-End Throughput • Total throughput of proxy’s clients are limited to about 9Gbps in Prism 15 Proxy maximum Throughput [Gbps] 10G 10 • Total throughput of Prism’s clients exceeds 10Gbps in 2MB data transfer 5 • Finally reached to about 18.7Gbps in maximum 0 1 10 100 1000 2000 5000 10000 50000 75000 100000 Object Size [KB] Prism can use bandwidth of uplinks which proxy’s clients can’t use 39