6.888 Lecture 9: Wireless/Op4cal Datacenters Mohammad Alizadeh and - PowerPoint PPT Presentation

6.888 Lecture 9: Wireless/Op4cal Datacenters Mohammad Alizadeh and Dinesh Bharadia ² Many thanks to George Porter (UCSD) and Vyas Sekar (Berkeley) Spring 2016 1

Datacenter Fabrics Spine Leaf 1000s of server ports Scale out designs (VL2, Fat-tree) Ø LiTle to no oversubscrip4on Ø Cost, power, complexity 2

Mul4ple switching layers (Why?) ² hTps://code.facebook.com/posts/360346274145943/introducing-data-center-fabric-the- 3 next-genera4on-facebook-data-center-network/

Building Block: Merchant Silicon Switching Chips Switch ASIC 6 pack Facebook Wedge Limited radix: 16x40Gbps High power: 17 W/port ² Image courtesy of Facebook 4

Long cables (fiber) ² hTps://code.facebook.com/posts/360346274145943/introducing-data-center-fabric-the- 5 next-genera4on-facebook-data-center-network/

Scale-out packet-switch fabrics Large number of switches, fibers, op4cal transceivers Power hungry Hard to expand = Core transceiver ... S N,0 S N,1 S N,k/2 = Edge transceiver N-Layers ... S 2,0 S 2,1 S 2,2 S 2,3 S 2,k ... S 1,0 S 1,1 S 1,2 S 1,3 S 1,k ... S 0,0 S 0,1 S 0,2 S 0,3 S 0,k H i H i H i H i H i

Beyond Packet-Switched DC Fabrics 60 GHz RF Op4cal circuit switching [Flyways, MirrorMirror] [Helios, cThrough, Mordio, ReacTor, …] = Edge transceiver OCS kxk Pkt ... S 0,0 S 0,1 S 0,2 S 0,3 S 0,k H i H i H i H i H i ² Fig. from presenta4on by Xia Zhou Free-space Op4cs [FireFly] Steerable Links 7

Integra4ng Microsecond Circuit Switching into the Data Center ² Slides based on presenta4on by George Porter (UCSD) 8

Key idea: Hybrid Circuit/Packet Networks = Edge transceiver OCS kxk Pkt ... S 0,0 S 0,1 S 0,2 S 0,3 S 0,k H i H i H i H i H i Why build hybrid switch?

Circuit vs. Packet Switching Observa4on: Correlated traffic è Circuits Electrical Packet Op.cal Circuit $500/port $500/port Rate free (10/40/100/400/+) 10 Gb/s fixed rate 240 mW/port 12 W/port No transceivers Transceivers (OEO) No buffering Buffering Duty cycle overhead Per-packet switching Out-of-band control In-band control

Disadvantages of Circuits = Edge transceiver Despite advantages, OCS kxk Pkt circuits present different service model: ... S 0,0 S 0,1 S 0,2 S 0,3 S 0,k H i H i H i H i H i – Point-to-point connec4vity } – Must wait for circuit to affects throughput, latency be assigned } – Circuit “down” while affects network duty cycle; being reconfigured overall efficiency

Stability Increases with Aggrega4on Inter-Data Center Where is the Sweet Spot? Inter-Pod Inter-Rack 1. Enough Stability Inter-Server 2. Enough Traffic Inter-Process Inter-Thread 12

Mordia OCS model S 0 S 0 OCS kxk S 1 S 1 à ... S 2 S 2 S 0 S 1 S 2 S 3 S k S 3 S 3 … … S k S k • Directly connects inputs to outputs Bi-par4te graph • Reconfigura4on 4me: 10us – “Night” 4me (Tn): no traffic during reconfigura4on – “Day” 4me (Td): circuits/mapping established • Duty cycle: Td / (Td+Tn)

Previous approaches: Hotspot Scheduling Step 1. Observe network traffic Step 2. Compute schedule TM S Assign circuits to elephants OCS Step 3. Reconfigure X Reconfig X 1. Observe 2. Compute 3. Reconfig X 1. Observe 2. Compute 3. Reconfig 1. Observe 2. Compute Time

Limita4ons of Hotspot Scheduling config 3 1. Observe 3 1. Observe TM( t ) 3 1. Observe Time Goal Reconfig 3 3 3 3 3 3 3 3 3 1. Observe 2 TM( t ) 3 3 3 3 3 3 3 3 3 1. Observe 2 3 3 3 3 3 3 3 3 3 1. Observe 2 3 3 1. Observe 2 Time

Traffic Matrix Scheduling Step 1. Gather traffic matrix TM Step 2. Scale TM into TM´ TM TM´ Birkhoff von-Neumann Step 3. Decompose TM´ into schedule Decomposi4on P 1 P 2 P N t 1 t 2 t N + + + Step 4. Execute schedule in hardware t 1 t 2 t N

BvN Decomposi4on T has to be k’ could be large doubly-stochas4c ( in worst case) ² Suppose: T is a scaled doubly-stochas4c matrix

Scheduling circuit switch configura4on: bipar4te graph matching Traffic Matrix: T 1 4 1 4 4 1 4 1 4 1 n = 5 nodes 4me

Scheduling configura4on of circuit switch modeled as bipar4te graph matching Traffic Matrix: T 1 4 1 4 1 4 1 4 1 4 n = 5 nodes 4me

Scheduling configura4on of circuit switch modeled as bipar4te graph matching Traffic Matrix: T 1 0 1 0 1 0 1 0 1 0 n = 5 nodes 4me reconfiguration delay

Scheduling configura4on of circuit switch modeled as bipar4te graph matching Traffic Matrix: T 0 1 0 1 0 1 0 1 0 1 n = 5 nodes 4me

Scheduling configura4on of circuit switch modeled as bipar4te graph matching Traffic Matrix: T 0 0 0 0 0 0 0 0 0 0 n = 5 nodes 4me

Scheduling maximize throughput in 4me-window W Traffic Matrix: T 1 4 1 4 4 1 ?? 4 1 4 1 n = 5 nodes 4me W

Problem Statement maximize s.t. number of matchings permuta4on matrices dura4on

Eclipse: Greedy Algorithm (with provable guarantees) ² Venkatakrishnan et al., “Costly Circuits, Submodular Schedules, Hybrid Switch Scheduling for Data Centers”, To appear in SIGMETRICS 2016. 25

Discussion 26

Firefly ² Slides based on presenta4on by Vyas Sekar (CMU) 27

Why FSO instead of RF? RF (e.g. 60GHZ) FSO (Free Space op4cal) Wide beam è Narrow beam è Faster steering of beams Slow steering of beams High interference Zero interference Limited ac4ve links No limit on ac4ve links Limited Throughput High Throughput 28

Today’s FSO Cost: $15K per FSO Size: 3 s³ Power: 30w Non steerable • Current: bulky, power-hungry, and expensive • Required: small, low power and low expense 29

Why Size, Cost, Power Can be Reduced? • Tradi4onal use : outdoor, long haul ‒ High power ‒ Weatherproof • Data centers: indoor, short haul • Feasible roadmap via commodity fiber op4cs ‒ E.g. Small form transceivers (Op4cal SFP) 30

FSO Design Overview fiber op4c cables Diverging beam Lens focal distance Parallel beam Focusing lens lens Collima4ng lens Large core fiber op4c cables SFP • large cores (> 125 microns) are more robust 31

FSO Link Performance Effect of vibra4ons, etc. 6mm movement tolerance Range up to 24m tested 6 mm 6 mm FSO link is as robust as a wired link 32

Steerability Shortcomings of current FSOs ü Cost FSO design ü Size using SFP ü Power • Not Steerable Via Switchable mirrors or Galvo mirrors Shortcomings of current FSOs 33

Steerability via Switchable Mirror • Switchable Mirror: glass mirror • Electronic control, low latency Ceiling mirror SM in “mirror” mode B C A 34

Steerability via Galvo Mirror • Galvo Mirror: small rota4ng mirror • Very low latency Ceiling mirror Galvo Mirror B C A 35

How to design FireFly network? Goals: Robustness to current and future traffic Budget & Physical Constraints Design parameters – Number of FSOs? – Number of steering mirrors? – Ini4al mirrors’ configura4on Performance metric – Dynamic bisec4on bandwidth 36

Discussion 37

Next Time: Rack-Scale Compu4ng 38

39