Random Regular Graph & Generalized De Bruijn Graph with k -shortest Path Routing Peyman Faizian , Md Atiqul Mollah, Xin Yuan Scott Pakin, Michael Lang Florida State University Los Alamos National Laboratory
Where it All Started • Random Regular topologies(Jellyfish) have been proposed for data centers and HPC clusters • They are known to have some good topological properties • They achieve higher performance in compare to Fat Tree under certain traffic patterns
We are Going to do this the Hard Way • Derive theoretical bounds for the following key metrics on any DRG • Diameter • Average k -shortest path length • Load balancing property • Evaluate RRG based on these criteria • Introduce a near-optimal DRG and compare it to RRG through simulation
What is a Random Regular Topology • Random graph between ToR switches • Each switch has k ports • r ports connected to other switches • k-r ports connected to processing nodes • r -regular random interconnect topology( RRG ) [Singla et al; NSDI’12]
What We Know About RRG • High bisection bandwidth • High network capacity • Sufficient short path diversity • k -shortest path routing is preferred [Singla et al; NSDI’12]
What is Needed for K -Path Routing • High bisection bandwidth • Minimal resource usage • Low diameter • Low average k -shortest path length • Good load balancing • Even distribution of paths over SD pairs • Even distribution of load over all network links
What Should We Compare RRG to • RRG has been compared to one of the best available designs • Fat Tree • We want to compare it to the best possible design • Random Regular Graphs are a special case of Directed Regular Graphs • We derived the optimal bounds for all discussed properties on Regular Graphs 4 Lemmas. Some Graph Theory!
Diameter 66% 33% 𝐸𝑗𝑏𝑛𝑓𝑢𝑓𝑠 𝑝𝑔 𝐸𝑆𝐻(𝑂, 𝑠) ≥ 𝑚𝑝 𝑠 𝑂 𝑠 − 1 + 1 − 1
Average Shortest Path Length 14% ℎ−1 𝑘𝑠 𝑘 + ℎ𝑆 𝑘=1 𝐵𝑤𝑓𝑠𝑏𝑓 𝑙 𝑡ℎ𝑝𝑠𝑢𝑓𝑡𝑢 𝑞𝑏𝑢ℎ 𝑚𝑓𝑜𝑢ℎ 𝑔𝑝𝑠 𝐸𝑆𝐻(𝑂, 𝑠) ≥ 𝑙(𝑂 − 1)
Average k -Shortest Path Length ℎ−1 𝑘𝑠 𝑘 + ℎ𝑆 𝑘=1 𝐵𝑤𝑓𝑠𝑏𝑓 𝑙 𝑡ℎ𝑝𝑠𝑢𝑓𝑡𝑢 𝑞𝑏𝑢ℎ 𝑚𝑓𝑜𝑢ℎ 𝑔𝑝𝑠 𝐸𝑆𝐻 𝑂, 𝑠 = 𝑀𝐿(𝑂, 𝑠, 𝑙) ≥ 𝑙(𝑂 − 1)
Path Distribution Network Degree=30, N=901 𝑠 + 𝑠 2 + ⋯ + 𝑠 𝐼 min 𝑜𝑣𝑛𝑐𝑓𝑠 𝑝𝑔 𝑞𝑏𝑢ℎ𝑡 𝑝𝑔 𝑚𝑓𝑜𝑢ℎ ≤ 𝐼 ∶ 𝑂 − 1
Load Balancing • All-to-All communication pattern • Use K -shortest paths between each SD pair • Monitor the number of flows passing each link • Pick the load value on the most loaded link in the network
Load Balancing max 𝑚𝑗𝑜𝑙 𝑚𝑝𝑏𝑒 𝑔𝑝𝑠 𝐸𝑆𝐻 𝑂, 𝑠 = 𝑁𝑀(𝑂, 𝑠) ≥ 𝑀𝐿 𝑂, 𝑠, 𝑙 × (𝑂 − 1) 𝑠
A Quick Recap • We compared these topological properties of Jellyfish with optimal: • Diameter • Average k-shortest path length • Path distribution • Load distribution • In some of the cases Jellyfish is far from theoretical bounds • But is there any near optimal r -regular topology?
Generalized de Bruijn Graph • We proved that GDBG has: • Near optimal diameter • Near optimal average k -shortest path length • Near optimal path distribution • Near optimal load distribution 𝐻𝐸𝐶𝐻(6,2) 7 Lemmas. Some More Math! 𝑜𝑝𝑒𝑓 𝑗 𝑑𝑝𝑜𝑜𝑓𝑑𝑢𝑡 𝑢𝑝 𝑜𝑝𝑒𝑓𝑡: 𝑗 ∗ 𝑒, 𝑗 ∗ 𝑒 + 1, … , 𝑗 + 1 ∗ 𝑒 − 1 𝑛𝑝𝑒 𝑂
GDBG: A Near Optimal r -Regular Topology
GDBG: A Near Optimal r -Regular Topology
There is more… • GDBG is a near optimal r-regular topology • It can be used as a simulation benchmark to evaluate other topologies in this family • We are going to evaluate RRG
Simulation • Topology • Generalized de Bruijn Graph(almost optimal) • Jellyfish • Routing • Hop limited all path routing (GDBG) • k -shortest path routing (Jellyfish) • Traffic Pattern • Node level random permutation • Node level shift • Switch level random permutation • Switch level shift • Metric • Aggregate throughput
Node Level Throughput N=150, Network Degree=8
Switch Level Throughput
Conclusion • Derived theoretical bounds for the following key metrics on any DRG • Average k -shortest path length • Load balancing property • RRG is near optimal in terms of average k -shortest path length • RRG is far from optimal for all other metrics • GDBG was found near optimal for all metrics • GDBG was used as a simulation benchmark to evaluate RRG • Depending on traffic pattern, RRG is not always near optimal
Thanks for your time Questions
Recommend
More recommend
![W HAT DOES IT MEAN ? Given a graph G with vertex set [ n ] : Pr ( G ( n , p ) = G ) = p e ( G ) ( 1](https://c.sambuz.com/997618/w-hat-does-it-mean-s.jpg)
