More Than A Network: Distributed OLTP on Clusters of Hardware - PowerPoint PPT Presentation

More Than A Network: Distributed OLTP on Clusters of Hardware Islands Danica Porobic , Pınar Tözün, Raja Appuswamy, Anastasia Ailamaki 1

Multisocket multicores 21 cycles 72 cycles 2x12-core 237 cycles Intel Xeon E5-2650L v3 threads core core core core core core core core L1 L1 L1 L1 L1 L1 L1 L1 L1 L2 L2 L2 L2 L2 L2 L2 L2 Island L3 L3 L3 memory controller memory controller Inter-socket links inter-socket links inter-socket links Inter-socket links socket 1 socket 0 Challenge: non-uniform communication 2

OLTP on Hardware Islands Shared-everything Island shared-nothing Shared-nothing  stable  robust middle ground  fast  not optimal  sensitive to workload Optimal configuration depends on workload and hw 3

Rack-scale hardware platforms • Abundant non-uniform parallelism – Need to scale across many cores • Large main memories – Datasets are memory resident • Network & DRAM converge – Need to scale across multiple nodes Complex hierarchy of Hardware Islands 4

How different are clusters of Islands? • Does Island topology still matter in the cluster environment? • Does faster communication always improve throughput? • How do scale-up designs perform when used in distributed deployments? 5

Experimental setup • Shore-MT • Silo • 2 x 6-core Intel Xeon • 8 x 10-core Intel Xeon X5660 E7-L8867 • 10 Gbps Ethernet • Unix sockets and shared memory • TCP/IP and shared communication memory communication • Partition-sensitive microbenchmark • TPC-C and partition- sensitive microbenchmark 6

Distributed deployments … … … scale-up hybrid scale-out 7

Scaling out across the cluster Shore, TCP/IP TPC-C Payment TPC-C New Order 160 600 140 500 120 Throughput (KTps) Throughput (KTps) 100 400 80 300 60 200 40 100 20 0 0 1 2 4 8 1 2 4 8 Number of servers Number of servers scale-up hybrid scale-out No configuration is optimal for every cluster 8

Impact of placement Shore, TCP/IP TPC-C New Order TPC-C Payment 45 200 180 40 160 35 Throughput (KTps) Throughput (KTps) 140 30 120 25 100 20 80 15 60 10 40 5 20 0 0 scale-out scale-up scale-out scale-up OS Bound Thread migrations hurt performance & predictability 9

Partition sensitive microbenchmark • Single site version – probe/update N rows from the local site • Multisite version – probe/update 1 row from the local site – probe/update N-1 rows uniformly from any site – sites may reside on the same instance 10

Impact of fast communication Shore-MT Reading 20 rows Updating 20 roes 90 50 45 80 Throughput (KTps) Throughput (KTps) 40 70 35 60 30 50 25 40 20 30 15 20 10 10 5 0 0 scale-out scale-up scale-out scale-up scale-out scale-up scale-out scale-up Local 20% multisite Local 20% multisite TCP/IP Shared memory Read-only: helps, Updates: little impact 11

Scaling out a scale-up system Silo Reading 2 rows Updating 2 rows 40 14 12 Throughput (MTps) Throughput (MTps) 30 10 8 20 6 4 10 2 0 0 scale-out scale-up scale-out scale-up scale-out scale-up scale-out scale-up Local 1% multisite Local 20% multisite Sockets Shared memory Distributed updates cause severe throughput drop 12

Why don’t updates scale out? Read-write Read-only RW aborts 1 60 Normalized throughput 50 0.8 40 %Aborts 0.6 30 0.4 20 0.2 10 0 0 0 10 20 30 40 50 60 70 80 90 100 % distributed transactions Multicore-optimized OCC very sensitive to delays 13

OLTP on a Cluster of Islands • Scale-up designs sensitive to scale-out delays • Islands-awareness required, but insufficient for optimal cluster deployments • Fast communication can improve throughput, but does not guarantee improvement Thank you! 14