ERMIA: Fast Memory-Optimized Database System for Heterogeneous Workloads Kangnyeon Kim Tianzheng Wang Ryan Johnson Ippokratis Pandis
Heterogeneous is the new OLTP Convenient/traditional Emerging workloads W W R W R RR W R R R RR RRRR R R R R R W R R W Short write-intensive Short write-intensive Short read-only Longer read- mostly Worst case for current systems, esp. CC 2
Modern OCC 101 • Decentralized optimistic Database concurrency control (OCC) D A B’ • Silo [Tu et al. ’13 ] • FOEDUS [Kimura ’15] B’ Write set • No locking for reads A Read set • Locally-cached writes Transaction 2 • Commit phase A B Read set 1. Lock writes Clobbered D Write set 2. Verify reads abort Transaction 1 3. Apply writes 3
OCC: Not a panacea Fair Starved Aggregate commit ratio 6 5 NewOrder Payment 4 OrderStatus 3 Delivery 2 StockLevel 1 Read-mostly 0 ERMIA Silo-OCC Ideal ERMIA Silo-OCC TPC-C + TPC-C long read-mostly tx 4
OCC + Read-mostly = Wasted cycles TPC-C with Commit NewOrder TPC-C long read-mostly Commit Payment 100% Commit Delivery Commit OrderStatus 80% Commit StockLevel Commit Read-mostly 60% Abort NewOrder 40% Abort Payment Abort Delivery 20% Abort OrderStatus 0% Abort StockLevel Silo-OCC ERMIA-SSN Silo-OCC ERMIA-SSN Abort Read-mostly Read-mostly tx: fair and robust CC needed 5
ERMIA comes to the rescue Logical layer: robust && fair CC Snapshot Serial Safety Isolation Net (SSN) Read-friendly, fair, robust, timely abort Serializable Database == Lock- Scalable centralized log free Index E poch-based In-memory R esource M anagement data Scalable physical layer 6
Scalable Centralized Redo Logging Private The only global log buffer communication Upon commit, LSN = XADD(current LSN, log size) Async. filling Log Durable Current Current LSN LSN LSN XADD Scalability + Global Ordering 7
Latch-free Indirection Array • Object IDs rather than pointers in leaf nodes • No update propagation to secondary indexes Versions in durable log Lock-free Index Fast recovery: OIDs in leaf nodes fetch header only OID Address 1 V1 V2 V3 2 3 … CAS to install new version … … 8
The Serial Safety Net T4 T1 T1, T2, T3: commit time predecessors of T4 π (T4) T2 T4’s earliest T3 successor dependency order Forbid $ P T : π(T) ≤ c(P) ≤ c(T) c(T3): commit time “exclusion window ” of T The Serial Safety Net: Efficient Concurrency Control on Modern Hardware T. Wang, R. Johnson, A. Fekete, I. Pandis, DaMoN ’15 9
Visualizing SSN T4 T2 T1 T3 T1 (T1) π (T4) T2 T4 π (T2) T3 commit time T5 Exclusion window satisfied T2 T2 T4 ?? T1 T1 π (T2) π (T2) T3 Exclusion window violation dependency order 10
Evaluation • 24-core, quad-socket Xeon E7-4807 • Everything in main memory • OCC vs. ERMIA-SI vs. ERMIA-SSN • “Convenient”/traditional OLTP • Original TPC-C • Original TPC-E • Heterogeneous OLTP • TPC-C with read-mostly transaction ( TPC-C-hybrid ) • TPC-E with read-mostly transaction ( TPC-E-hybrid ) 11
“Convenient” Higher is better TPC-C TPC-E Comparable performance to OCC 12
Robust heterogeneous performance Higher is better TPC-C-hybrid TPC-E-hybrid (Much) better overall throughput 13
Robust heterogeneous performance Higher is better Lower is better (Much) lower abort rate for read-mostly tx 14
Conclusions • Heterogeneous OLTP : fair && robust CC needed Fair, robust, Read-friendly serializable • ERMIA = Snapshot Isolation + SSN Scalable + Lock/latch-free physical layer • Also performs well for traditional OLTP Find out more in our paper and code repo! https://github.com/ermia-db Thank you! 15
Recommend
More recommend
