STMBench7: A Benchmark for Software Transactional Memory Rachid - PowerPoint PPT Presentation

STMBench7: A Benchmark for Software Transactional Memory Rachid Guerraoui 1 Michał Kapałka 1 Jan Vitek 2 1 EPFL, Switzerland 2 Purdue University, USA EuroSys 2007 Michał Kapałka (EPFL) STMBench7 EuroSys 2007 1 / 23

Introduction Goal of the Talk 1 Recall the idea of software transactional memory 2 Present STMBench7: a benchmark for STM implementations Michał Kapałka (EPFL) STMBench7 EuroSys 2007 2 / 23

Introduction New Trends in Hardware CPU clock speed Common (mis)interpretation of Moore’s law. . . Year Michał Kapałka (EPFL) STMBench7 EuroSys 2007 3 / 23

Introduction New Trends in Hardware CPU clock speed ? . . . now challenged Year Michał Kapałka (EPFL) STMBench7 EuroSys 2007 3 / 23

Introduction New Trends in Hardware Solution: going multi-core # cores per CPU ⇓ New challenges for the masses of programmers: 1 Exploit parallelism 2 Manage concurrency Year Michał Kapałka (EPFL) STMBench7 EuroSys 2007 4 / 23

Introduction Current Approaches to Multi-threading synchronized(this) { Explicit locking is hard this.x.credit(5); Deadlock, priority inversion this.y.debit(5); Fault-tolerance issues } Wait-free/obstruction-free computing: not for mortals (each algorithm a PODC/DISC paper. . . ) Michał Kapałka (EPFL) STMBench7 EuroSys 2007 5 / 23

Introduction Software Transactional Memory (STM) Multi-threading made easier: atomic { thread synchronization via acc1.x.credit(5); in-memory transactions acc2.y.debit(5); } Does not share the inherent problems of locking ⇒ commit or abort Michał Kapałka (EPFL) STMBench7 EuroSys 2007 6 / 23

Introduction STM implementations are coming! TL2 RSTM Haskell STM SI-STM OSTM LSA-STM DSTM . . . DSTM2 ASTM SXM Which one is best? Michał Kapałka (EPFL) STMBench7 EuroSys 2007 7 / 23

Introduction Open Problems 1 Which implementation/algorithm is best? 2 What are the trade-offs? 3 Are the overheads acceptable? theoretical ⇒ Need evaluation tools experimental ⇒ STMBench7 Michał Kapałka (EPFL) STMBench7 EuroSys 2007 8 / 23

Introduction Evaluation So Far Thread 2 Design features Thread 1 Thread 3 Simple data structures “Toy” applications put remove get ⇒ Need realistic benchmark Hash table R-B Tree Skip list ⇓ Total throughput Michał Kapałka (EPFL) STMBench7 EuroSys 2007 9 / 23

Introduction STM Benchmark Recipe 1 Realistic workload 2 Multi-threading 3 Non-trivial concurrency 4 Baseline for comparison + std. benchmark requirements STMBench7 is a first step towards such a benchmark Michał Kapałka (EPFL) STMBench7 EuroSys 2007 10 / 23

Introduction Example Output of STMBench7 5000 4500 4000 Maximum latency [ms] 3500 3000 2500 2000 1500 R/T1 medium 1000 R/T1 coarse W/T2 medium 500 W/T2 coarse 0 0 2 4 6 8 10 12 14 16 Number of threads Michał Kapałka (EPFL) STMBench7 EuroSys 2007 11 / 23

Introduction Example Output of STMBench7 14000 Total throughput [operations / s] 12000 10000 8000 6000 4000 medium 2000 ASTM coarse 0 0 5 10 15 20 25 30 35 Number of threads Michał Kapałka (EPFL) STMBench7 EuroSys 2007 11 / 23

Architecture of STMBench7 Realistic Workload 1 Realistic workload 2 Multi-threading 3 Non-trivial concurrency 4 Baseline for comparison STMBench7 is based on the OO7 benchmark: Well-known in OO database world Aims at being realistic (CAD/CAM/CASE applications) Already used for transactional monitors Way not enough! Michał Kapałka (EPFL) STMBench7 EuroSys 2007 12 / 23

Architecture of STMBench7 OO7 Data Structure (Main Tree) Module Manual CA Complex Assemblies (CA) CA CA CA 7 levels . . . . . . . . . . . . Base Assemblies (BA) BA BA Michał Kapałka (EPFL) STMBench7 EuroSys 2007 13 / 23

Architecture of STMBench7 OO7 Data Structure (Design Library) . . . Base Assemblies (BA) BA BA . . . Design Library CP CP CP (Composite Parts CP) . . . . . . AP Graph Atomic Parts (AP) Doc Michał Kapałka (EPFL) STMBench7 EuroSys 2007 14 / 23

Architecture of STMBench7 OO7 / STMBench7 Data Structure Summary Large tree with a graph in each leaf 6 indexes Can be traversed in any direction Michał Kapałka (EPFL) STMBench7 EuroSys 2007 15 / 23

Architecture of STMBench7 OO7 Limitations Client operation OO7: Single client TTC of isolated long Data operations measured structure Mostly static structure ⇓ Time to complete Michał Kapałka (EPFL) STMBench7 EuroSys 2007 16 / 23

Architecture of STMBench7 From OO7 to STMBench7 Thread 2 Thread 1 Thread 3 op2 STMBench7: op1 op3 Multiple threads Various-length operations Dynamic structure Data structure OO7-related code: ∼ 50 % ⇓ Latency, throughput Michał Kapałka (EPFL) STMBench7 EuroSys 2007 17 / 23

Architecture of STMBench7 Operation Types in STMBench7 Traversals: access most objects Short traversals: access object on some random path Short operations: search and/or simple updates Structure modification operations Michał Kapałka (EPFL) STMBench7 EuroSys 2007 18 / 23

Architecture of STMBench7 Operation Types in STMBench7 Traversals: access most objects Short traversals: access object on some random path Short operations: search and/or simple updates Structure modification operations ⇓ 1 Realistic workload 2 Multi-threading 3 Non-trivial concurrency 4 Baseline for comparison Michał Kapałka (EPFL) STMBench7 EuroSys 2007 18 / 23

Architecture of STMBench7 Built-in Locking Techniques 1 Realistic workload 2 Multi-threading 3 Non-trivial concurrency 4 Baseline for comparison Michał Kapałka (EPFL) STMBench7 EuroSys 2007 19 / 23

Architecture of STMBench7 Built-in Locking Techniques 1 Realistic workload 2 Multi-threading 3 Non-trivial concurrency 4 Baseline for comparison Coarse-grained locking: single RW lock Medium-grained locking: one RW lock per level, global RW lock for structure modifications. Michał Kapałka (EPFL) STMBench7 EuroSys 2007 19 / 23

Architecture of STMBench7 Porting Issues Languages: Current implementation in Java 5 ( ∼ 5000 lines) C# port being developed (group of M. Herlihy) C++ version coming soon (with M. Scott) STMs: Experiments done with ASTM Easy to plug other STMs (although no single STM interface) Michał Kapałka (EPFL) STMBench7 EuroSys 2007 20 / 23

Concluding Remarks Summary STMBench7 is a first attempt at realistic benchmarking STMs Data structure suggestive of CAD, CAM, CASE and similar applications, but also some on-line services Code already available (and ready for use) Open-source (BSD license) First experiments show that STMBench7 is a “stress test” for STM (see the paper) Michał Kapałka (EPFL) STMBench7 EuroSys 2007 21 / 23

Concluding Remarks Open Problems (Recalled) 1 Which implementation/algorithm is best? 2 What are the trade-offs? 3 Are the overheads acceptable? theoretical (stay tuned. . . ) ⇒ Need evaluation tools experimental ⇒ STMBench7 Michał Kapałka (EPFL) STMBench7 EuroSys 2007 22 / 23

Concluding Remarks Open Problems (Recalled) 1 Which implementation/algorithm is best? 2 What are the trade-offs? 3 Are the overheads acceptable? theoretical (stay tuned. . . ) ⇒ Need evaluation tools experimental ⇒ STMBench7 Future directions: Add fine-grained locking Data validation Michał Kapałka (EPFL) STMBench7 EuroSys 2007 22 / 23

Concluding Remarks Questions? lpd.epfl.ch/kapalka/stmbench7.php Michał Kapałka (EPFL) STMBench7 EuroSys 2007 23 / 23