Hyrise-R: Scale-out and Hot-Standby through Lazy Master Replication for Enterprise Applications David Schwalb, Jan Kossmann, Martin Faust, Stefan Klauck, Matthias Uflacker, Hasso Plattner Hasso Plattner Institute, University of Potsdam, Germany IMDM 2015
Motivation Customers Sales Managers Decision Support New enterprise applications .. OLAP, Search and Read-Only Applications □ Growing number of users Read-Only on Transactional Schema Replicas □ Increasingly complex queries OLXP □ Interactive data exploration Master Node < 1 Second OLTP .. require scalability Operational Reporting Data Entry [1] & New Applications Scale-up vs. scale-out Hyrise-R (+ availability) Stefan Klauck Chart 2
Related Work Theoretical replication models and comparison [2] □ Eager vs. lazy □ Group vs. master Implementations □ Postgres-R – Eager group replication based on shadow copies [3] □ ScyPer – Lazy master replication with row layout for master node [4] □ .. Hyrise-R Stefan Klauck Chart 3
Hyrise Storage engine developed at HPI for research and prototyping, initially focused on main memory processing and hybrid storage layouts of tables □ Dictionary and bit-vector compression □ Main/delta architecture with merge process □ Hybrid row and column layouts of tables □ Supports vertical and horizontal partitioning Hyrise-R Stefan Klauck Chart 4
Hyrise – Research History A Common Database Approach Non volatile Memory for OLTP and OLAP Main Memory Optimized Hyrise-NV Index Structures MVCC Data Aging HYRISE- A Main Memory Frontend Hybrid Storage Engine SQL 2013 2015 2009 2011 Replication Merge Process SGI installation Hyrise-R SSICLOPS Stefan Klauck TAMEX: A Task-Based Chart 5 Query Execution Framework
Hyrise-R Read workload Write workload Dispatcher R Cluster R R Request Handler Request Handler Request Handler Logger Logger Logger ● ● ● Data Data Data Hyrise-R Storage Storage Storage Stefan Klauck Cluster Interface Cluster Interface Cluster Interface R Chart 6 Primary Node Replica 1 Replica n R
Dispatcher Redirect queries to cluster nodes Read workload Write workload □ Transactional workload -> master node □ Reads -> all cluster nodes Dispatcher R Cluster R R Request Handler Request Handler Request Handler Logger Logger Logger ● ● ● Data Data Data Hyrise-R Storage Storage Storage Stefan Klauck Cluster Interface Cluster Interface Cluster Interface R Primary Node Replica 1 Replica n Chart 7 R
Replication Mechanism Logs are written to file system + send to cluster interface Cluster interface sends (dictionary encoded) log information to replicas Frequency is configurable and based on Read workload Write workload □ Number of calls □ Exceeding buffer size □ Time since last transmission Dispatcher R Cluster R R Request Handler TCP with nanomsg Request Handler Request Handler □ Survey pattern allows replicas Logger Logger Logger ● Hyrise-R to acknowledge reception ● ● Data Data Data Stefan Klauck Storage Storage Storage □ Heartbeat protocol for failover Cluster Interface Cluster Interface Cluster Interface Chart 8 R Primary Node Replica 1 Replica n R
Why Hyrise-R is a good fit for Enterprise Applications Customers Sales Managers Decision Support OLAP, Search and Read-Only Applications Read-Only on Transactional Schema Replicas OLXP Master Node < 1 Second OLTP Hyrise-R Operational Reporting Data Entry Stefan Klauck & New Applications [1] Chart 9
Evaluation on Amazon EC2 cluster 5 machines with □ Intel Xeon E5-2666 v3 (36cCPUs; 10 cores @ 2.9GHz) □ 60 GiB main memory Hyrise-R Stefan Klauck Chart 10
Conclusion Hyrise-R – a system to cluster Hyrise instances using lazy master replication □ Dictionary compressed logs for updating replicas □ Heartbeat protocol for failover □ Benchmarks on Amazon EC2 cluster Future Work □ Extend query dispatching and distribution □ Extend mixed workload measurements (ch-beCHmark) Hyrise-R Stefan Klauck Chart 11
References [1] H. Plattner. The Impact of Columnar In-Memory Databases on Enterprise Systems. 2014 [2] J. Gray, P. Helland, P. O’Neil, and D. Shasha. The dangers of replication and a solution. 1996 [3] B. Kemme and G. Alonso. Don’t be lazy, be consistent: Postgres-r, a new way to implement database replication. 2000 [4] T. Mühlbauer, W. Rödiger, A. Reiser, A. Kemper, and T. Neumann. Scyper: Elastic olap Hyrise-R throughput on transactional data. 2013 Stefan Klauck Chart 12
Thanks Stefan Klauck stefan.klauck@hpi.de http://epic.hpi.de
Recommend
More recommend
