Information Systems Group The Mimicking Octopus Towards a one-size-fits-all Architecture for Database Systems Alekh Jindal Supervisor: Prof. Dr. Jens Dittrich VLDB PhD Workshop September 13, 2010
Database Landscape Information Systems Group OLAP Streaming Archival System System Search OLTP Engine Motivation September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 2
Database Landscape Information Systems Group Company Information System OLAP Streaming Archival System System Search OLTP Engine Motivation September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 2
Database Landscape Airline Company Information Systems Group Integration Cost Licensing Cost Maintenance Cost Several Applications DBA Cost Evolving Applications Reporting Engineering Cost OLAP Streaming Archival System System Booking Cheap Fares Archives Eventual Integration Search OLTP Hard-coded optimizations ETL style data pipelines Engine Flight Search Hard-coded data layouts Ticket Booking Motivation September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 2
Problem Statement Information Systems Group • Single database system • Automatic adaption • Improved performance • Lower cost • Better maintainability Motivation September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 3
OctopusDB Overview Information Systems Group • One-size-fits-all architecture • Abstract storage concept: Storage Views(SV) • Single optimization problem: SV Selection • Holistic SV optimizer OctopusDB September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 4
System Architecture Information Systems Group API Primary Log Store Storage View Store Result Log SV Purging & Checkpointing Recovery Manager Query Catalog Storage View Catalog Holistic SV Optimizer Transaction Manager • No hard-coded store • All operations recorded as logical log entries in a primary log on stable storage using WAL OctopusDB September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 5
System Architecture Information Systems Group API Primary Log Store Storage View Store Result Log SV Purging & Checkpointing Recovery Manager Query Catalog Storage View Catalog Holistic SV Optimizer Transaction Manager • No hard-coded store • All operations recorded as logical log entries in a primary log on stable storage using WAL OctopusDB September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 5
System Architecture Information Systems Group API Primary Log Store Storage View Store Result Log SV Purging & Checkpointing Recovery Manager Query Catalog Storage View Catalog Holistic SV Optimizer Transaction Manager • No hard-coded store • All operations recorded as logical log entries in a primary log on stable storage using WAL OctopusDB September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 5
Storage Views Information Systems Group • Arbitrary physical representations of data • Different layouts under a single umbrella Storage Views September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 6
Storage Views Information Systems Group • Arbitrary physical representations of data • Different layouts under a single umbrella Primary Log SV Row SV Column SV Index SV Storage Views September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 6
Storage Views Information Systems Group • Arbitrary physical representations of data • Different layouts under a single umbrella Primary Secondary Log SV Partial Index SV Row SV Bag-partitioned SV Column SV Key-consolidated SV Index SV Vertically/Horizontally Partitioned SV Storage Views September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 6
Storage Views Information Systems Group • Arbitrary physical representations of data • Different layouts under a single umbrella Primary Secondary Log SV Partial Index SV Row SV Bag-partitioned SV Column SV Key-consolidated SV Index SV Vertically/Horizontally Partitioned SV ... any hybrid combination of the above Storage Views September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 6
Use-case Scenario* Information Systems Group • Flight booking system • Tables: Tickets , Customers • Tickets : several attributes, frequently updated • Customers : fewer attributes • Queries: SELECT C.* FROM Tickets T, Customers C WHERE T.customer_id=C.id AND T.a 1 =x 1 AND T.a 2 =x 2 ... AND T.a n =x n * Inspired from Unterbrunner et al. in PVLDB, 2009. Storage Views September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 7
Flight Booking System Information Systems Group customers, 01, <tom, 25, SELECT C.* customers, 02, <marc, 23, FROM Tickets T , Customers C customers 03, <felix, 20, WHERE T.customer_id=C.id Customers customers, 03, <felix, 20, AND T.a 1 =x 1 .... AND T.a n =x n ..... tickets, 301, <paris, rome, E,...> Tickets tickets, 302, <moscow, berlin, B,...> tickets, 303, <tokyo, beijing, E,...> tickets, 303, <tokyo, beijing, B,..> ..... tickets.customer_id σ a 1 =x 1 .... a n =x n π customer.* ( ( )) customers.id Result Log SV customers, 01, <tom, 25, tom@abc.com, ...> customers, 02, <marc, 23, marc@abc.com, ...> tickets, 301, <paris, rome, E,...> tickets, 302, <moscow, berlin, B,...> tickets, 303, <tokyo, beijing, E,...> customers 03, <felix, 20, felix@abc.com, ...> customers, 03, <felix, 20, felix@xyz.com, ...> tickets, 303, <tokyo, beijing, B,..> ..... ..... Storage Views September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 8
Bag-partitioning Information Systems Group customers, 01, <tom, 25, SELECT C.* customers, 02, <marc, 23, FROM Tickets T , Customers C customers 03, <felix, 20, WHERE T.customer_id=C.id Customers customers, 03, <felix, 20, AND T.a 1 =x 1 .... AND T.a n =x n ..... tickets, 301, <paris, rome, E,...> tickets, 302, <moscow, berlin, B,...> tickets, 301, <paris, rome, E,...> Tickets tickets, 303, <tokyo, beijing, E,...> tickets, 302, <moscow, berlin, B,...> tickets, 303, <tokyo, beijing, B,..> tickets, 303, <tokyo, beijing, E,...> ..... tickets, 303, <tokyo, beijing, B,..> ..... tickets log σ bag=tickets Log SV Result Log SV tickets.customer_id ) ) ( σ σ ( π Log SV customer.id a 1 =x 1 ... a n =x n bag=customers * s . e r m o t u s c customers, 01, <tom, 25, customers, 02, <marc, 23, customers log tickets, 301, <paris, rome, E,...> tickets, 302, <moscow, berlin, B,...> customers, 01, <tom, 25, tom@abc.com, ...> tickets, 303, <tokyo, beijing, E,...> customers, 02, <marc, 23, marc@abc.com, ...> customers 03, <felix, 20, customers 03, <felix, 20, felix@abc.com, ...> customers, 03, <felix, 20, customers, 03, <felix, 20, felix@xyz.com, ...> tickets, 303, <tokyo, beijing, B,..> ..... ..... ..... Storage Views September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 9
Key-consolidation Information Systems Group customers, 01, <tom, 25, SELECT C.* customers, 02, <marc, 23, FROM Tickets T , Customers C customers 03, <felix, 20, WHERE T.customer_id=C.id Customers customers, 03, <felix, 20, AND T.a 1 =x 1 .... AND T.a n =x n ..... tickets, 301, <paris, rome, E,...> tickets, 301, <paris, rome, E,...> Tickets tickets, 302, <moscow, berlin, B,...> tickets, 302, <moscow, berlin, B,...> tickets, 303, <tokyo, beijing, B,..> tickets, 303, <tokyo, beijing, E,...> ..... tickets, 303, <tokyo, beijing, B,..> ..... )) σ ( ( γ Γ bag=tickets Log SV bag,key recent Result Log SV tickets.customer_id ) ) γ ( ( σ ( σ Γ bag,key ( )) π Log SV recent customer.id a 1 =x 1 ... a n =x n bag=customers * s . e r m o t u s c customers, 01, <tom, 25, customers, 02, <marc, 23, customers tickets, 301, <paris, rome, E,...> tickets, 302, <moscow, berlin, B,...> tickets, 303, <tokyo, beijing, E,...> customers, 01, <tom, 25, tom@abc.com, ...> customers 03, <felix, 20, customers, 02, <marc, 23, marc@abc.com, ...> customers, 03, <felix, 20, customers, 03, <felix, 20, felix@xyz.com, ...> tickets, 303, <tokyo, beijing, B,..> ..... ..... ..... Storage Views September 13, 2010 Towards a one-size-fits-all Database Architecture - Alekh Jindal 10
Recommend
More recommend
