Motivations L-Store Evaluation Conclusions L-Store: A Real-time OLTP and OLAP System Mohammad Sadoghi † Souvik Bhattacharjee ‡ , Bishwaranjan Bhattacharjee # , Mustafa Canim # † Exploratory Systems Lab † University of California, Davis ‡ University of Maryland, College Park # IBM T.J. Watson EDBT’18 (March 27, 2018) Mohammad Sadoghi (UC Davis) EDBT’18 1 / 16
Motivations L-Store Evaluation Conclusions Data Management at Macroscale: The Four V’s of Big Data John Doe Mohammad Sadoghi (UC Davis) EDBT’18 2 / 16
Motivations L-Store Evaluation Conclusions Data Management at Macroscale: The Four V’s of Big Data John Doe Mohammad Sadoghi (UC Davis) EDBT’18 2 / 16
Motivations L-Store Evaluation Conclusions Data Management at Microscale: Volume & Velocity Data Velocity Sales OLTP (Write-optimized) Mohammad Sadoghi (UC Davis) EDBT’18 2 / 16
Motivations L-Store Evaluation Conclusions Data Management at Microscale: Volume & Velocity Data Velocity Data is Stale Sales OLAP Extract-Transform-Load OLTP (Read-optimized) (ETL) (Write-optimized) Reports Data Volume Mohammad Sadoghi (UC Davis) EDBT’18 2 / 16
Motivations L-Store Evaluation Conclusions Data Management at Microscale: Volume & Velocity Data Velocity Sales OLAP Extract-Transform-Load OLTP (Read-optimized) (ETL) (Write-optimized) Reports Data Volume Mohammad Sadoghi (UC Davis) EDBT’18 2 / 16
Motivations L-Store Evaluation Conclusions One Size Does not Fit All As of 2012 Mohammad Sadoghi (UC Davis) EDBT’18 3 / 16
Motivations L-Store Evaluation Conclusions One Size Does not Fit All As of 2017 Mohammad Sadoghi (UC Davis) EDBT’18 3 / 16
Motivations L-Store Evaluation Conclusions Data Management at Microscale: Volume & Velocity Sales OLAP+OLTP (Read & Write- optimized) Reports Mohammad Sadoghi (UC Davis) EDBT’18 4 / 16
Motivations L-Store Evaluation Conclusions Storage Layout Conflict Read Optimized (compressed, read-only pages) Columnar Storage Row-based Storage Write Optimized (uncompressed in-place updates) Write-optimized (i.e., uncompressed & row-based) vs. read-optimized (i.e., compressed & column-based) layouts Mohammad Sadoghi (UC Davis) EDBT’18 5 / 16
Motivations L-Store Evaluation Conclusions Unifying OLTP and OLAP: Velocity & Volume Dimensions Observed Trends In operational databases, there is a pressing need to close the gap between the write-optimized layout for OLTP (i.e., row-wise) and the read-optimized layout for OLAP (i.e., column-wise). Mohammad Sadoghi (UC Davis) EDBT’18 6 / 16
Motivations L-Store Evaluation Conclusions Unifying OLTP and OLAP: Velocity & Volume Dimensions Observed Trends In operational databases, there is a pressing need to close the gap between the write-optimized layout for OLTP (i.e., row-wise) and the read-optimized layout for OLAP (i.e., column-wise). Introducing a lineage-based storage architecture , a contention-free update mechanism over a native columnar storage in order to Mohammad Sadoghi (UC Davis) EDBT’18 6 / 16
Motivations L-Store Evaluation Conclusions Unifying OLTP and OLAP: Velocity & Volume Dimensions Observed Trends In operational databases, there is a pressing need to close the gap between the write-optimized layout for OLTP (i.e., row-wise) and the read-optimized layout for OLAP (i.e., column-wise). Introducing a lineage-based storage architecture , a contention-free update mechanism over a native columnar storage in order to lazily and independently stage stable data from a write-optimized layout (i.e., OLTP) into a read-optimized layout (i.e., OLAP) Mohammad Sadoghi (UC Davis) EDBT’18 6 / 16
Motivations L-Store Evaluation Conclusions Lineage-based Storage Architecture (LSA): Intuition Index Points to Stable LIDs (i.e., anchored RID) RID j In-page Lineage Tacking Tail Pages (Append-only) Base Pages Latest Version (Read-only) RID k (monotonically Base increasing RIDs) Version RID i ( anchored RIDs) RID i Lineage Mapping (indirection layer, stable LID-to-RID mapping) Physical Update Independence: De-coupling data & its updates (reconstruction via in-page lineage tracking and lineage mapping) Mohammad Sadoghi (UC Davis) EDBT’18 7 / 16
Motivations L-Store Evaluation Conclusions Lineage-based Storage Architecture (LSA): Intuition Monotonically Increasing Lineage (updates are assigned RIDs in an increasing order) Index Points to Stable LIDs (i.e., anchored RID) RID j In-page Lineage Tacking Tail Pages (Append-only) Base Pages Latest Version (Read-only) RID k (monotonically Base increasing RIDs) Version RID i ( anchored RIDs) RID i Append-only Updates (physical update independence) Lineage Mapping (indirection layer, stable LID-to-RID mapping) Physical Update Independence: De-coupling data & its updates (reconstruction via in-page lineage tracking and lineage mapping) Mohammad Sadoghi (UC Davis) EDBT’18 7 / 16
Motivations L-Store Evaluation Conclusions Lineage-based Storage Architecture (LSA): Intuition Monotonically Monotonically Increasing Increasing Lineage In-page Lineage (updates are assigned RIDs Index in an increasing order) Lazy Update Consolidation (snapshot reconstruction via lineage mapping & in-page tracking) Points to Stable LIDs (i.e., anchored RID) RID j In-page Lineage Tacking Tail Pages (Append-only) Base Pages Latest In-page Lineage Tacking Version (Read-only) RID k (monotonically Base increasing RIDs) Version RID i Consolidated Data (stable anchored RIDs) (Read-only) RID i Data Block RIDs Remain Unchanged Append-only (stable reference, anchored RIDs) Updates (physical update independence) Lineage Mapping (indirection layer, stable LID-to-RID mapping) Physical Update Independence: De-coupling data & its updates (reconstruction via in-page lineage tracking and lineage mapping) Mohammad Sadoghi (UC Davis) EDBT’18 7 / 16
Motivations L-Store Evaluation Conclusions Lineage-based Storage Architecture (LSA): Overview Columnar Storage Base Pages (read-only) Tail Pages Page Directory (append-only) Range Partitioning Record (spanning over a set of aligned columns) Overview of the lineage-based storage architecture ( base pages and tail pages are handled identically at the storage layer) Mohammad Sadoghi (UC Davis) EDBT’18 8 / 16
Motivations L-Store Evaluation Conclusions L-Store: Detailed Design Read Optimized (compressed, read-only pages) Columnar Storage Range Partitioning Base Pages (read-only) Records are range-partitioned and compressed into a set of ready-only base pages (accelerating analytical queries) Mohammad Sadoghi (UC Davis) EDBT’18 9 / 16
Motivations L-Store Evaluation Conclusions L-Store: Detailed Design Read Optimized Updated Columns (compressed, read-only pages) Corresponding Write Optimized Columns (uncompressed, append-only updates) Base Pages Tail Pages (read-only) (append-only) Recent updates for a range of records are clustered in their tails pages (transforming costly point updates into an amortized analytical-like query) Mohammad Sadoghi (UC Davis) EDBT’18 9 / 16
Motivations L-Store Evaluation Conclusions L-Store: Detailed Design Read Optimized Updated Columns (compressed, read-only pages) Different Versions of the Record Tail Record Base Record (latest version) (older version) Write Optimized (uncompressed, append-only updates) Base Pages Tail Pages (read-only) (append-only) Recent updates for a range of records are clustered in their tails pages (transforming costly point updates into an amortized analytical-like query) Mohammad Sadoghi (UC Davis) EDBT’18 9 / 16
Motivations L-Store Evaluation Conclusions L-Store: Detailed Design Read Optimized (compressed, read-only pages) Pre-allocated Space (lazily) Write Optimized (uncompressed, append-only updates) Base Pages Tail Pages (read-only) (append-only) Recent updates are strictly appended, uncompressed in the pre-allocated space (eliminating the read/write contention) Mohammad Sadoghi (UC Davis) EDBT’18 9 / 16
Motivations L-Store Evaluation Conclusions L-Store: Detailed Design Read Optimized (compressed, read-only pages) Indirection Column (back pointer to the previous version) Forward Pointer to the Latest Version of the Record Write Optimized (uncompressed, append-only updates) Indirection Column (uncompressed, in-place update) Achieving (at most) 2-hop access to the latest version of any record (avoiding read performance deterioration for point queries) Mohammad Sadoghi (UC Davis) EDBT’18 9 / 16
Motivations L-Store Evaluation Conclusions L-Store: Detailed Design Read Optimized (compressed, read-only pages) Indirection Column (back pointer to the previous version) Write Optimized New Version (uncompressed, append-only updates) Indirection Column (uncompressed, in-place update) Achieving (at most) 2-hop access to the latest version of any record (avoiding read performance deterioration for point queries) Mohammad Sadoghi (UC Davis) EDBT’18 9 / 16
Recommend
More recommend
