a b c d a b c d a b c d a b c d a b c d a b c d a b c d a
play

A B C D A B C D A B C D A B C D A B C D A B C D A B C D A B - PowerPoint PPT Presentation

Oct 04, 2023 •224 likes •389 views

CS 591: Da Data S Systems Arch chitect ctures Prof. Manos Athanassoulis mathan@bu.edu http://manos.athanassoulis.net/classes/CS591 CS591 progress bar Storage Layouts Rows vs Cols vs Hybrid A B C D A B C D A B C D A B C D A B C D A B

  1. CS 591: Da Data S Systems Arch chitect ctures Prof. Manos Athanassoulis mathan@bu.edu http://manos.athanassoulis.net/classes/CS591

  2. CS591 progress bar Storage Layouts Rows vs Cols vs Hybrid A B C D A B C D A B C D A B C D A B C D A B C D A B C D A B C D A B C D A B C D A B C D

  3. CS591 progress bar Storage Layouts Rows vs Cols vs Hybrid New Hardware Flash Storage Multi-core

  4. CS591 progress bar Storage Layouts Rows vs Cols vs Hybrid New Hardware Flash Storage Multi-core Indexing When to use? UpBit or scan index

  5. CS591 progress bar Storage Layouts Rows vs Cols vs Hybrid New Hardware A=10 A=20 A=30 Flash Storage UB UB UB Multi-core 0 0 0 0 1 0 Indexing 0 0 1 0 0 0 When to use? UpBit 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0

  6. CS591 progress bar Storage Layouts NoSQL Engines Rows vs Cols vs Hybrid LSM-Trees Hash-based New Hardware Flash Storage Multi-core memory storage Indexing When to use? UpBit Bloom fence buf buffer filters pointers X

  7. CS591 progress bar Storage Layouts NoSQL Engines Rows vs Cols vs Hybrid LSM-Trees Hash-based New Hardware Flash Storage Multi-core Indexing When to use? UpBit Read-Copy-Update In-Place-Update Increasing Logical Address LA = 0 LA = ∞ Stable Mutable Read-Only Disk In-Memory Figure 5: Logical Address Space in

  8. CS591 progress bar Storage Layouts NoSQL Engines Rows vs Cols vs Hybrid LSM-Trees Hash-based New Hardware Indexing Flash Storage Data Skipping Multi-core Adaptive Indexing Indexing When to use? UpBit year grade course year grade course year course grade t 2 t 2 t 1 t 1 2011 A AI A 2011 AI 2012 A DB t 3 t 3 t 2 2011 B OS 2011 OS t 2 A 2011 A AI t 3 2011 B OS t 1 t 1 t 3 2012 DB B 2012 A DB t 4 2013 C DB t 4 t 4 t 4 2013 DB 2013 C DB C

  9. CS591 progress bar Storage Layouts NoSQL Engines Rows vs Cols vs Hybrid LSM-Trees Hash-based New Hardware Indexing Flash Storage Data Skipping Multi-core Adaptive Indexing Indexing When to use? Index Index Index Index Column Column Column Column UpBit < 6 < 13 >= 6 < 13 >= 13 ? sorted >= 13 < 27 Q 0 =[13,42) Q 1 =[6,27) Q 2 ... Q n < 42 >=27 < 42 >= 42 >= 42

  10. CS591 progress bar Storage Layouts NoSQL Engines Rows vs Cols vs Hybrid LSM-Trees Hash-based New Hardware Indexing Flash Storage Data Skipping Multi-core Adaptive Indexing Indexing Scientific Data Management When to use? In-situ Query Processing UpBit Adaptive Partitioning BF BF+BTree BF BTree BF BF BTree Cache Raw Data File Positional Map

  11. CS591 progress bar Storage Layouts NoSQL Engines Rows vs Cols vs Hybrid LSM-Trees Hash-based New Hardware Indexing Flash Storage Data Skipping Multi-core Adaptive Indexing Indexing Scientific Data Management When to use? In-situ Query Processing UpBit Today: Array Data

  12. Today: Array Data Storage Manager Up to now: uni uni-dim dimensio nsional nal data (integers, real, string) Array Data: mu multi-dim dimensio nsional nal data why is this a challenge? No unique order (cannot sort!) How to store? Co Concepts : multi-dimensional arrays, storage manager, tiles, thread-safe, dense vs. sparse arrays, global cell order, fragments, dense vs. sparse fragments, consolidation

  13. New Paradigms CS591 progress bar Storage Layouts NoSQL Engines Rows vs Cols vs Hybrid LSM-Trees Hash-based New Hardware Indexing Flash Storage Data Skipping Multi-core Adaptive Indexing Indexing Scientific Data Management When to use? In-situ Query Processing UpBit Today: Array Data

  14. New Paradigms CS591 progress bar Storage Layouts NoSQL Engines Rows vs Cols vs Hybrid LSM-Trees Distributed DB ML for Systems Hash-based Database Systems Automatic Data New Hardware at Global Scale Indexing System Design Flash Storage Data Skipping Learned Indexes MapReduce Multi-core Adaptive Indexing Learn Data Distributions Computing at Scale Indexing for Indexing Scientific Data Management When to use? Data Calculator In-situ Query Processing Systems for ML UpBit Synthesize Indexes Today: Array Data ML building blocks

  15. Do not forget: re reviews ws You can skip up to 3 reviews 18 classes: 5 long + 10 short + 3 skipped ne new r w rule ule : you can do extra long reviews, 1 long counts as 3 short Normally for full marks: 5 long + 10 short or 6 long + 7 short or 7 long + 4 short or 8 long + 1 short

  16. Do not forget: pr projec ect Do not leave your project work for last minute! th every group in OH to discuss progress Until Tu Tuesday April 16 th April 30 and May 2 project presentations: problem + approach + results + open questions Project presentations will also be peer-evaluated

Recommend

More recommend