manylogs
play

Manylogs Improving CMR/SMR Disk Bandwidth & Latency Tiratat - PowerPoint PPT Presentation

Oct 26, 2022 •405 likes •941 views

Manylogs Improving CMR/SMR Disk Bandwidth & Latency Tiratat Patana-anake , Vincentius Martin, Nora Sandler, Cheng Wu, and Haryadi S. Gunawi 2 Manylogs @ MSST 16 Got 100% of the read bandwidth User 1 Big Read 3 Many Got 50%

  1. Manylogs Improving CMR/SMR Disk Bandwidth & Latency Tiratat Patana-anake , Vincentius Martin†, Nora Sandler, Cheng Wu, and Haryadi S. Gunawi †

  2. 2 Manylogs @ MSST ‘16 Got 100% of the read bandwidth User 1 Big Read

  3. 3 Many Got 50% of the read bandwidth Fair Share User 1 Big Read User 2 Big Read

  4. 4 Manylogs @ MSST ‘16 1 Still Fair! 2 1/N bandwidth 3 … N

  5. 5 Manylogs @ MSST ‘16 Our Oh no! 5% bandwidth?! User 1 Big Read User 2 Small Durable Writes

  6. 6 Manylogs @ MSST ‘16 Our m More Bandwidth Faster Please! Latency Please! User 1 Big Read User 2 Small Durable Writes

  7. 7 Manylogs @ MSST ‘16 Ordered Journaling Data Journaling J J J J J J J J Journal Big Read Small Writes Seek

  8. 8 Manylogs @ MSST ‘16 Ordered Journaling Data Journaling J J J J J J J J Journal Big Read Small Writes Seek

  9. 9 Manylogs @ MSST ‘16 Problems with Current Journaling Ordered Journaling Data Journaling

  10. 10 Manylogs @ MSST ‘16 Problems with Current Journaling First Write Ordered Journaling Data Jo Second Write J

  11. 11 Manylogs @ MSST ‘16 Introducing Manylogs Single Log J Manylogs 10 MB J J J J J J J 100 MB

  12. 12 Manylogs @ MSST ‘16 Small writes made durable to the Manylogs nearest log without seeking J J J J J J J J J J J J Journal Big Read Small Writes Seek

  13. 13 Manylogs @ MSST ‘16 Manylogs q Reserved log spaces uniformly across the disk § 10 MB every 100 MB q Follow the disk head (last big I/O) q Redirect Small Writes (e.g. ≤ 256 KB) § Nearest log: log closest to last big I/O q Sequential Writes are left untouched 10 MB J J J J J J J 100 MB

  14. 14 Manylogs @ MSST ‘16 Increased Read Throughput Manylogs Reduced Write Latency

  15. 15 Manylogs @ MSST ‘16 Where are logs on the disk? 10 MB J J J J J J J 100 MB The log space = whole platter

  16. 16 Manylogs @ MSST ‘16 Where are logs on the disk? 10 MB J J J J J J J 100 MB The log space = whole platter

  17. 17 Manylogs @ MSST ‘16 Same cylinder = No seek! J J J J J J J Log for others in the same cylinder

  18. 18 Manylogs @ MSST ‘16 Latency Ratio of Max (ms) Read Bandwidth User 2 User 1 4KB Random Writes 128MB Sequential Reads At different intensities Ordered vs. Data vs. writes/s writes/s Adaptive vs. Manylogs writes/s • 320 writes/s Disk

  19. 19 Manylogs @ MSST ‘16 Adaptive Journaling q Middle ground between ordered journaling and data journaling q Single-log design q Prabhakaran et al., ATC ‘05

  20. 20 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 100% 80% Ratio of Max Bandwidth 60% 40% 20% 0% 40 80 160 320 Random Writes per Second (IOPS)

  21. 21 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 100 80% Ratio of Max Bandwidth 57% at 40 IOPS 60% 40% 20% 5% at 320 IOPS 0% 40 80 160 320 Random Writes per Second (IOPS)

  22. 22 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 100% 80% 73% at 40 IOPS Ratio of Max Bandwidth 60% 40% 20% 9% at 320 IOPS 0% 40 80 160 320 Random Writes per Second (IOPS)

  23. 23 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 100% 80% Ratio of Max Bandwidth 60% 40% 20% 0% 40 80 160 320 Random Writes per Second (IOPS)

  24. 24 Manylogs @ MSST ‘16 Ordered Data Manylogs gives daptive Manylogs the most bandwidth 50% of max 100% bandwidth at extreme IOPS 80% of Max Bandwidth 60% Hey! What about my latency? 40 80 160 320 Random Writes per Second (IOPS)

  25. 25 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 600 500 Average sync Latency (ms) 400 300 200 100 40 80 160 320 Random Writes per Second (IOPS)

  26. 26 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 600 500 Average sync Latency (ms) 400 300 200 100 0 40 80 160 320 Random Writes per Second (IOPS)

  27. 27 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 600 500 Average sync Latency (ms) 400 300 200 100 0 40 80 160 320 Random Writes per Second (IOPS)

  28. 28 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 600 500 Average sync Latency (ms) 400 300 200 100 0 40 80 160 320 Random Writes per Second (IOPS)

  29. 29 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs WOW! Fast latency 600 at extreme IOPS! 500 Average sync Latency (ms) 400 300 200 100 0 40 80 160 320 Random Writes per Second (IOPS)

  30. 30 Manylogs @ MSST ‘16 Ordered Data Results Adaptive Manylogs 100% 600 500 Average sync Latency (ms) 80% Ratio of Max Bandwidth 400 60% 300 40% 200 20% 100 0% 0 40 80 160 320 Random Writes per Second (IOPS)

  31. 31 Manylogs @ MSST ‘16 Results

  32. 32 Manylogs @ MSST ‘16 Latency Ratio of Max (ms) Read Bandwidth User 2 User 1 “fileserver” 128MB Sequential Reads • Using Filebench • Multi-threaded • 2, 4, 8 instances

  33. 33 Manylogs @ MSST ‘16 Ordered Data ogs Manylogs provides the best outcomes! 100% 4000 80% Ratio of Max Bandwidth Operation Latency (ms) 3000 60% 2000 40% 1000 20% 0% 0 2 4 8 fileserver Instances

  34. 34 Manylogs @ MSST ‘16 Checkpointing Data Journaling Manylogs q Periodically q “Lazy” or § Usually every 5 “Off-hours” secs q Rarely full q Journal can get because just filled fast small writes are because all redirected writes are in the q Log Swapping journal!

  35. 35 Manylogs @ MSST ‘16 Log Swapping J J J J Hot Area! Cold Area! J J J J J J J J Journal Big Read Small Writes Seek

  36. 36 Manylogs @ MSST ‘16 Integrations q File System (MLFS) § Durability-Only Mode (O_DUR) q SMR Disk (MLSMR) q RAID

  37. 37 Manylogs @ MSST ‘16 Cassandra Write Path Writes Commit SSTable Memtable Log Memory Disk

  38. 38 Manylogs @ MSST ‘16 Cassandra Write Path Writes Commit SSTable Memtable Log Memory Disk

  39. 39 Manylogs @ MSST ‘16 Cassandra Write Path Requires Fast Ideally SYNC Durability write Writes Commit SSTable Memtable Log But in practice, background Triggered write e.g. every 10 seconds Flush Random writes are the problem 😗 Commit Commit SSTable Memtable Log Log Temp File No location needed Memory Disk

  40. 40 Manylogs @ MSST ‘16 open(file, O_DUR); q Need fast durability but not location constraints q Content of files will be put in Manylogs regardless of the write size q Never checkpoint their content q Random writes are not a problem anymore!

  41. 41 Manylogs @ MSST ‘16 Latency Ratio of Max (ms) Read Bandwidth User 2 User 1 MongoDB HDFS • 1 instance • 2 instances • 4 instances

  42. 42 Manylogs @ MSST ‘16 Data Most Bandwidth & Lowest Manylogs Latency with Manylogs 250 MongoDB Write Latency (ms) 80% 200 Ratio of Max Bandwidth 60% 150 40% 100 20% 50 0% 0 1DB 2DB 4DB MongoDB Instances (w/Default flush period)

  43. 43 Manylogs @ MSST ‘16 Manylogs & SMR One non-shingled surface = log space

  44. 44 Manylogs @ MSST ‘16 Manylogs & SMR Non-Shingled Tracks Shingled Band

  45. 45 Manylogs @ MSST ‘16 Latency Latency (ms) (ms) User 2 User 1 Build Server Back-end Server (WBS) Trace (LM-TBE) Trace

  46. 46 Manylogs @ MSST ‘16 Manylogs SMR (MLSMR) SMR Single-log SMR (SLSMR) 100 100 80 80 60 60 Percentile 40 40 20 20 ML-SMR SL-SMR 0 0 0 10 80 30 40 50 Latency (ms)

  47. 47 Manylogs @ MSST ‘16 Manylogs & RAID User 2 User 1 Big Read Small Durable Writes Disk #1 Disk #2 Disk #3 Disk #4 Bandwidth drops Max Bandwidth! up to 50% Mingzhe Hao, Gokul Soundararajan, Deepak Kenchammana-Hosekote, Andrew A. Chien, and Haryadi S. Gunawi. "The T ail at Store: A Revelation from Millions of Hours of Disk and SSD Deployments." FAST’16.

  48. 48 Manylogs @ MSST ‘16 Latency Ratio of Max (ms) Read Bandwidth User 2 User 1 4KB Random Writes 128MB Sequential Reads At different intensities • 40 writes/s • 80 writes/s • 160 writes/s • 320 writes/s

  49. 49 Manylogs @ MSST ‘16 Ordered Data Manylogs 10x Bandwidth Speed-up 14x Latency Speed-up at 320 IOPS! 600 500 Average sync Latency (ms) 80% Ratio of Max Bandwidth 400 60% 300 40% 200 20% 100 0% 0 40 80 160 320 Random Writes per Second (IOPS) per Disk

  50. 50 Manylogs @ MSST ‘16 More in the paper q Block-Level Manylogs q Other workloads § Sequential Writes § “varmail” § More Traces q Log Size q Logged Write Size q Mapping Table

  51. 51 Manylogs @ MSST ‘16 Manylogs q Reserved log spaces uniformly across the disk q Redirect small writes to the nearest log q Can help with NoSQL , SMR , RAID , and more! q Provide up to 5x speed-up on average J J J J J J J J

  52. 52 Manylogs @ MSST ‘16 Manylogs Bandwidth Latency Speed-up Speed-up vs. Ordered 3.7x 5.7x vs. Adaptive 2.7x 2.0x vs. Single-log SMR 1.3x

  53. 53 Manylogs @ MSST ‘16 Thank you! Questions? http://ucare.cs.uchicago.edu

Recommend

More recommend