fla lashnet t fla lash netw twork ork sta tack c k co de
play

Fla lashNet: t: Fla lash/Netw twork ork Sta tack C k Co-De - PowerPoint PPT Presentation

May 25, 2023 •124 likes •605 views

Fla lashNet: t: Fla lash/Netw twork ork Sta tack C k Co-De Design ign Animesh Trivedi, Nikolas Ioannou, Bernard Metzler, Patrick Stuedi, Jonas Pfefferle, Ioannis Koltsidas, Kornilios Kourtis, and Thomas R. Gross IBM Research and ETH

  1. Fla lashNet: t: Fla lash/Netw twork ork Sta tack C k Co-De Design ign Animesh Trivedi, Nikolas Ioannou, Bernard Metzler, Patrick Stuedi, Jonas Pfefferle, Ioannis Koltsidas, Kornilios Kourtis, and Thomas R. Gross IBM Research and ETH Zurich, Switzerland

  2. Modern Distributed Systems  data intensive  run on 100-1000s of servers  performance depends upon both network and storage SYSTOR 2017, Haifa 2

  3. Modern Distributed Systems  performance depends upon both network and storage SYSTOR 2017, Haifa 3

  4. Modern Distributed Systems ● StackMap: Low-Latency Networking with the OS Stack and Dedicated NICs, USENIX'16 ● Network Stack Specialization for Performance, SIGCOMM'14 ● mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems, NSDI'14 ● MegaPipe: A New Programming Interface for Scalable Network I/O, OSDI'12  performance depends upon ● … both network and storage SYSTOR 2017, Haifa 4

  5. Modern Distributed Systems ● StackMap: Low-Latency Networking with the OS Stack and Dedicated NICs, USENIX'16 ● Network Stack Specialization for Performance, SIGCOMM'14 ● mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems, NSDI'14 ● MegaPipe: A New Programming Interface for Scalable Network I/O, OSDI'12  performance depends upon ● … both network and storage ● NVMeDirect: A User-space I/O Framework for Application-specific Optimization on NVMe SSDs, HotStorage'16 ● OS I/O Path Optimizations for Flash Solid-state Drives, USENIX'14 ● Linux Block IO: Introducing Multi-queue SSD Access on Multi-core Systems, SYSTOR'13 ● When Poll is Better Than Interrupt, FAST'12 ● … SYSTOR 2017, Haifa 5

  6. Modern Distributed Systems ● StackMap: Low-Latency Networking with the OS Stack and Dedicated NICs, USENIX'16 ● Network Stack Specialization for Performance, SIGCOMM'14 ● mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems, NSDI'14 ● MegaPipe: A New Programming Interface for Scalable Network I/O, OSDI'12  performance depends upon ● … both network and storage ● NVMeDirect: A User-space I/O Framework for Application-specific Optimization on NVMe SSDs, HotStorage'16 ● OS I/O Path Optimizations for Flash Solid-state Drives, USENIX'14 ● Linux Block IO: Introducing Multi-queue SSD Access on Multi-core Systems, SYSTOR'13 ● When Poll is Better Than Interrupt, FAST'12 ● … SYSTOR 2017, Haifa 6

  7. Modern Distributed Systems ● StackMap: Low-Latency Networking with the OS Stack and Dedicated NICs, USENIX'16 ● Network Stack Specialization for Performance, SIGCOMM'14 ● mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems, NSDI'14 ● MegaPipe: A New Programming Interface for Scalable Network I/O, OSDI'12  performance depends upon ● … both network and storage ● NVMeDirect: A User-space I/O Framework for Application-specific Optimization on NVMe SSDs, HotStorage'16 ● OS I/O Path Optimizations for Flash Solid-state Drives, USENIX'14 ● Linux Block IO: Introducing Multi-queue SSD Access on Multi-core Systems, SYSTOR'13 ● When Poll is Better Than Interrupt, FAST'12 ● … SYSTOR 2017, Haifa 7

  8. The Cost of the Gap 1400 1200 1000 800 K IOPS 600 400 200 0 spec. block IO netperf iSCSI KV NFS HDFS SYSTOR 2017, Haifa 8

  9. The Cost of the Gap 1400 1200 1000 800 K IOPS 600 400 200 0 spec. block IO netperf iSCSI KV NFS HDFS SYSTOR 2017, Haifa 9

  10. The Cost of the Gap 1400 1200 1000 800 K IOPS 600 400 200 0 spec. block IO netperf iSCSI KV NFS HDFS SYSTOR 2017, Haifa 10

  11. The Reason for the Gap client flash storage SYSTOR 2017, Haifa 11

  12. The Reason for the Gap 1 request 2 request processing client flash 3 response storage SYSTOR 2017, Haifa 12

  13. The Reason for the Gap 1 request 2 request processing client flash 3 response storage performance = network IO + server time + storage IO SYSTOR 2017, Haifa 13

  14. The Reason for the Gap 1 request 2 request processing client flash 3 response storage performance = network IO + server time + storage IO application involvement scheduling fs lookups and overheads ... SYSTOR 2017, Haifa 14

  15. A Detailed Look: send usersp userspace kern ernel 1. TCP/IP processing SYSTOR 2017, Haifa 15

  16. A Detailed Look: send 3. receive request usersp userspace kern ernel 2. receive processing 1. TCP/IP processing SYSTOR 2017, Haifa 16

  17. A Detailed Look: send 3. receive request usersp userspace kern ernel 4. fs translation 2. receive processing 5. block I/O 1. TCP/IP processing SYSTOR 2017, Haifa 17

  18. A Detailed Look: send 3. receive 7. response request transmission userspace usersp kern ernel 4. fs translation 2. receive processing 5. block I/O 1. TCP/IP 6. flash I/O processing completion SYSTOR 2017, Haifa 18

  19. A Detailed Look: send 3. receive 7. response request transmission userspace usersp kern ernel 4. fs translation 8. send 2. receive processing 5. block I/O 1. TCP/IP 6. flash I/O 9.TX done processing completion SYSTOR 2017, Haifa 19

  20. A Detailed Look: sendfjle 3. receive 7. response request transmission userspace usersp kern ernel 4. fs translation 8. send 2. receive processing 5. block I/O 1. TCP/IP 6. flash I/O 9.TX done processing completion SYSTOR 2017, Haifa 20

  21. A Detailed Look: sendfjle 3. receive 7. response request transmission userspace usersp kern ernel 4. fs translation 8. send 2. receive processing 5. block I/O 1. TCP/IP 6. flash I/O 9.TX done processing completion SYSTOR 2017, Haifa 21

  22. The FlashNet Approach 3. receive request usersp userspace kern ernel 4. fs translation 2. receive processing 5. block I/O 1. TCP/IP processing SYSTOR 2017, Haifa 22

  23. The FlashNet Approach eliminate application involvement 3. receive request userspace usersp kern ernel 4. fs translation 2. receive processing 5. block I/O 1. TCP/IP processing SYSTOR 2017, Haifa 23

  24. The FlashNet Approach eliminate application involvement 3. receive request usersp userspace reduce file kern ernel 4. fs translation system overheads 2. receive processing 5. block I/O 1. TCP/IP processing SYSTOR 2017, Haifa 24

  25. The FlashNet Approach eliminate application involvement 3. receive request usersp userspace reduce file kern ernel 4. fs translation system overheads 2. receive processing 5. block I/O enable direct network and storage interaction 1. TCP/IP processing SYSTOR 2017, Haifa 25

  26. The FlashNet Approach eliminate application involvement RDMA usersp userspace reduce file kern ernel 4. fs translation system overheads 2. RDMA processing 5. block I/O enable direct network and storage interaction 1. TCP/IP processing SYSTOR 2017, Haifa 26

  27. The FlashNet Approach eliminate application involvement RDMA usersp userspace reduce file kern ernel simple fs layout system overheads 2. RDMA processing 5. block I/O enable direct network and storage interaction 1. TCP/IP processing SYSTOR 2017, Haifa 27

  28. The FlashNet Approach eliminate application involvement RDMA usersp userspace reduce file kern ernel simple fs layout system overheads 2. RDMA processing use VM 5. block I/O enable direct network and storage interaction 1. TCP/IP processing SYSTOR 2017, Haifa 28

  29. The FlashNet Approach usersp userspace 2. RDMA v to L 6. response kern ernel a B A transmission 2. RDMA processing 3. block I/O 1. TCP/IP 6. flash I/O 7.TX done processing completion SYSTOR 2017, Haifa 29

  30. FlashNet: A Co-Designed Network and Storage Stack 64-bit LBA space flash controller ● flash virtualization ● I/O management ● ... SYSTOR 2017, Haifa 30

  31. FlashNet: A Co-Designed Network and Storage Stack ● contigious file allocation ● supporting m & local file I/O m a p ● … ContigFS flash controller SYSTOR 2017, Haifa 31

  32. FlashNet: A Co-Designed Network and Storage Stack ● lazy RDMA pinning ● resolving flash & file addresses ● … ContigFS RDMA controller flash controller SYSTOR 2017, Haifa 32

  33. FlashNet: A Co-Designed Network and Storage Stack FlashNet I/O stack ContigFS RDMA controller flash controller SYSTOR 2017, Haifa 33

  34. FlashNet: A Co-Designed Network and Storage Stack file server application virtual address LBA ContigFS RDMA STag PBA controller flash controller network control setup expanding to storage SYSTOR 2017, Haifa 34

  35. FlashNet: A Co-Designed Network and Storage Stack file server application virtual address LBA ContigFS RDMA STag PBA controller flash controller network control setup expanding to storage data path from a flash device to a client buffer SYSTOR 2017, Haifa 35

  36. FlashNet: A Co-Designed Network and Storage Stack client server application nfs ContigFS RDMA iSCSI controller flash RNIC controller SKB network control setup expanding to storage data path from a flash device to a client buffer SYSTOR 2017, Haifa 36

  37. Performance Evaluation How efficient is FlashNet's IO path? Does it help with applications? ...more in the paper 9-machine cluster testbed CPU : dual socket E5-2690, 2.9 GHz, 16 cores DRAM : 256 GB, DDR3 1600 MHz NIC : 40Gbit/s Ethernet 3xNVMe Flash : 6.6 GB/sec (read), 2.7 GB/sec (write) peak 4kB read IOPS: 1.3 M SYSTOR 2017, Haifa 37

Recommend

NE NETW TWORK ORK Presented by: Eng. S. M. Kinoti Ag. DG -KURA TH FEBR UB ON 8 TH Meeti

NE NETW TWORK ORK Presented by: Eng. S. M. Kinoti Ag. DG -KURA TH FEBR UB ON 8 TH Meeti

1 UPGRADING RADING OF NAI AIROB ROBI I URB RBAN AN RO ROAD AD NE NETW TWORK ORK Presented by: Eng. S. M. Kinoti Ag. DG -KURA TH FEBR UB ON 8 TH Meeti ting g at NAIR IROB OBI I CLUB BRUAR ARY, 2017 17 OUTL TLINE INE 2

510 views • 30 slides
cryptography in ITU-T and ISO/IEC Hao Hao Qi Qin* CAS Q S Quant ntum Netw twork rk Co., .,

cryptography in ITU-T and ISO/IEC Hao Hao Qi Qin* CAS Q S Quant ntum Netw twork rk Co., .,

QCrypt 2020 Industry session August 12, 2020 E-meeting Standardization of quantum cryptography in ITU-T and ISO/IEC Hao Hao Qi Qin* CAS Q S Quant ntum Netw twork rk Co., ., Ltd td. *qinhao@casquantumnet.com Quantum key distribution:

161 views • 13 slides
Ultra L Low Power er Infer eren ence a e at the v very e edge o of the n netw twork

Ultra L Low Power er Infer eren ence a e at the v very e edge o of the n netw twork

Ultra L Low Power er Infer eren ence a e at the v very e edge o of the n netw twork Tiny ML Summit March 20-21 2019 Sunnyvale Eric Flamand, CTO & CoFounder of Greenwaves Technologies Wh Who a o are w we? French based startup

644 views • 25 slides
Trackin acking g th the e Rus ussi sian n Bus usin ines ess s Netw twork rk (RBN)

Trackin acking g th the e Rus ussi sian n Bus usin ines ess s Netw twork rk (RBN)

Trackin acking g th the e Rus ussi sian n Bus usin ines ess s Netw twork rk (RBN) Knowing and understanding the RBN is a useful objective, but surely the main goal is to stop them? RBNexploit.blogspot.com Cambridge University

554 views • 34 slides
How to Displa lay Multiple Ad Netwo works? What is an Ad Netw twork? Popular Ad network

How to Displa lay Multiple Ad Netwo works? What is an Ad Netw twork? Popular Ad network

How to Displa lay Multiple Ad Netwo works? What is an Ad Netw twork? Popular Ad network rks How to integrate Ad Ad Networks Revenu nue Models There are three differ ferent ways app developers make re revenue from their apps

1.04k views • 40 slides
Fi Financing nancing the gas gas tr trans ansmis mission sion netw twork ork Analyzing

Fi Financing nancing the gas gas tr trans ansmis mission sion netw twork ork Analyzing

Fi Financing nancing the gas gas tr trans ansmis mission sion netw twork ork Analyzing the effects of duration dependent multipliers Eren am | Dominic minic Lencz ncz | Simon Schulte | ENERDAY - Dresden | 12.04.2019 Agenda da

208 views • 16 slides
New York Ne ork & Ne New Jersey UpdateJu July y 2018 Ne Netw twork rk Membe mber

New York Ne ork & Ne New Jersey UpdateJu July y 2018 Ne Netw twork rk Membe mber

New York Ne ork & Ne New Jersey UpdateJu July y 2018 Ne Netw twork rk Membe mber r Webi bina nar Hosted by: William OHearn, Communications/Outreach Manager, NJ and NY Elizabeth Barminski, Working Group Coordinator

703 views • 21 slides
Maxi ximis isin ing customer valu lue fr from th the netw twork in in a hig igh-DER fu

Maxi ximis isin ing customer valu lue fr from th the netw twork in in a hig igh-DER fu

1 SA Power Networks Maxi ximis isin ing customer valu lue fr from th the netw twork in in a hig igh-DER fu futu ture AEMC / ARENA Regulatory DEIP dive, 6 th June 2019 The fu futu ture of f th the dis istribution network

283 views • 16 slides
Artificial Artificial neur neural al ne netw twork ork alg algorithm rithms fo for Heal

Artificial Artificial neur neural al ne netw twork ork alg algorithm rithms fo for Heal

Artificial Artificial neur neural al ne netw twork ork alg algorithm rithms fo for Heal Healthcar care Decision Decisi on Syste Systems and and Di Diagnos agnostic ic Im Image Processing ocessing T A H M I N A Z E B I N R E S E A R C H F E

649 views • 27 slides
Building the gigabit society Eelco Blok KPN at a glance Mobile 4G netw twork rk with 99%

Building the gigabit society Eelco Blok KPN at a glance Mobile 4G netw twork rk with 99%

BoR (16) 199 Building the gigabit society Eelco Blok KPN at a glance Mobile 4G netw twork rk with 99% 99% 99% geographi hic and popula lation ion coverage World ld first st natio ionwid ide LoRa network ork for 99% 99%

325 views • 8 slides
Multi lti-le level l Attentio ion Netw twork rk usin sing Text xt, Audio io and Vide

Multi lti-le level l Attentio ion Netw twork rk usin sing Text xt, Audio io and Vide

Multi lti-le level l Attentio ion Netw twork rk usin sing Text xt, Audio io and Vide ideo for r Depressio ion Predic ictio ion Anupama Ray Siddharth Kumar, Rutvik Reddy, Prerana Mukherjee Ritu Garg IBM Research, India

634 views • 19 slides
Approach to Coding Reform A Smart Gr A Smart Growth th Ne Netw twork W ork Web ebin inar

Approach to Coding Reform A Smart Gr A Smart Growth th Ne Netw twork W ork Web ebin inar

Enabling Better Places: An Incremental Approach to Coding Reform A Smart Gr A Smart Growth th Ne Netw twork W ork Web ebin inar ar July July 17, 17, 2019 2019 Lyn ynn Rich Richar ards: Con s: Congr gress f ess for th or the Ne

878 views • 32 slides
The he Capac apacity ity of of th the Ag e Aging ng Service vices s Netw twork: ork:

The he Capac apacity ity of of th the Ag e Aging ng Service vices s Netw twork: ork:

The he Capac apacity ity of of th the Ag e Aging ng Service vices s Netw twork: ork: Top op 10 Thi hings ngs to to Kno now w fo for Reautho uthorization rization 1 1965 1965 1972 1972 1973 1973 1975 1975 OAA,

771 views • 40 slides
Con Convol oluti tion onal Neural Netw twork orks Presented by Tristan Maidment Adapted

Con Convol oluti tion onal Neural Netw twork orks Presented by Tristan Maidment Adapted

Con Convol oluti tion onal Neural Netw twork orks Presented by Tristan Maidment Adapted from Ke Yus Slides Ou Outlin line Neural Network recap Building blocks of CNNs Architecture of CNNs Visualizing and understanding

808 views • 70 slides
Lectu ture 6 6 reca recap Prof. Leal-Taix and Prof. Niessner 1 Ne Neural Ne Netw twork

Lectu ture 6 6 reca recap Prof. Leal-Taix and Prof. Niessner 1 Ne Neural Ne Netw twork

Lectu ture 6 6 reca recap Prof. Leal-Taix and Prof. Niessner 1 Ne Neural Ne Netw twork Width Depth Prof. Leal-Taix and Prof. Niessner 2 Gr Gradi dient De Descent fo for Neural Netwo works ks " =: ! " ' " (

1.42k views • 79 slides
in Regards to Tack. David Branscome Jr. Operations Branscome Paving Company Effects on Tack

in Regards to Tack. David Branscome Jr. Operations Branscome Paving Company Effects on Tack

Problems Contractors Face in Regards to Tack. David Branscome Jr. Operations Branscome Paving Company Effects on Tack Due to Short Time Frame Reduced hours of actual paving lay down time Keeping quality while trying to get the

457 views • 11 slides
1 in a Nutshell 2019 Pass the S ALT Workshop Overview 2 Introduction to Elastic S tack

1 in a Nutshell 2019 Pass the S ALT Workshop Overview 2 Introduction to Elastic S tack

Elastic S tack for S ecurity Monitoring 1 in a Nutshell 2019 Pass the S ALT Workshop Overview 2 Introduction to Elastic S tack Beats Logstash Elasticsearch Kibana Elastic S tack Alerting and S ecurity Introductory Workshop! 3

809 views • 54 slides
Copying Garbage Collection Guido Tack 14. November 2001 tack@ps.uni-sb.de 1 The idea of Copying

Copying Garbage Collection Guido Tack 14. November 2001 tack@ps.uni-sb.de 1 The idea of Copying

Copying Garbage Collection Guido Tack 14. November 2001 tack@ps.uni-sb.de 1 The idea of Copying GC Two semi-spaces ( From -space and To -space) Only From-space active At GC time, copy the live nodes from From-Space to

1.45k views • 116 slides
On Evolution of On Evolution of C2 Netw ork Topology C2 Netw ork Topology IC ICCR CRTS 2010

On Evolution of On Evolution of C2 Netw ork Topology C2 Netw ork Topology IC ICCR CRTS 2010

On Evolution of On Evolution of C2 Netw ork Topology C2 Netw ork Topology IC ICCR CRTS 2010 TS 2010 Paper 082 National Key Laboratory of Science and Technology on C4ISR Dr. Songhua Huang NRIEE, CETC, China Introduction C2 netw orks

458 views • 16 slides
SCADA SCADA Sec Securit urity SC SCADA Ne Netwo twork rk Sec Security urity Jodi Jensen

SCADA SCADA Sec Securit urity SC SCADA Ne Netwo twork rk Sec Security urity Jodi Jensen

SCADA SCADA Sec Securit urity SC SCADA Ne Netwo twork rk Sec Security urity Jodi Jensen Operations Support Manager Western Area Power Administration Sub Substation Ne station Netwo twork rk Sec Security urity Tyler Stinson

627 views • 13 slides
Interpretability and Visualization of Deep Neural Networks Au Aude Oliva MI MIT Convoluti

Interpretability and Visualization of Deep Neural Networks Au Aude Oliva MI MIT Convoluti

Interpretability and Visualization of Deep Neural Networks Au Aude Oliva MI MIT Convoluti tional Neura ral Netw twork rks convolution max-pooling normalization full connected Alexnet Ea Each ch laye yer learns prog ogressive

356 views • 34 slides
Ne NetBouncer uncer: A : Act ctiv ive D e Device and ice and Link Failure Lo Li

Ne NetBouncer uncer: A : Act ctiv ive D e Device and ice and Link Failure Lo Li

Ne NetBouncer uncer: A : Act ctiv ive D e Device and ice and Link Failure Lo Li Localization in Data Ce Center r Netw twork orks Presented by Akash Kulkarni Problems that may occur in Data Center Routing misconfigurations

670 views • 22 slides
SDG DG 4.1.1 .1 Measur asurem ement ent Tools ools - Ap Applicat licatio ions ns in th

SDG DG 4.1.1 .1 Measur asurem ement ent Tools ools - Ap Applicat licatio ions ns in th

GAM AML L Webinar binar SDG DG 4.1.1 .1 Measur asurem ement ent Tools ools - Ap Applicat licatio ions ns in th the Wak ake e of COV OVID ID-19 19 Dr. Modupe upe Adefe feso so-Olat Olateju ju PAL AL Netw twork/T ork/TEP

558 views • 7 slides
Session 2 : European Medicines Netw ork Coordinating role of the Agency w ithin the netw ork-

Session 2 : European Medicines Netw ork Coordinating role of the Agency w ithin the netw ork-

Session 2 : European Medicines Netw ork Coordinating role of the Agency w ithin the netw ork- clarifying roles and responsibilities Presented by: Dr Peter Arlett EMA An agency of the European Union Content A bit of law A bit of

821 views • 17 slides

More recommend