external buffer
play

External buffer Raslan Darawsheh Mellanox External buffer First - PowerPoint PPT Presentation

Dec 30, 2022 •419 likes •544 views

x External buffer Raslan Darawsheh Mellanox External buffer First was introduced by Olivier in his presentation in 2016. The support for External buffer has been there since DPDK 18.05 2 External buffer: Contd 3 External

  1. x External buffer Raslan Darawsheh Mellanox

  2. External buffer • First was introduced by Olivier in his presentation in 2016. The support for External buffer has been there since DPDK 18.05​ • 2

  3. External buffer: Cont’d 3

  4. External buffer: Cont’d 4

  5. External buffer: Cont’d mbuf EXT_ATTACHED_MBUF struct rte_mbuf_ext_shared_info { rte_mbuf_extbuf_free_callback_t free_cb; /**< Free callback function */ mbuf void *fcb_opaque; /**< Free callback argument */ rte_atomic16_t refcnt_atomic; /**< Atomically accessed refcnt */ buf_addr }; data_off refcnt ▪ Attach external buffer (non-mbuf) to a mbuf rte_mbuf: pkt_len 128 data_len • Attached rte_pktmbuf_attach_extbuf() buf_len shinfo - shinfo must be provided External buffer priv:usually 0 User managed buffer ▪ HEADROOM:128 HEADROOM:128 rte_pktmbuf_ext_shinfo_init_helper() ▪ › Helper function to simply spare a few bytes at the end of the buffer for shared data. - m->ol_flags |= EXT_ATTACHED_MBUF; - rte_pktmbuf_reset_headroom() is needed to packet reset HEADROOM Shared Data • Detached by refcnt=1 rte_pktmbuf_detach_extbuf() - Same as rte_pktmbuf_detach() - rte_mbuf_ext_refcnt_update(m->shinfo, -1); - Call shinfo->free_cb() if m->shinfo->refcnt gets zero 5

  6. External Buffer USE CASES

  7. Use cases : #1 Storage applications External Buffer mbuf(i) Single buffer is shared by multiple Shared Data shinfo Mbufs. refcnt=3 buf_addr + data_off No need to copy data to mbuf for Tx Common shinfo external buffer is read- only packet(i) mbuf(j) shinfo buf_addr + data_off mbuf(k) packet (j) shinfo buf_addr + data_off packet (k) 7

  8. Use cases: #2 GPU GPUs (Graphics Processing Units) are being used to accelerate complex and time consuming tasks in a range of application​ . Typically GPU’s don’t manage the packet send/receive , only the data processing. With the current mbuf scheme the GPU must copy in the packet data from the host memory. The external mbuf enables true zero copy for the GPU and hence improve the performance significantly​ 8

  9. Use cases: GPU – Cont’d The Mempool on host memory is populated w/ mbuf descriptor allocated from the host memory and it has some external buffer attached to it. 9

  10. Default behavior With External buffer Rx-pps: 5532097 ~ 5.5 Mpps. Rx-pps: 10869938 ~ 10.8 Mpps. Single Core Tx-pps: 5539771 ~ 5.5 Mpps. Tx-pps: 10869938 ~ 10.8 Mpps. CPU cycles/packet=213 CPU cycles/packet=98 (217% improvement). GPU use case Some testpmd POC Result Default behavior With External buffer RX-pps: 11852669 ~ 11.8 Mpps. Rx-pps: 38054341 ~ 38.05 Mpps. 4 Cores Tx-pps: 11852669 ~ 11.8 Tx-pps: 38054341 ~ 38.05 Mpps. Mpps. CPU cycles/packet=910 CPU cycles/packet=160 (568% Improvement).

  11. IND_ATTACHED_MBUF or EXT_ATTACHED_MBUF is set in m- • >ol_flags even though it is not an offload. Open question: • Would It be better to have an explicit way to say an mbuf is read • only ? Currently, it is implicit. mbuf/ext_buf having refcnt > 1 . Future plan

  12. “ Thank you all ”

Recommend

Staff has concerns over the setbacks of the second floor patio and will be conditioning that the

Staff has concerns over the setbacks of the second floor patio and will be conditioning that the

From: Liska, Andrew &lt;Andrew.Liska@minneapolismn.gov&gt; Sent: Wednesday, August 19, 2020 2:34 PM To: Michael Subject: RE: [EXTERNAL] RE: [EXTERNAL] RE: [EXTERNAL] RE: [EXTERNAL] RE: [EXTERNAL] RE: [EXTERNAL] 1219 31st W Joyce Church Hi,

366 views • 3 slides
Massive Data Algorithmics Lecture 5: External Search Trees Massive Data Algorithmics Lecture 5:

Massive Data Algorithmics Lecture 5: External Search Trees Massive Data Algorithmics Lecture 5:

Construction Buffer tree Massive Data Algorithmics Lecture 5: External Search Trees Massive Data Algorithmics Lecture 5: External Search Trees Construction Buffer tree B-tree Construction In internal memory we can sort N elements in O ( N

281 views • 13 slides
Buffer Manager Each relation is can be mapped onto many files (each file containing data from one

Buffer Manager Each relation is can be mapped onto many files (each file containing data from one

Buffer Management The database buffer is the mediator between the basic file system and the tuple-oriented file system. The buffer managers task is to make the pages addressable in main memory and to coordinate the writing of pages to disk

1.12k views • 8 slides
Introduction Buffer Overflows Buffer overflows were the most common form of security

Introduction Buffer Overflows Buffer overflows were the most common form of security

Introduction Buffer Overflows Buffer overflows were the most common form of security vulnerability in the 90s. Buffer overflows dominate in the area of remote network penetration vulnerabilities. CS 465 They represent one of the

651 views • 4 slides
Buffer Overflows with Content 2 A Process Stack Buffer Overflow Common Techniques employed

Buffer Overflows with Content 2 A Process Stack Buffer Overflow Common Techniques employed

1 Buffer Overflows with Content 2 A Process Stack Buffer Overflow Common Techniques employed in buffer overflow exploits to create backdoors Execution of additional network services via the INETD daemon The addition of new users

497 views • 30 slides
Lab 2: Buffer Overflows Fengwei Zhang SUSTech CS 315 Computer Security 1 Buffer Overflows

Lab 2: Buffer Overflows Fengwei Zhang SUSTech CS 315 Computer Security 1 Buffer Overflows

Lab 2: Buffer Overflows Fengwei Zhang SUSTech CS 315 Computer Security 1 Buffer Overflows One of the most common vulnerabilities in software Programming languages commonly associated with buffer overflows including C and C++

1.14k views • 17 slides
Buffer Overflows and Defenses CS 419: Computer Security Lecture 10 and 11 Buffer Overflow a

Buffer Overflows and Defenses CS 419: Computer Security Lecture 10 and 11 Buffer Overflow a

Buffer Overflows and Defenses CS 419: Computer Security Lecture 10 and 11 Buffer Overflow a very common attack mechanism from 1988 Morris Worm to Code Red, Slammer, Sasser and many others prevention techniques known still of major concern

873 views • 53 slides
TinyOS Determine when Fill message Specify Pass buffer message buffer Network Communication

TinyOS Determine when Fill message Specify Pass buffer message buffer Network Communication

Inter-Node Communication General idea: Sender: TinyOS Determine when Fill message Specify Pass buffer message buffer Network Communication buffer with data Recipients to OS can be reused Computer Network Receiver:

176 views • 3 slides
Week 03 Lectures PostgreSQL Buffer Manager 1/95 PostgreSQL buffer manager: provides a shared

Week 03 Lectures PostgreSQL Buffer Manager 1/95 PostgreSQL buffer manager: provides a shared

Week 03 Lectures PostgreSQL Buffer Manager 1/95 PostgreSQL buffer manager: provides a shared pool of memory buffers for all backends all access methods get data from disk via buffer manager Buffers are located in a large region of shared

505 views • 25 slides
Riparian Buffer or Riparian Forest Buffer Equivalency Demonstration and Offsetting Water

Riparian Buffer or Riparian Forest Buffer Equivalency Demonstration and Offsetting Water

Implementation of Act 162 of 2014 Riparian Buffer or Riparian Forest Buffer Equivalency Demonstration and Offsetting Water Resources Advisory Committee August 12, 2015 Tom Wolf, Governor John Quigley, Secretary Agenda 1. Overview of Act 162

530 views • 37 slides
Chapter 10 Buffer Overflow Buffer Overflow Common attack mechanism first wide use by

Chapter 10 Buffer Overflow Buffer Overflow Common attack mechanism first wide use by

Chapter 10 Buffer Overflow Buffer Overflow Common attack mechanism first wide use by the Morris Worm in 1988 Prevention techniques known NX bit, stack canaries, ASLR Still of major concern Recent examples:

354 views • 21 slides
Buffer Trees Lars Arge. The Buffer Tree: A New Technique for Optimal I/O Algorithms . In

Buffer Trees Lars Arge. The Buffer Tree: A New Technique for Optimal I/O Algorithms . In

Buffer Trees Lars Arge. The Buffer Tree: A New Technique for Optimal I/O Algorithms . In Proceedings of Fourth Workshop on Algorithms and Data Structures (WADS), Lecture Notes in Computer Science Vol. 955, Springer-Verlag, 1995, 334-345. 1

518 views • 27 slides
AKC Education Webinar Series: Social Media Resources Buffer: What Is It? Buffer is a software

AKC Education Webinar Series: Social Media Resources Buffer: What Is It? Buffer is a software

AKC Education Webinar Series: Social Media Resources Buffer: What Is It? Buffer is a software application for the web and mobile, designed to manage accounts in social networks, by providing the means for a user to schedule posts to Twitter,

506 views • 35 slides
More Vulnerabilities (buffer overreads, format string, integer overflow, heap overflows) Chester

More Vulnerabilities (buffer overreads, format string, integer overflow, heap overflows) Chester

More Vulnerabilities (buffer overreads, format string, integer overflow, heap overflows) Chester Rebeiro Indian Institute of Technology Madras Buffer Overreads 2 Buffer Overread Example 3 Buffer Overread Example len read from command line

906 views • 76 slides
Massive Data Algorithmics Lecture 5: External Search Trees Massive Data Algorithmics Lecture 5:

Massive Data Algorithmics Lecture 5: External Search Trees Massive Data Algorithmics Lecture 5:

Construction Buffer tree Priority Queue Summary Massive Data Algorithmics Lecture 5: External Search Trees Massive Data Algorithmics Lecture 5: External Search Trees Construction Buffer tree Priority Queue Summary B-tree Construction In

479 views • 14 slides
Buffer Software Security overflows and other memory safety vulnerabilities Buffer overflow

Buffer Software Security overflows and other memory safety vulnerabilities Buffer overflow

Last time We continued By launching our 1st section: Buffer Software Security overflows and other memory safety vulnerabilities Buffer overflow fundamentals This time We will finish up By looking at Buffer Overflow overflows

1.33k views • 103 slides
k + -buffer: Fragment Synchronized k-buffer I3D 2014 Andreas A. Vasilakis &amp; Ioannis Fudos {

k + -buffer: Fragment Synchronized k-buffer I3D 2014 Andreas A. Vasilakis &amp; Ioannis Fudos {

k + -buffer: Fragment Synchronized k-buffer I3D 2014 Andreas A. Vasilakis &amp; Ioannis Fudos { abasilak,fudos } @cs.uoi.gr Department of Computer Science &amp; Engineering, University of Ioannina, Greece 16 March 2014 Introduction

782 views • 44 slides
Database Systems IIB: DBMS-Implementation Chapter 5: The Buffer Cache Prof. Dr. Stefan Brass

Database Systems IIB: DBMS-Implementation Chapter 5: The Buffer Cache Prof. Dr. Stefan Brass

Storage Hierarchy, Buffer Manager Disk/Buffer Performance in Oracle Database Systems IIB: DBMS-Implementation Chapter 5: The Buffer Cache Prof. Dr. Stefan Brass Martin-Luther-Universit at Halle-Wittenberg Wintersemester 2019/20

623 views • 50 slides
Delta Pointers: Buffer Overflow Checks Without the Checks Tadde us Kroes &amp; Koen Koning

Delta Pointers: Buffer Overflow Checks Without the Checks Tadde us Kroes &amp; Koen Koning

Delta Pointers: Buffer Overflow Checks Without the Checks Tadde us Kroes &amp; Koen Koning Erik van der Kouwe Herbert Bos Cristiano Giuffrida June 19, 2018 Preview buffer[10] secret 2 Preview buffer[10] secret buffer[5] 2 Preview

934 views • 74 slides
IT 4500 : Information Security Secure Programming Dr Joe Francom Buffer Overflow What it is? A

IT 4500 : Information Security Secure Programming Dr Joe Francom Buffer Overflow What it is? A

IT 4500 : Information Security Secure Programming Dr Joe Francom Buffer Overflow What it is? A buffer overflow occurs when a program or process tries to store more data in a buffer (temporary data storage area) than it was intended to hold.

221 views • 3 slides
Smashing the Buffer Smashing the Buffer Miroslav tampar Miroslav tampar (mstampar@zsis.hr )

Smashing the Buffer Smashing the Buffer Miroslav tampar Miroslav tampar (mstampar@zsis.hr )

Smashing the Buffer Smashing the Buffer Miroslav tampar Miroslav tampar (mstampar@zsis.hr ) (mstampar@zsis.hr ) Summary BSidesVienna 2014, Vienna (Austria) November 22nd, 2014 2 Buffer overflow (a.k.a.)

462 views • 35 slides
Buffer Overflow CS461/ECE422 Spring 2012 Reading Material

Buffer Overflow CS461/ECE422 Spring 2012 Reading Material

Buffer Overflow CS461/ECE422 Spring 2012 Reading Material Based on Chapter 11 of the text Outline Buffer Overflow Stack Buffer Overflow Shell

783 views • 28 slides
Shared buffer laboratory 2 implements a shared buffer Process loop Ke yboard wait for

Shared buffer laboratory 2 implements a shared buffer Process loop Ke yboard wait for

slide 1 gaius Shared buffer laboratory 2 implements a shared buffer Process loop Ke yboard wait for keyboard int port get character put character into buffer end Read (char *ch) shared by the application process Read (char *ch); shared

398 views • 28 slides
CS241 Computer Organization Spring 2015 Buffer Overflow 4-022015 Outline Linking

CS241 Computer Organization Spring 2015 Buffer Overflow 4-022015 Outline Linking

CS241 Computer Organization Spring 2015 Buffer Overflow 4-022015 Outline Linking &amp; Loading, continued Buffer Overflow Read: CSAPP2: section 3.12: out-of-bounds memory references &amp; buffer overflow K&amp;R:

775 views • 43 slides

More recommend