reex amining direct cache access to optimize i o
play

Reex amining Direct Cache Access to Optimize I/O Intensive - PowerPoint PPT Presentation

Nov 28, 2023 •371 likes •830 views

Reex amining Direct Cache Access to Optimize I/O Intensive Applications for Multi- hundred- gigabit Netw orks Alireza Farshin * , Amir Roozbeh *+ , Gerald Q. Maguire Jr. * , Dejan Kosti * KTH Royal Institute of Technology, School of Electrical

  1. Reex amining Direct Cache Access to Optimize I/O Intensive Applications for Multi- hundred- gigabit Netw orks Alireza Farshin * , Amir Roozbeh *+ , Gerald Q. Maguire Jr. * , Dejan Kostić * KTH Royal Institute of Technology, School of Electrical Engineering and Computer Science (EECS) + Ericsson Research

  2. Traditional I/O 1. I/O device DMAs* packets to main memory 2. CPU later fetches them to cache I/O Device * Direct Memory Access (DMA) 2020-07-02 2

  3. Traditional I/O 1. I/O device DMAs* packets to main memory 2. CPU later fetches them to cache Inefficient: Large number of accesses to main • memory High access latency (>60ns) • Unnecessary memory bandwidth usage • I/O Device * Direct Memory Access (DMA) 2020-07-02 3

  4. Direct Cache Access (DCA) 1. I/O device DMAs packets to main memory 2. DCA exploits TPH* to prefetch a portion of packets into cache 3. CPU later fetches them from cache Prefetch I/O Device * PCIe Transaction protocol Processing Hint (TPH) 2020-07-02 4

  5. Direct Cache Access (DCA) 1. I/O device DMAs packets to main memory 2. DCA exploits TPH* to prefetch a portion of packets into cache 3. CPU later fetches them from cache Prefetch Still inefficient in terms of memory bandwidth usage • Requires OS intervention and support from processor • I/O Device * PCIe Transaction protocol Processing Hint (TPH) 2020-07-02 5

  6. Intel Data Direct I/O (DDIO) DDIO in Xeon processors since • Xeon E5 DMA packets or descriptors • directly to/from Last Level Cache (LLC) I/O Device 2020-07-02 6

  7. Trends More in-network computing + offloading capabilities Push costly calculations into the network and perform state teful functions at the processor, which makes applications more I/O intensive. 2020-07-02 7

  8. Pressure from these trends Every 6.72 ns a new (64-B+20-B*) packet arrives at 100 Gbps More in-network computing + offloading capabilities Faster link speeds Multi-hundred-gigabit networks cannot tolerate memory access and interarrival time of packets continues to shrin ink * 7B preamble + 1B start-of-frame delimiter +12B inter-frame gap = 20B 2020-07-02 8

  9. DCA matters because Without DCA we are unable to process I/O at line rate, thus increasing packet loss or latency when utilizing multi-hundred-gigabit networks. 2020-07-02 9

  10. Forw arding Packets at 100 Gbps 100 G 100 Gbps Device under Test Packet Forwarding Packets Generator 1400 99 th Percentile Latency (µs) 1200 1000 800 Intel Xeon Gold 6140 600 400 Mellanox ConnectX-5 200 0 Each NIC is placed in 100 Gbps 200 Gbps a PCIe 3.0 16x slot* 100 Rate Gbps * A PCIe 3.0 16x slot is capable of providing ~125 Gbps effective full-duplex bandwidth. 2020-07-02 10

  11. What happens at 200 Gbps? When forwarding 2x100 G 00 Gbps at 200 Gbps, 30% higher latency for the NIC forwarding at 100 Gbps Device under Test Packet Forwarding Packets Generator 1400 99 th Percentile Latency (µs) 1200 1000 30% 800 Intel Xeon Gold 6140 600 400 Mellanox ConnectX-5 200 0 Each NIC is placed in 100 Gbps 200 Gbps a PCIe 3.0 16x slot* 100 100 Latency of the first NIC, when Gbps Gbps forwarding at indicated aggregate rate * A PCIe 3.0 16x slot is capable of providing ~125 Gbps effective full-duplex bandwidth. 2020-07-02 11

  12. How does DDIO w ork? CPU Socket Writing packets/descriptors: C C C C DDIO overwrites a cache line if if it is already C C C C present in any ny LLC ways ( ≡ write update or hit) C C C C Logical Write to the Same cache line LLC Sending/Receiving Already Packets via DDIO Present In LLC 2020-07-02 12

  13. How does DDIO w ork? CPU Socket Writing packets/descriptors: C C C C DDIO overwrites a cache line if if it is already C C C C present in any ny LLC ways ( ≡ write update or hit) C C C C Otherwise, DDIO allocates a cache line in a limited portion of LLC ( ≡ write allocate or miss) Logical Allocate a cache LLC line Sending/Receiving Not Packets via DDIO Present In LLC 2020-07-02 13

  14. How does DDIO w ork? CPU Socket Writing packets/descriptors: C C C C DDIO overwrites a cache line if if it is already C C C C present in any ny LLC ways ( ≡ write update or hit) C C C C Otherwise, DDIO allocates a cache line in a limited portion of LLC ( ≡ write allocate or miss) Logical Reading packets/descriptors: LLC NIC reads a cache line if it is already present in any LLC ways ( ≡ read hit) Sending/Receiving Packets via DDIO Otherwise, NIC reads it from main memory ( ≡ read miss) 2020-07-02 14

  15. How does DDIO w ork? CPU Socket Designed a set of micro-benchmarks to learn C C C C about DDIO: C C C C • Which ways are used for allocation? C C C C • How does DDIO interact with other applications? Logical • Does DMA via a remote CPU socket LLC pollute LLC? Sending/Receiving Packets via DDIO 2020-07-02 15

  16. LLC w ays used by DDIO I/O Application C0 Logical LLC 1 2 3 4 5 6 7 8 9 10 11 Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology 2020-07-02 16

  17. LLC w ays used by DDIO I/O Cache-sensitive Application Application + C0 C1 Logical LLC 1 2 3 4 5 6 7 8 9 10 11 Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 17

  18. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 18

  19. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 19

  20. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 20

  21. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 21

  22. LLC w ays used by DDIO 10 I/O Cache-sensitive Contention with code/data causes a rise in Application Application + Sum of Cache Misses (Million) the cache misses of the I/O application 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 22

  23. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 23

  24. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 24

  25. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 25

  26. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 26

Recommend

Cache Performance Associativity Replacement Samira Khan Cache Performance March 28,

Cache Performance Associativity Replacement Samira Khan Cache Performance March 28,

3/28/17 Agenda Review from last lecture Cache access Cache Performance Associativity Replacement Samira Khan Cache Performance March 28, 2017 Direct-Mapped Cache: Placement and Access Cache Abstraction and Metrics 00 | 000

518 views • 16 slides
L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache, and all blocks initially marked as not valid , Given the main memory word addresses 0 1 2 3 4 3 4 15, calculate Cache hit rate. Cache Index

654 views • 9 slides
Direct Map Cache and Set Associative Cache (Revision) Lecture 14 CDA 3103 07-07-2014 Example 1

Direct Map Cache and Set Associative Cache (Revision) Lecture 14 CDA 3103 07-07-2014 Example 1

Direct Map Cache and Set Associative Cache (Revision) Lecture 14 CDA 3103 07-07-2014 Example 1 A set-associative cache consists of 64 lines, or slots, divided into four-line sets. Main memory contains 4K blocks of 128 words each. Show the

357 views • 19 slides
Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and

Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and

Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers Norm Jouppi In Context At the time, CPU performance was really beginning to pull away from DRAM performance Increased

329 views • 22 slides
Improving Cache Performance AMAT: Average Memory Access Time AMAT = T hit + Miss Rate x Miss

Improving Cache Performance AMAT: Average Memory Access Time AMAT = T hit + Miss Rate x Miss

Improving Cache Performance AMAT: Average Memory Access Time AMAT = T hit + Miss Rate x Miss Penalty Small Hit Time: On the critical (common case) path Requires small, direct-mapped cache Small size and lack of associativity implies

286 views • 12 slides
1 Blocking Example Reducing Conflict Misses by Blocking /* After */ for (jj = 0; jj < N; jj

1 Blocking Example Reducing Conflict Misses by Blocking /* After */ for (jj = 0; jj < N; jj

Reducing Misses by Compiler Optimizing Memory Layout or Access Pattern McFarling [1989] reduced caches misses by 75% Lecture 15: Application-level Cache on 8KB direct mapped cache, 4 byte blocks in software Instructions Optimizations

191 views • 3 slides
1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

Today Cache memory organization and operation Performance impact of caches Cache Memories The memory mountain Rearranging loops to improve spatial locality Using blocking to improve temporal locality CSci 2021: Machine

102 views • 8 slides
Cache 10/27/16 The Memory Hierarchy Smaller On 1 cycle to access Chip Faster Registers CPU

Cache 10/27/16 The Memory Hierarchy Smaller On 1 cycle to access Chip Faster Registers CPU

Cache 10/27/16 The Memory Hierarchy Smaller On 1 cycle to access Chip Faster Registers CPU Storage instrs Costlier can per byte directly Cache(s) ~10s of cycles to access access (SRAM) Main memory ~100 cycles to access

1.07k views • 64 slides
1 Classifying cache misses Cache Organization Classifying misses by causes (3Cs) Cache size,

1 Classifying cache misses Cache Organization Classifying misses by causes (3Cs) Cache size,

Cache Performance Metrics Cache miss rate: number of cache misses divided by number of accesses Lecture 14: Hardware Approaches for Cache Optimizations Cache hit time: the time between sending address and data returning from cache Cache

608 views • 4 slides
Access Control Access Control 1 Access Control Access control : ensures that all direct

Access Control Access Control 1 Access Control Access control : ensures that all direct

Access Control Access Control 1 Access Control Access control : ensures that all direct accesses to object are authorized a scheme for mapping users to allowed actions Protection objects : system resources for which protection is

576 views • 21 slides
1 Direct-mapped cache Tags and validation stickers Address (showing bit positions) 31 30 13 12

1 Direct-mapped cache Tags and validation stickers Address (showing bit positions) 31 30 13 12

Basic structure of memory hierarchy Exploiting memory hierarchy Speed Size Cost ($/bits) Technology Cache basics CPU < ns 32 high CPU registers 0.5-5 ns KB-MB $4K-$10K SRAM Cache per GB 50-70 ns MB-GB $100-$200 DRAM Main

190 views • 8 slides
Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency

Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency

Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency Requirements of performance, availability, and disconnected operation require us to relax the goal of semantic transparency. - HTTP 1.1 specification

598 views • 13 slides
Cache Refill/Access Decoupling for Vector Machines Christopher Batten, Ronny Krashinsky, Steve

Cache Refill/Access Decoupling for Vector Machines Christopher Batten, Ronny Krashinsky, Steve

Cache Refill/Access Decoupling for Vector Machines , Christopher Batten, Ronny Krashinsky, Steven Gerding, and Krste Asanovic, 3 7th International Symposium on Microarchitecture, Portland, Oregon, December 2004 Cache Refill/Access Decoupling for

1.33k views • 77 slides
Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL)

Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL)

Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL) WORKING SET CPU Cache Registers Cache I lev. Cache II lev. Cache III lev. Memory Disk Tape The cache is a memory with an access time some

572 views • 55 slides
Is Is There a Ge Genetic Ba Basi sis t s to R Race? Exam amining human an var ariation

Is Is There a Ge Genetic Ba Basi sis t s to R Race? Exam amining human an var ariation

Is Is There a Ge Genetic Ba Basi sis t s to R Race? Exam amining human an var ariation an and an ancestry usi using ng mt mtDNA CCSF Mission Campus February 10, 2018 No Note t to t teache achers: : the following slides are

583 views • 57 slides
Direct Access Electricity Administration/Finance Issues Cte May 16, 2013 Approved Direct Access

Direct Access Electricity Administration/Finance Issues Cte May 16, 2013 Approved Direct Access

Direct Access Electricity Administration/Finance Issues Cte May 16, 2013 Approved Direct Access (DA) Service Providers Contracts in place for two 1. major DA Service Providers They will both offer pricing, 2. with the award going to the

186 views • 14 slides
Cache Performance Samira Khan March 28, 2017 Agenda Review from last lecture Cache

Cache Performance Samira Khan March 28, 2017 Agenda Review from last lecture Cache

Cache Performance Samira Khan March 28, 2017 Agenda Review from last lecture Cache access Associativity Replacement Cache Performance Cache Abstraction and Metrics Address Tag Store Data Store (stores (is the address in

904 views • 61 slides
MINUTE OPTIMIZE YOUR PH MONITORING OPTIMIZE WITH HAVING CHALLENGES MEASURING

MINUTE OPTIMIZE YOUR PH MONITORING OPTIMIZE WITH HAVING CHALLENGES MEASURING

MINUTE OPTIMIZE YOUR PH MONITORING OPTIMIZE WITH HAVING CHALLENGES MEASURING PH IN HARSH APPLICATIONS? OPTIMIZE WITH WHAT IS THE IMPACT OF CURRENT TECHNOLOGY: Reference electrode failures Short

454 views • 8 slides
Exam amining M MDD a and M MHD HD as Syntac actic C Complexity M y Meas asures with I

Exam amining M MDD a and M MHD HD as Syntac actic C Complexity M y Meas asures with I

Exam amining M MDD a and M MHD HD as Syntac actic C Complexity M y Meas asures with I Interm rmediate J Japan anese L Learn arner r Corpus Da Data Saeko Komori (Chubu University , Japan) Masatoshi Sugiura (Nagoya University ,

938 views • 34 slides
Cache Performance and Set Associative Cache Lecture 12 CDA 3103 06-30-2014 5.1 Introduction

Cache Performance and Set Associative Cache Lecture 12 CDA 3103 06-30-2014 5.1 Introduction

Cache Performance and Set Associative Cache Lecture 12 CDA 3103 06-30-2014 5.1 Introduction Principle of Locality Programs access a small proportion of their address space at any time Temporal locality Items accessed recently

1.03k views • 59 slides
Cache Refill/Access Decoupling for Vector Machines Christopher Batten, Ronny Krashinsky, Steve

Cache Refill/Access Decoupling for Vector Machines Christopher Batten, Ronny Krashinsky, Steve

Cache Refill/Access Decoupling for Vector Machines Christopher Batten, Ronny Krashinsky, Steve Gerding, Krste Asanovi Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology December 8, 2004 Cache

884 views • 76 slides
NJ Direct 10/Aetna Freedom 10 v. Horizon Direct Access 10 Brown & Brown Benefit Advisors 1

NJ Direct 10/Aetna Freedom 10 v. Horizon Direct Access 10 Brown & Brown Benefit Advisors 1

NJ Direct 10/Aetna Freedom 10 v. Horizon Direct Access 10 Brown & Brown Benefit Advisors 1 Montgomery Township Board of Education SEHBP NJ Direct 10/Aetna Freedom 10 vs Horizon Direct Access 10 NJ Direct 10/Aetna Freedom 10 Horizon

472 views • 36 slides
NOW Handout Page 1 1 KB Direct Mapped Cache, 32B blocks Review: Set Associative Cache For a 2

NOW Handout Page 1 1 KB Direct Mapped Cache, 32B blocks Review: Set Associative Cache For a 2

Review: Who Cares About the Memory Hierarchy? Processor Only Thus Far in Course: EECS 252 Graduate Computer CPU cost/performance, ISA, Pipelined Execution Proc 1000 Architecture CPU 60%/yr. CPU-DRAM Gap Moores Law

277 views • 8 slides
MoCA Access: Multi-Gigabit & Beyond Sponsored By Optimize your experience today Enable

MoCA Access: Multi-Gigabit & Beyond Sponsored By Optimize your experience today Enable

MoCA Access: Multi-Gigabit & Beyond Sponsored By Optimize your experience today Enable popups within your browser. Turn on your systems sound to hear the streaming presentation. Questions? Submit them to the presenters at any

576 views • 38 slides

More recommend