sangam a multi component core cache prefetcher
play

Sangam: A Multi-component Core Cache Prefetcher Mainak Chaudhuri, - PowerPoint PPT Presentation

Dec 14, 2023 •871 likes •1.61k views

Sangam: A Multi-component Core Cache Prefetcher Mainak Chaudhuri, Nayan Deshmukh Introduction The word Sangam refers to a confluence of 3 rivers which corresponds to 3 core components in our prefetcher We achieve 40.3% speedup over

  1. Sangam: A Multi-component Core Cache Prefetcher Mainak Chaudhuri, Nayan Deshmukh

  2. Introduction • The word ‘Sangam’ refers to a confluence of 3 rivers which corresponds to 3 core components in our prefetcher • We achieve 40.3% speedup over no prefetching for 46 single core workloads • For 4 core we achieve 19.5% speedup over no prefetching for 100 multiprogramed workloads (45 homo, 55 hetro)

  3. Sangam IP-Delta-based IP-based stride Adaptive sequence prefetcher degree Next- predictor line prefetcher Recent access filter Encode residual Last PQ Entry? prefetches as metadata Inject prefetch Sequence Complete? Stop

  4. Sangam All the components have a common base degree d All the components have a common base degree d

  5. Where?

  6. Where? • Where to place the prefetcher • L1 allows for better learning whereas L2, L3 allows for more hardware resources

  7. Where? • Where to place the prefetcher • L1 allows for better learning whereas L2, L3 allows for more hardware resources Speedup at different levels of cache 1.34 1.32 1.3 Speedup 1.28 L1 prefetcher 1.26 L2 prefetcher 1.24 1.22 1.2 IP-stride IP-delta

  8. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . . . . . . . . .

  9. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . . . IP . . . . . .

  10. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . . . IP . . . . . . offset

  11. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . . . IP . . . . . . offset

  12. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . . . IP . . . . . . offset -

  13. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . . . IP . . . . . . offset -

  14. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . . . IP . . . . . . offset h -

  15. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . . . IP . . . . . . offset h -

  16. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . Delta . . IP . . . . . . offset h -

  17. IP-Delta-based Sequence predictor • Uses both control-flow and data-flow information to predict a sequence of accesses IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . . . Delta Confidence . . IP . . . . . . offset h -

  18. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . IP . . . . . . offset

  19. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . IP . . . . . . offset

  20. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . IP . . . . . . offset -

  21. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . IP . . . . . . offset -

  22. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . IP . . . . . . offset

  23. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . IP . . . k th . . . offset

  24. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . IP . . . (d+1) th k th . . . offset

  25. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . . IP . . . (d+1) th k th . . . offset

  26. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . h . IP . . . (d+1) th k th . . . offset

  27. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . h . IP . . . (d+1) th k th . . . offset

  28. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset . . . h . IP . . . (d+1-k) th (d+1) th k th . . . offset

  29. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset Delta Delta . . . h . IP . . . (d+1-k) th (d+1) th k th . . . offset

  30. IP-Delta-based Sequence predictor • Learning IP Table IP-Delta Table Last IP Last d+1 deltas h(IP, Delta) Next d deltas offset Delta Delta . . . h . IP . . . (d+1-k) th Confidence (d+1) th k th . . . offset

  31. IP-based stride prefetcher • We use IP based stride predictor when IP-delta predictor can no longer offer predictions • This covers both cases when either the entry is missing from IP-delta table or the sequence is below confidence threshold IP Table Last IP Last d+1 deltas offset . . . IP . . .

  32. IP-based stride prefetcher • We use IP based stride predictor when IP-delta predictor can no longer offer predictions • This covers both cases when either the entry is missing from IP-delta table or the sequence is below confidence threshold IP Table Last IP Last d+1 deltas offset . . We use the IP stride . IP predictor when the last two deltas seen for IP are equal . . .

  33. Next-line prefetcher • Maintaining coverage at the cost of accuracy leads to overall better performance • Used when both IP-delta and IP stride prefetcher cannot offer prediction • Feedback directed degree selection Degree 1 2 ... d 2 1 Hits ... 0 4 4 4 Insertions ... Next-line buffer

  34. Next-line prefetcher • Maintaining coverage at the cost of accuracy leads to overall better performance • Used when both IP-delta and IP stride prefetcher cannot offer prediction • Feedback directed degree selection X+1 1 Degree 1 2 ... d 2 1 Hits ... 0 4 4 4 Insertions ... Next-line buffer

  35. Next-line prefetcher • Maintaining coverage at the cost of accuracy leads to overall better performance • Used when both IP-delta and IP stride prefetcher cannot offer prediction • Feedback directed degree selection X+2 X+1 2 1 Degree 1 2 ... d 2 1 Hits ... 0 4 4 4 Insertions ... Next-line buffer

  36. Next-line prefetcher • Maintaining coverage at the cost of accuracy leads to overall better performance • Used when both IP-delta and IP stride prefetcher cannot offer prediction • Feedback directed degree selection X+d X+2 X+1 d 2 1 Degree 1 2 ... d 2 1 Hits ... 0 4 4 4 Insertions ... Next-line buffer

  37. Next-line prefetcher • Maintaining coverage at the cost of accuracy leads to overall better performance • Used when both IP-delta and IP stride prefetcher cannot offer prediction • Feedback directed degree selection X+d X+2 d 2 Degree 1 2 ... d X+1 2 1 Hits ... 0 1 4 4 4 Insertions ... Next-line buffer

  38. Next-line prefetcher • Maintaining coverage at the cost of accuracy leads to overall better performance • Used when both IP-delta and IP stride prefetcher cannot offer prediction • Feedback directed degree selection X+d X+2 d 2 Degree 1 2 ... d X+1 2 1 Hits ... 0 1 4 4 4 Insertions ... Next-line buffer

  39. Next-line prefetcher • Maintaining coverage at the cost of accuracy leads to overall better performance • Used when both IP-delta and IP stride prefetcher cannot offer prediction • Feedback directed degree selection X+d X+2 d 2 Degree 1 2 ... d X+1 2 1 Hits ... 0 1 5 4 4 Insertions ... Next-line buffer

Recommend

Instruction Prefetcher Ali Ansari (Sharif) Fatemeh Golshan (Sharif) Pejman Lotfi-Kamran (IPM)

Instruction Prefetcher Ali Ansari (Sharif) Fatemeh Golshan (Sharif) Pejman Lotfi-Kamran (IPM)

MANA: Microarchitecting an Instruction Prefetcher Ali Ansari (Sharif) Fatemeh Golshan (Sharif) Pejman Lotfi-Kamran (IPM) Hamid Sarbazi-Azad (Sharif, IPM) Instruction Cache Misses Server applications o Multi-megabyte instruction footprint o

822 views • 18 slides
Increasing the Efficiency of an Embedded Multi-Core Bytecode Processor Using an Object Cache

Increasing the Efficiency of an Embedded Multi-Core Bytecode Processor Using an Object Cache

Faculty of Computer Science, Institute for Computer Engineering, Chair for VLSI-Design, Diagnostic und Architecture Increasing the Efficiency of an Embedded Multi-Core Bytecode Processor Using an Object Cache Processor Using an Object Cache

641 views • 28 slides
Real-Time Cache Management for Multi-Core Virtualization Hyoseung Kim 1,2 Raj Rajkumar 2 1

Real-Time Cache Management for Multi-Core Virtualization Hyoseung Kim 1,2 Raj Rajkumar 2 1

EMSOFT 2016 Real-Time Cache Management for Multi-Core Virtualization Hyoseung Kim 1,2 Raj Rajkumar 2 1 University of Riverside, California 2 Carnegie Mellon University EMSOFT 2016 Benefits of Multi-Core Processors Consolidation of real-time

561 views • 24 slides
Modeling Cache Sharing for MPI Programs on Multi-core Machines Bin Bao, Chen Ding University of

Modeling Cache Sharing for MPI Programs on Multi-core Machines Bin Bao, Chen Ding University of

Modeling Cache Sharing for MPI Programs on Multi-core Machines Bin Bao, Chen Ding University of Rochester Nov 10, 2011 The 10th Workshop On

346 views • 20 slides
A Coordinated Approach for Practical OS-Level Cache Management in Multi-Core Real-Time Systems

A Coordinated Approach for Practical OS-Level Cache Management in Multi-Core Real-Time Systems

ECRTS 2013 A Coordinated Approach for Practical OS-Level Cache Management in Multi-Core Real-Time Systems Hyoseung Kim Arvind Kandhalu Prof. Raj Rajkumar Electrical and Computer Engineering Carnegie Mellon University ECRTS 2013 Why

264 views • 24 slides
Cache Impact on Program Performance T. Yang. UCSB CS240A. 2017 Multi-level cache in computer

Cache Impact on Program Performance T. Yang. UCSB CS240A. 2017 Multi-level cache in computer

Cache Impact on Program Performance T. Yang. UCSB CS240A. 2017 Multi-level cache in computer systems Topics Performance analysis for multi-level cache Cache performance optimization through program transformation Processor Caching

530 views • 37 slides
1 Classifying cache misses Cache Organization Classifying misses by causes (3Cs) Cache size,

1 Classifying cache misses Cache Organization Classifying misses by causes (3Cs) Cache size,

Cache Performance Metrics Cache miss rate: number of cache misses divided by number of accesses Lecture 14: Hardware Approaches for Cache Optimizations Cache hit time: the time between sending address and data returning from cache Cache

608 views • 4 slides
Decentralized Dynamic Scheduling across Heterogeneous Multi core across Heterogeneous Multi

Decentralized Dynamic Scheduling across Heterogeneous Multi core across Heterogeneous Multi

Decentralized Dynamic Scheduling across Heterogeneous Multi core across Heterogeneous Multi core Desktop Grids J Jaehwan Lee , Pete Keleher, Alan Sussman h L P t K l h Al S Department of Computer Science University of Maryland Multi

357 views • 25 slides
Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency

Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency

Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency Requirements of performance, availability, and disconnected operation require us to relax the goal of semantic transparency. - HTTP 1.1 specification

599 views • 13 slides
Common-Reflection-Surface stack for OBS and Introduction VSP geometries and multi-component

Common-Reflection-Surface stack for OBS and Introduction VSP geometries and multi-component

9 th SBGf Conference, Salvador 2005 Boelsen & Mann Common-Reflection-Surface stack for OBS and Introduction VSP geometries and multi-component seismic Traveltime formulas General case reflection data OBS and VSP Multi-component data

1.57k views • 89 slides
Efficient Wake-Up Scheduling for Efficient Wake-Up Scheduling for Multi-Core Systems Multi-Core

Efficient Wake-Up Scheduling for Efficient Wake-Up Scheduling for Multi-Core Systems Multi-Core

TH EDA Efficient Wake-Up Scheduling for Efficient Wake-Up Scheduling for Multi-Core Systems Multi-Core Systems Ming-Chao Lee*, Yiyu Shi**, Yu-Guan Chen*, Diana Marculescu***, Shih-Chieh Chang* * Dept. of CS, National Tsing Hua

338 views • 21 slides
Hardware Execution Throttling for Multicore Resource Management Xiao Zhang Sandhya Dwarkadas

Hardware Execution Throttling for Multicore Resource Management Xiao Zhang Sandhya Dwarkadas

Hardware Execution Throttling for Multicore Resource Management Xiao Zhang Sandhya Dwarkadas Kai Shen 1 The Multi-Core Challenge Multi-core chip Dominant on market Last level on-chip cache is commonly shared by sibling cores,

286 views • 13 slides
Multi-core model checking for biological applications Jaco van de Pol 22 November 2013

Multi-core model checking for biological applications Jaco van de Pol 22 November 2013

UNIVERSITY OF TWENTE. Formal Methods & Tools. Multi-core model checking for biological applications Jaco van de Pol 22 November 2013 NWPT13, Tallinn ... Multi-core Reachability Multi-core LTL model checking Timed Automata

1.24k views • 119 slides
From CPU-GPU to heterogeneous multi-core Yesterday (2000-2010) Homogeneous multi-core Discrete

From CPU-GPU to heterogeneous multi-core Yesterday (2000-2010) Homogeneous multi-core Discrete

Simty : Generalized SIMT execution on RISC-V CARRV 2017 Sylvain Collange INRIA Rennes / IRISA sylvain.collange@inria.fr From CPU-GPU to heterogeneous multi-core Yesterday (2000-2010) Homogeneous multi-core Discrete components Graphics

254 views • 22 slides
Real-Time Multi/Many-Core Architecture Heechul Yun 1 Real-Time Multi/Many-Core Architecture

Real-Time Multi/Many-Core Architecture Heechul Yun 1 Real-Time Multi/Many-Core Architecture

Real-Time Multi/Many-Core Architecture Heechul Yun 1 Real-Time Multi/Many-Core Architecture Projects on Real-Time CPU Architectures Assigned Papers Shedding the Shackles of Time-Division Multiplexing, RTSS, 2018 Deterministic

749 views • 31 slides
Spinor dynamics in a multi- component Fermi gas Ulrich Ebling, Andr Eckardt and Maciej

Spinor dynamics in a multi- component Fermi gas Ulrich Ebling, Andr Eckardt and Maciej

Spinor dynamics in a multi- component Fermi gas Ulrich Ebling, Andr Eckardt and Maciej Lewenstein Quantum Technologies conference Warsaw, 10.09.2012 Spinor dynamics in a multi- component Fermi gas Outline Description by density matrix /

393 views • 15 slides
SEAL Component Model ROOT/CORE Meeting 8 April 2005 P. Mato / CERN Component Model Overview

SEAL Component Model ROOT/CORE Meeting 8 April 2005 P. Mato / CERN Component Model Overview

SEAL Component Model ROOT/CORE Meeting 8 April 2005 P. Mato / CERN Component Model Overview A way of resolving software complexity A generic set of software building blocks components , services Exposing well defined interface(s)

422 views • 9 slides
Memory Access Latency Joshua San Miguel Natalie Enright Jerger Chip Multiprocessor main memory

Memory Access Latency Joshua San Miguel Natalie Enright Jerger Chip Multiprocessor main memory

Load Value Approximation: Approaching the Ideal Memory Access Latency Joshua San Miguel Natalie Enright Jerger Chip Multiprocessor main memory shared caches, network-on-chip private cache miss private cache private cache core core core

482 views • 18 slides
Doppelgnger: A Cache for Approximate Computing Joshua San Miguel Jorge Albericio Andreas

Doppelgnger: A Cache for Approximate Computing Joshua San Miguel Jorge Albericio Andreas

Doppelgnger: A Cache for Approximate Computing Joshua San Miguel Jorge Albericio Andreas Moshovos Natalie Enright Jerger Cache Hierarchy main memory shared last-level cache private caches processor core 2 Cache Hierarchy main memory

1.34k views • 79 slides
Complexity-Effec/ve Mul/core Cache Coherence MCC2012 Stefanos

Complexity-Effec/ve Mul/core Cache Coherence MCC2012 Stefanos

Complexity-Effec/ve Mul/core Cache-Coherence Complexity-Effec/ve Mul/core Cache Coherence MCC2012 Stefanos Kaxiras & Alberto Ros 1. Ros &

862 views • 39 slides
L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache, and all blocks initially marked as not valid , Given the main memory word addresses 0 1 2 3 4 3 4 15, calculate Cache hit rate. Cache Index

654 views • 9 slides
CENG3420 Lecture 13: Multi-Threading & Multi-Core Bei Yu byu@cse.cuhk.edu.hk (Latest

CENG3420 Lecture 13: Multi-Threading & Multi-Core Bei Yu byu@cse.cuhk.edu.hk (Latest

CENG3420 Lecture 13: Multi-Threading & Multi-Core Bei Yu byu@cse.cuhk.edu.hk (Latest update: March 14, 2018) Spring 2018 1 / 38 Overview Introduction Amdahls Law Thread-Level Parallelism (TLP) Multi-Cores 2 / 38 Overview

737 views • 46 slides
A Multi-Layered Image Cache A Multi-Layered Image Cache for Scientific Visualization for

A Multi-Layered Image Cache A Multi-Layered Image Cache for Scientific Visualization for

A Multi-Layered Image Cache A Multi-Layered Image Cache for Scientific Visualization for Scientific Visualization Eric Lamar and Valerio Pascucci Eric Lamar and Valerio Pascucci Center for Applied Scientific Computing Center for Applied

511 views • 24 slides
A review of dimensionality reduction in high-dimensional data using multi-core and many-core

A review of dimensionality reduction in high-dimensional data using multi-core and many-core

Second Workshop on Software Challenges to Exascale Computing (13 th , 14 th December 2018, New Delhi) A presentation on A review of dimensionality reduction in high-dimensional data using multi-core and many-core architecture by Mr.

604 views • 39 slides

More recommend