cache memory
play

Cache Memory Chapter 17 S. Dandamudi Outline Introduction - PowerPoint PPT Presentation

Mar 18, 2023 •236 likes •802 views

Cache Memory Chapter 17 S. Dandamudi Outline Introduction Types of cache misses How cache memory works Types of caches Why cache memory works Example implementations Pentium Cache design basics PowerPC

  1. Cache Memory Chapter 17 S. Dandamudi

  2. Outline • Introduction • Types of cache misses • How cache memory works • Types of caches • Why cache memory works • Example implementations ∗ Pentium • Cache design basics ∗ PowerPC • Mapping function ∗ MIPS ∗ Direct mapping • Cache operation summary ∗ Associative mapping • Design issues ∗ Set-associative mapping ∗ Cache capacity • Replacement policies ∗ Cache line size • Write policies ∗ Degree of associatively • Space overhead 2003  S. Dandamudi Chapter 17: Page 2 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  3. Introduction • Memory hierarchy ∗ Registers ∗ Memory ∗ Disk ∗ … • Cache memory is a small amount of fast memory ∗ Placed between two levels of memory hierarchy » To bridge the gap in access times – Between processor and main memory (our focus) – Between main memory and disk (disk cache) ∗ Expected to behave like a large amount of fast memory 2003  S. Dandamudi Chapter 17: Page 3 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  4. Introduction (cont’d) 2003  S. Dandamudi Chapter 17: Page 4 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  5. How Cache Memory Works • Prefetch data into cache before the processor needs it ∗ Need to predict processor future access requirements » Not difficult owing to locality of reference • Important terms ∗ Miss penalty ∗ Hit ratio ∗ Miss ratio = (1 – hit ratio) ∗ Hit time 2003  S. Dandamudi Chapter 17: Page 5 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  6. How Cache Memory Works (cont’d) Cache read operation 2003  S. Dandamudi Chapter 17: Page 6 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  7. How Cache Memory Works (cont’d) Cache write operation 2003  S. Dandamudi Chapter 17: Page 7 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  8. Why Cache Memory Works • Example for (i=0; i<M; i++) for(j=0; j<N; j++) X[i][j] = X[i][j] + K; ∗ Each element of X is double (eight bytes) ∗ Loop is executed (M * N) times » Placing the code in cache avoids access to main memory – Repetitive use (one of the factors) – Temporal locality » Prefetching data – Spatial locality 2003  S. Dandamudi Chapter 17: Page 8 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  9. How Cache Memory Works (cont’d) 300 250 Execution time (ms Column-order 200 150 100 Row-order 50 0 500 600 700 800 900 1000 Matrix size 2003  S. Dandamudi Chapter 17: Page 9 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  10. Cache Design Basics • On every read miss ∗ A fixed number of bytes are transferred » More than what the processor needs – Effective due to spatial locality • Cache is divided into blocks of B bytes » b -bits are needed as offset into the block b = log 2 B » Block are called cache lines • Main memory is also divided into blocks of same size ∗ Address is divided into two parts 2003  S. Dandamudi Chapter 17: Page 10 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  11. Cache Design Basics (cont’d) B = 4 bytes b = 2 bits 2003  S. Dandamudi Chapter 17: Page 11 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  12. Cache Design Basics (cont’d) • Transfer between main memory and cache ∗ In units of blocks ∗ Implements spatial locality • Transfer between main memory and cache ∗ In units of words • Need policies for ∗ Block placement ∗ Mapping function ∗ Block replacement ∗ Write policies 2003  S. Dandamudi Chapter 17: Page 12 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  13. Cache Design Basics (cont’d) Read cycle operations 2003  S. Dandamudi Chapter 17: Page 13 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  14. Mapping Function • Determines how memory blocks are mapped to cache lines • Three types ∗ Direct mapping » Specifies a single cache line for each memory block ∗ Set-associative mapping » Specifies a set of cache lines for each memory block ∗ Associative mapping » No restrictions – Any cache line can be used for any memory block 2003  S. Dandamudi Chapter 17: Page 14 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  15. Mapping Function (cont’d) Direct mapping example 2003  S. Dandamudi Chapter 17: Page 15 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  16. Mapping Function (cont’d) • Implementing direct mapping ∗ Easier than the other two ∗ Maintains three pieces of information » Cache data – Actual data » Cache tag – Problem: More memory blocks than cache lines � Several memory blocks are mapped to a cache line – Tag stores the address of memory block in cache line » Valid bit – Indicates if cache line contains a valid block 2003  S. Dandamudi Chapter 17: Page 16 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  17. Mapping Function (cont’d) 2003  S. Dandamudi Chapter 17: Page 17 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  18. Mapping Function (cont’d) Direct mapping Reference pattern: 0, 4, 0, 8, 0, 8, 0, 4, 0, 4, 0, 4 Hit ratio = 0% 2003  S. Dandamudi Chapter 17: Page 18 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  19. Mapping Function (cont’d) Direct mapping Reference pattern: 0, 7, 9, 10, 0, 7, 9, 10, 0, 7, 9, 10 Hit ratio = 67% 2003  S. Dandamudi Chapter 17: Page 19 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  20. Mapping Function (cont’d) Associative mapping 2003  S. Dandamudi Chapter 17: Page 20 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  21. Mapping Function (cont’d) Associative mapping Reference pattern: 0, 4, 0, 8, 0, 8, 0, 4, 0, 4, 0, 4 Hit ratio = 75% 2003  S. Dandamudi Chapter 17: Page 21 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  22. Mapping Function (cont’d) Address match logic for associative mapping 2003  S. Dandamudi Chapter 17: Page 22 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  23. Mapping Function (cont’d) Associative cache with address match logic 2003  S. Dandamudi Chapter 17: Page 23 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  24. Mapping Function (cont’d) Set-associative mapping 2003  S. Dandamudi Chapter 17: Page 24 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  25. Mapping Function (cont’d) Address partition in set-associative mapping 2003  S. Dandamudi Chapter 17: Page 25 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  26. Mapping Function (cont’d) Set-associative mapping Reference pattern: 0, 4, 0, 8, 0, 8, 0, 4, 0, 4, 0, 4 Hit ratio = 67% 2003  S. Dandamudi Chapter 17: Page 26 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  27. Replacement Policies • We invoke the replacement policy ∗ When there is no place in cache to load the memory block • Depends on the actual placement policy in effect ∗ Direct mapping does not need a special replacement policy » Replace the mapped cache line ∗ Several policies for the other two mapping functions » Popular: LRU (least recently used) » Random replacement » Less interest (FIFO, LFU) 2003  S. Dandamudi Chapter 17: Page 27 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  28. Replacement Policies (cont’d) • LRU ∗ Expensive to implement » Particularly for set sizes more than four • Implementations resort to approximation ∗ Pseudo-LRU » Partitions sets into two groups – Maintains the group that has been accessed recently – Requires only one bit » Requires only ( W -1) bits ( W = degree of associativity) – PowerPC is an example � Details later 2003  S. Dandamudi Chapter 17: Page 28 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  29. Replacement Policies (cont’d) Pseudo-LRU implementation 2003  S. Dandamudi Chapter 17: Page 29 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

Recommend

What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache

What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache

What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache Performance Proc/Regs L1-Cache Bigger Faster L2-Cache Memory hierarchy concept, cache L3-Cache (optional) design fundamentals, set-associative

303 views • 4 slides
Chapter 4 Cache Memory Contents Computer memory system overview Characteristics of

Chapter 4 Cache Memory Contents Computer memory system overview Characteristics of

Chapter 4 Cache Memory Contents Computer memory system overview Characteristics of memory systems Memory hierarchy Cache memory principles Elements of cache design Cache size Mapping function Replacement

793 views • 64 slides
General Cache Mechanics CPU Block: unit of data in cache and memory. (a.k.a. line) Memory

General Cache Mechanics CPU Block: unit of data in cache and memory. (a.k.a. line) Memory

General Cache Mechanics CPU Block: unit of data in cache and memory. (a.k.a. line) Memory Hierarchy: Cache Smaller, faster, more expensive. Cache 8 9 14 3 Stores subset of memory blocks . (lines) Data is moved in block units Memory

467 views • 8 slides
Lecture 23: Cache, Memory, Virtual Memory Todays topics: Cache examples, caching

Lecture 23: Cache, Memory, Virtual Memory Todays topics: Cache examples, caching

Lecture 23: Cache, Memory, Virtual Memory Todays topics: Cache examples, caching policies Main memory system Virtual memory 1 Example 1 32 KB 4-way set-associative data cache array with 32 byte line sizes How many

436 views • 19 slides
Cache Memory Chapter 17 S. Dandamudi Outline Introduction Types of cache misses

Cache Memory Chapter 17 S. Dandamudi Outline Introduction Types of cache misses

Cache Memory Chapter 17 S. Dandamudi Outline Introduction Types of cache misses How cache memory works Types of caches Why cache memory works Example implementations Cache design basics Pentium PowerPC

813 views • 28 slides
Memory Hierarchy: Cache Memory hierarchy Cache basics Locality Cache organization Cache-aware

Memory Hierarchy: Cache Memory hierarchy Cache basics Locality Cache organization Cache-aware

Memory Hierarchy: Cache Memory hierarchy Cache basics Locality Cache organization Cache-aware programming How does execution time grow with SIZE? int[] array = new int[SIZE]; fillArrayRandomly(array); int s = 0; for (int i = 0; i &lt;

847 views • 48 slides
Cache Systems CPU Main Main CPU Memory Memory 400MHz 10MHz Cache 10MHz Memory Hierarchy

Cache Systems CPU Main Main CPU Memory Memory 400MHz 10MHz Cache 10MHz Memory Hierarchy

Cache Systems CPU Main Main CPU Memory Memory 400MHz 10MHz Cache 10MHz Memory Hierarchy Design Bus 66MHz Bus 66MHz Chapter 5 and Appendix C Data object Block transfer transfer Main CPU Cache Memory 1 4 Example: Two-level

353 views • 10 slides
1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

Today Cache memory organization and operation Performance impact of caches Cache Memories The memory mountain Rearranging loops to improve spatial locality Using blocking to improve temporal locality CSci 2021: Machine

102 views • 8 slides
Tutorial 9 : cache memory Why use a cache ? Main memory (VRAM/DRAM) is slow ! To deal with

Tutorial 9 : cache memory Why use a cache ? Main memory (VRAM/DRAM) is slow ! To deal with

Tutorial 9 : cache memory Why use a cache ? Main memory (VRAM/DRAM) is slow ! To deal with this, the -machine speed is reduced to match the memory read and write speed To make the machine faster, one can use a intermediate smaller

219 views • 8 slides
Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory

Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory

Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory system performance is critical in modern architectures DO I = 1, M Accessing memory takes much longer than DO J = 1, N accessing cache

364 views • 19 slides
Virtual Memory 1 Memory Hierarchy Memory 4GB Cache 1M Registers 1K Question: What if

Virtual Memory 1 Memory Hierarchy Memory 4GB Cache 1M Registers 1K Question: What if

Virtual Memory 1 Memory Hierarchy Memory 4GB Cache 1M Registers 1K Question: What if we want to run a process that requires 10GB memory? 2 Memory Hierarchy Virtual Memory Memory Cache Registers Answer: Pretend we had something

737 views • 45 slides
1 Implementation Snoop Caches Implementing Snooping Caches Write Races: Multiple processors

1 Implementation Snoop Caches Implementing Snooping Caches Write Races: Multiple processors

An Example Snoopy Protocol Invalidation protocol, write-back cache Each block of memory is in one state: Cache Coherence and Memory Clean in all caches and up-to-date in memory (Shared) Consistency OR Dirty in exactly one cache

507 views • 3 slides
P age 1 Review: Cache perf ormance What are all the aspects of cache organization that impact

P age 1 Review: Cache perf ormance What are all the aspects of cache organization that impact

Who Cares About the Memory Hierarchy? CS252 Graduate Computer Architecture Proc 1000 Lecture 4 CPU 60%/yr. Moores Law Performance CPU- DRAM Gap Cache Design 100 Processor-Memory Performance Gap: (grows 50% / year) 10

380 views • 9 slides
LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to

LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to

LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to recently-used code and data, main memory acts as a cache for magnetic disk. The mechanism by which this is accomplished is known as virtual memory

974 views • 34 slides
1 Direct-mapped cache Tags and validation stickers Address (showing bit positions) 31 30 13 12

1 Direct-mapped cache Tags and validation stickers Address (showing bit positions) 31 30 13 12

Basic structure of memory hierarchy Exploiting memory hierarchy Speed Size Cost ($/bits) Technology Cache basics CPU &lt; ns 32 high CPU registers 0.5-5 ns KB-MB $4K-$10K SRAM Cache per GB 50-70 ns MB-GB $100-$200 DRAM Main

190 views • 8 slides
Lecture 24: Cache, Memory, Security Todays topics: Caching policies Main memory

Lecture 24: Cache, Memory, Security Todays topics: Caching policies Main memory

Lecture 24: Cache, Memory, Security Todays topics: Caching policies Main memory system Hardware security intro 1 Cache Misses On a write miss, you may either choose to bring the block into the cache (write-allocate) or

146 views • 10 slides
Lecture 23: Cache, Memory, Security Todays topics: Caching policies Main memory

Lecture 23: Cache, Memory, Security Todays topics: Caching policies Main memory

Lecture 23: Cache, Memory, Security Todays topics: Caching policies Main memory system Hardware security intro 1 Cache Misses On a write miss, you may either choose to bring the block into the cache (write-allocate) or

460 views • 11 slides
Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL)

Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL)

Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL) WORKING SET CPU Cache Registers Cache I lev. Cache II lev. Cache III lev. Memory Disk Tape The cache is a memory with an access time some

572 views • 55 slides
Cache Memory Raul Queiroz Feitosa Content Memory Hierarchy Principle of Locality Some

Cache Memory Raul Queiroz Feitosa Content Memory Hierarchy Principle of Locality Some

Cache Memory Raul Queiroz Feitosa Content Memory Hierarchy Principle of Locality Some Definitions Cache Architectures Fully Associative Direct Mapping Set Associative Replacement Policy Main Memory Update Polic

675 views • 40 slides
Lecture 20: Cache Hierarchies, Virtual Memory Todays topics: Cache hierarchies

Lecture 20: Cache Hierarchies, Virtual Memory Todays topics: Cache hierarchies

Lecture 20: Cache Hierarchies, Virtual Memory Todays topics: Cache hierarchies Virtual memory Reminder: Assignment 8 will be posted soon (due Tue 11/21) 1 Example Access Pattern Byte address Assume that addresses are 8

456 views • 17 slides
L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache, and all blocks initially marked as not valid , Given the main memory word addresses 0 1 2 3 4 3 4 15, calculate Cache hit rate. Cache Index

654 views • 9 slides
Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides

Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides

Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides courtesy of Alvin Lebeck) Software DSM 101 Software DSM 101 Software-based distributed shared memory (DSM) provides an illusion of shared memory on

236 views • 23 slides
Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides

Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides

Distributed Memory and Cache Consistency Distributed Memory and Cache Consistency (some slides courtesy of Alvin Lebeck) Software DSM 101 Software DSM 101 Software-based distributed shared memory (DSM) provides an illusion of shared memory on

414 views • 12 slides
Memory hierarchy / Cache Hung-Wei Tseng Memory gap 3 Memory in stored program computer

Memory hierarchy / Cache Hung-Wei Tseng Memory gap 3 Memory in stored program computer

Memory hierarchy / Cache Hung-Wei Tseng Memory gap 3 Memory in stored program computer Processor PC instruction memory 120007a30: 0f00bb27 ldah gp,15(t12) 120007a34: 509cbd23 lda gp,-25520(gp) 120007a38: 00005d24 ldah t1,0(gp)

730 views • 42 slides

More recommend