ECE232: Hardware Organization and Design Lecture 23: Associative - PowerPoint PPT Presentation

ECE232: Hardware Organization and Design Lecture 23: Associative Caches Adapted from Computer Organization and Design , Patterson & Hennessy, UCB

Overview  Last time: Direct mapped cache  Pretty simple to understand • Every memory block goes in only one place in the cache  Somewhat limiting • May cause a lot of the cache to be unused  Idea! • Why not be more flexible: data can go into more than one place  Associative caches ECE232: Associative Caches 2

Cache addressing Memory How do you know if Cache  To Processor something is in the Block X cache? (Q1) CPU Block X If it is in the cache,  From Processor how to find it? (Q2) Block Y Traditional Memory  • Given an address, provide the data (has address Full/MSBs of Address decoder) Data Associative Memory  Tag • AKA “Content Addressable Memory” • Each line contain the address (or part of it) and the data ECE232: Associative Caches 3

Cache Organization Fully-associative: any memory  location can be stored anywhere in the cache Cache location and memory • address are unrelated Direct-mapped: each memory  location maps onto exactly one MSBs of Address cache entry Some of the memory address Data • LSBs of Address bit are used to index the cache Tag N-way set-associative: each  memory location can go into one of N sets ECE232: Associative Caches 4

Direct mapped cache (assume 1 byte/Block) 4-Block Direct Cache Block 0 can be  Memory Mapped Cache occupied by data from Memory blocks • 0000 2 0 0 0, 4, 8, 12 1 1 2 2 Cache Block 1 can be 3 3  0100 2 occupied by data from 4 5 Memory blocks • Cache 6 1, 5, 9, 13 Index 7 1000 2 8 Cache Block 2 can be  9 occupied by data from 10 Memory blocks • 11 2, 6, 10, 14 1100 2 12 13 Cache Block 3 can be 14  15 occupied by data from Memory blocks • 3, 7, 11, 15 Block Index ECE232: Associative Caches 5

Fully Associative Cache Block Memory Index 0 1 00 10 00 2 3 4 5 6 01 10 00 Memory block 7 8 address 9 10 offset 10 10 00 tag 11 12 13 11 10 00 14 15 tag 1 word 0110 0010 1110 1010 ECE232: Associative Caches 6

Fully Associative Cache: Block=1 Byte data Main CPU Memory address Tag Data ECE232: Associative Caches 7

Two-way Set Associative Cache Two direct-mapped caches operate in parallel • Cache Index selects a “set” from the cache (set includes 2 • blocks) The two tags in the set are compared in parallel • Data is selected based on the tag result • Cache Index Valid Cache Tag Cache Data Cache Data Cache Tag Valid Cache Block 0 Cache Block 0 : : : : : : Tag Tag Compare Compare 1 0 Mux Sel1 Sel0 OR Set Cache Block Hit ECE232: Associative Caches 8

4-way Set Associative Cache Allow block anywhere in a set  Address 31 30 12 11 10 9 8 3 2 1 0 Advantages:  22 8 Better hit rate • Disadvantage:  Index V Tag Data V Tag Data V Tag Data V Tag Data More tag bits • 0 1 More hardware • 2 Higher access time 253 • 254 255 22 32 4-to-1 multiplexor Hit Data A Four-Way Set-Associative Cache, Block size = 4 bytes ECE232: Associative Caches 9

Set Associative Cache - addressing TAG INDEX/Set # OFFSET Tag to check if have Index to select a Byte offset correct block anywhere set in cache in set ECE232: Associative Caches 10

Associative Caches Fully associative  Allow a given block to go in any cache entry • Requires all entries to be searched at once • Comparator per entry (expensive) • n -way set associative  Each set contains n entries • Block number determines which set • • (Block number) modulo (#Sets in cache) Search all entries in a given set at once • n comparators (less expensive) • ECE232: Associative Caches 11

Spectrum of Associativity For a cache with 8 entries  ECE232: Associative Caches 12

How Much Associativity Increased associativity decreases miss rate  But with diminishing returns • Simulation of a system with 64KB  D-cache, 16-word blocks, SPEC2000 1-way: 10.3% • 2-way: 8.6% • 4-way: 8.3% • 8-way: 8.1% • ECE232: Associative Caches 13

Set Associative Cache Organization ECE232: Associative Caches 14

Types of Cache Misses (for 3 organizations) Compulsory (cold start): location has never been accessed -  first access to a block not in the cache Capacity: since the cache cannot contain all the blocks of a  program, some blocks will be replaced and later retrieved Conflict: when too many blocks try to load into the same set,  some blocks will be replaced and later retrieved ECE232: Associative Caches 15

Cache Design Decisions For a given cache size  • Block (Line) size • Number of Blocks (Lines) • How is the cache organized • Write policy • Replacement Strategy Increase cache size   More Blocks (Lines) • More lines == Higher hit rate • Slower Memory • As many as practical ECE232: Associative Caches 16

Summary  Today: Associative caches  Provide more choices for block storage  More expensive in terms of hardware • Require comparators for tags  Many caches are set associative  Remember: • Direct mapped = 1 way set associative • Full associative = N way set associate (N is total blocks in cache) ECE232: Associative Caches 17