Presentation is loading The Latency Problem 100 ms Average : 9,3s - PowerPoint PPT Presentation

The Cache Sketch: Revisiting Expiration-based Caching in the Age of Cloud Data Management Felix Gessert, Michael Schaarschmidt, Wolfram Wingerath, Steffen Friedrich, Norbert Ritter gessert@informatik.uni-hamburg.de

Presentation is loading

The Latency Problem 100 ms Average : 9,3s Loading … -1% Revenue

The Latency Problem 400 ms -9% Visitors -1% Revenue

The Latency Problem 500 ms Average : 9,3s Loading … -20% Traffic -9% Visitors -1% Revenue

The Latency Problem 1s Average : 9,3s Loading … -7% Conversions -20% Traffic -9% Visitors -1% Revenue

If perceived speed is such an import factor ...what causes slow page load times?

State of of the art Two bottlenecks: : la latency und processing High Latency Processing Time

Network Latency Th The underlying problem of of hig igh page lo load tim times I. Grigorik, High performance browser networking. O’Reilly Media, 2013.

The lo low-latency vis ision Data is is served by by ubiquitous web-caches Low Latency Less Processing

The web‘s cachin ing model Staleness as as a a consequence of of scalability Expiration-based Revalidations Every object has a defined Allow clients and caches to Time-To-Live (TTL) check freshness at the server

The web‘s cachin ing model Staleness as as a a consequence of of scalability Expiration-based Revalidations Every object has a defined Allow clients and caches to Time-To-Live (TTL) check freshness at the server Stale Da Data

The web‘s cachin ing model Staleness as as a a consequence of of scalability Expiration-based Revalidations Every object has a defined Allow clients and caches to Time-To-Live (TTL) check freshness at the server Stale Da Data Research Question : Can database services leverage the web caching infrastructure for low latency with rich consistency guarantees?

Web Cachin ing Concepts Invalidation- and In and expiration-based caches Client Expiration-based Caches:  An object x is considered Browser Caches, Forward Proxies, fresh for TTL x seconds ISP Caches Expi pira ration on- based Caches  The server assigns TTLs for each object Request Cache Content Delivery Path Hits Invalidation-based Caches: In Networks, Reverse Proxies  Expose object eviction Invalida dation on- based Caches operation to the server Server/DB

The Cache Sketch approach Le Letting th the clie client handle cache coherence ss-Minimization Bloom filter 10101010 Client Needs Revalidation ? Client nt Cache he Sketch Staleness Browser Caches, Forward Proxies, ISP Caches Periodic at Expi pira ration on- at every Δ transaction based Caches connect seconds begin Request Cache 1 2 3 Content Delivery Path Hits Networks, Reverse Proxies Invalida dation on- Invalidation-Minimization based Caches 10101010 10201040 Invalidations, Records Non-expired Counting Report Expirations Bloom Filter Record Keys and Writes Server/DB 4 Serv rver r Cache he Sketch Needs Invalidation ?

The End to to End Path of of Request Th The Caching Hie ierarchy DB.posts.get(id) JavaScript Miss Miss Miss Client- Miss Miss Proxy (Browser-) ISP CDN Caches Cache Reverse- Caches Caches Proxy Cache Orestes Hit Updated by the Updated by server Cache Sketch

The End to to End Path of of Request Th The Caching Hie ierarchy GET /db/posts/{id} HTTP Miss Miss Miss Client- Miss Miss Proxy (Browser-) ISP CDN Caches Cache Reverse- Caches Caches Proxy Cache Orestes Hit Updated by the Updated by server Cache Sketch

The End to to End Path of of Request Th The Caching Hie ierarchy Cache-Hit Ca it: Return Object Cache-Miss or Ca or Revalid idation: Forward Request Miss Miss Miss Client- Miss Miss Proxy (Browser-) ISP CDN Caches Cache Reverse- Caches Caches Proxy Cache Orestes Hit Updated by the Updated by server Cache Sketch

The End to to End Path of of Request Th The Caching Hie ierarchy Return record from DB with caching TTL Miss Miss Miss Client- Miss Miss Proxy (Browser-) ISP CDN Caches Cache Reverse- Caches Caches Proxy Cache Orestes Hit Updated by the Updated by server Cache Sketch

The End to to End Path of of Request Th The Caching Hie ierarchy Reduced Flash-Crowd Higher Low Latency Database Load Protection Availability Miss Miss Miss Client- Miss Miss Proxy (Browser-) ISP CDN Caches Cache Reverse- Caches Caches Proxy Cache Orestes Hit Updated by the Updated by server Cache Sketch

The Cli lient Cache Sketch  Let c t be the client Cache Sketch generated at time t, containing the key key x of every record x that was written before it expired in all caches, i.e. every x for which holds: ∃ 𝑠(𝑦, 𝑢 𝑠 , 𝑈𝑈𝑀), 𝑥 𝑦, 𝑢 𝑥 ∶ 𝑢 𝑠 + 𝑈𝑈𝑀 > 𝑢 > 𝑢 𝑥 > 𝑢 𝑠 Hit key Cli lient nt Cache he Sketch GET request Miss no h 1 Cache he ... key find(key) 1 0 0 1 1 0 1 1 Bits = 1 h k yes key Revalidation k hash functions m Bloom filter bits

Slo low in initial page lo loads 1  Solution: Cached In Init itialization ◦ Clients load the Cache Sketch at connection ◦ Every non-stale cached record can be reused without degraded consistency purge(obj) Browser CDN Cache hashA(oid) hashB(oid) hashA(oid) hashB(oid) Flat(Counting Bloomfilter) 0 3 2 1 4 0 1 1 1 1 1 0

Slo low in initial page lo loads 1  Solution: Cached In Init itialization ◦ Clients load the Cache Sketch at connection ◦ Every non-stale cached record can be reused without degraded consistency purge(obj) False-Positive Hash- 𝑙 𝑔 ≈ 1 − 𝑓 −𝑙𝑜 𝑜 𝑙 = ln 2 ⋅ ( 𝑛 ) Functions: Rate: 𝑛 Browser CDN Cache With 20.000 distinct updates and 5% error rate: 11 KByte hashA(oid) hashB(oid) hashA(oid) hashB(oid) Flat(Counting Bloomfilter) 0 3 2 1 4 0 1 1 1 1 1 0

Slo low CRUD performance 2  Solution: Δ -Bounded Staleness ◦ Clients refresh the Cache Sketch so its age never exceeds Δ → Consistency guarantee : Δ -atomicity Expiration- Invalidation- Client Server based Caches based Caches Cache Sketch c t -time t Qu Query Cache Sketch fresh records Cache Hits -time t + Δ Revalidate record & Refresh Cache Sketch Fresh record & new Cache Sketch

Hig igh Abort Rates in in OCC 3  Solution: Conflict-Avoidant Optimistic Transactions ◦ Cache Sketch fetched with transaction begin ◦ Cac ached reads → Sh Shorter tr transaction duration → le less ss ab aborts Begin Transaction 1 REST-Server Bloom Filter Cache Writes Reads Cache ( Hidden ) REST-Server Client 2 Writes Cache DB Commit: read- & write-set versions Read all REST-Server 3 Committed OR aborted + stale objects 5 Writes ( Public ) validation 4 prevent conflicting validations Coordinator

Costly In Invalidations 4  Solution: In Invalidation Min inimization ◦ The server Cache Sketch tracks TTLs ◦ Invalidation on only ly necessary, if there are unexpired records 10101010 10201040 Counting Non-expired Report Expirations Bloom Filter Record Keys and Writes Server Cache he Sketch Needs Invalidation ?

End-to to-End Example Client Cache Expiration- Invalidation- Server Cache Server Client Sketch based Cache based Cache Sketch b={x 2 } I NITIALIZE b= c={(x 2 ,t 2 ),(x 3 ,t 3 )} c={(x 1 ,t 1 ) t = {(x 2, t 2 ), C ONNECT (x 3, t 3 ),(x 1, t 1 )} b t0 ={x 2 } R EAD x 3 b={x 2 } Q UERY x 3 R ESPONSE G ET false x 3 R ESPONSE x 3 R EAD x 2 Q UERY x 2 R ESPONSE R EVALIDATE R EPORT R EAD true x 2 x 2 ,t 4 c={(x 3 ,t 3 )} R ESPONSE b={x 2 } x 2 ,t 4 t = {(x 2, t 4 ), c={(x 2 ,t 4 ),(x 3 ,t 3 )} c={(x 2 ,t 4 )} (x 3, t 3 ),(x 1, t 1 )} W RITE x 1 P UT R EPORT W RITE x 1 =v x 1 b={x 1, x 2 } R ESPONSE t = {(x 2, t 4 ), I NVALIDATE (x 3, t 3 ),(x 1, t 1 )} R ESPONSE inv=true x 1 ok

TTL Estimation Determining th the best TTL TL  Problem: if TTL ≫ time to next write, then it is contained in Cache Sketch unnecessarily long  TTL TL Esti timator: finds „ best “ TTL  Trade-Off: Sh Shorter TTLs Lo Longer TTL TLs • less invalidations • Higher cache-hit rates • less stale reads • more invalidations

TTL Estimation Determining th the best TTL TL Idea: Id 1. Estimate average time to next write 𝐹[𝑈 𝑥 ] for each record 2. Weight 𝐹[𝑈 𝑥 ] using the cache miss rate Client Writes Reads ~ Poisson Caches Writes TTL Estimator Misses ~ Poisson Objective: Server -maximize Cache Hits -minimize Purges collect TTL λ m : Miss Rate -minimize Stale Reads λ w : Write Rate -bound Cache Sketch per record false positive rate λ m λ w

YCSB Monte Carlo lo Cachin ing Sim imulator (YMCA)  Goal: Analysis of arbitrary caching architectures using the standard YCSB benchmark ◦ Metric ics to to evaluate: Latency, Throughput, Cache Hits, Stale Reads, Invalidations Pluggable latency distributions Cache Miss Detector Expiration- Invalidation- purge Database YMCA YCSB SB based based CRUD CRUD Ser erver er Client CRUD workload wo Cache Cache Stale Read CRUD Pluggable simulated caches, Detector choosable topology