In fi nite Parallel Universes: State at the Edge Peter Bourgon - PowerPoint PPT Presentation

In fi nite Parallel Universes: State at the Edge Peter Bourgon · Fastly

In fi nite Parallel Universes: State at the Edge Peter Bourgon · Fastly

@ peterbourgon · Fastly In fi nite Parallel Universes Context Architecture Protocol Complications Conclusions

@peterbourgon

@ peterbourgon · Fastly In fi nite Parallel Universes · Context https://www.fastly.com/press/press-releases/fastly-expands-serverless-capabilities-launch-compute-edge

@ peterbourgon · Fastly In fi nite Parallel Universes · Context If Fastly provides compute at the edge , what about state ?

@ peterbourgon · Fastly In fi nite Parallel Universes · Context as of 31 Dec 2019 · https://www.fastly.com/network-map

@ peterbourgon · Fastly In fi nite Parallel Universes · Context 133ms as of 31 Dec 2019 · https://www.fastly.com/network-map

@ peterbourgon · Fastly In fi nite Parallel Universes · Context What are we doing here - Not a general purpose database - Operate in the request lifecycle - Local (POP) reads and writes - Eventually consistent - State con fl icts are normal ⇒ Sort of a writeable cache

@ peterbourgon · Fastly In fi nite Parallel Universes · Context Data model - Lean in to the physical constraints - A single, global truth? — No, a fi ction! - Multiple, simultaneous truths — Reality! - Converge toward a stable global state

@ peterbourgon · Fastly In fi nite Parallel Universes · Context State primitive - CRDT - Type-speci fi c methods + Merge - Associative, commutative, idempotent - Tolerates out-of-order, duplicate merges ⇒ Reduces higher-order complexity

@ peterbourgon · Fastly In fi nite Parallel Universes · Context Integer addition - Associative? (a + b) + c = a + (b + c) ✔ - Commutative? a + b = b + a ✔ - Idempotent? a + a = a ✘ ⇒ Not a CRDT

@ peterbourgon · Fastly In fi nite Parallel Universes · Context Set union - Associative? ({a} ∪ {b}) ∪ {c} = {a} ∪ ({b} ∪ {c}) — Yes - Commutative? {a} ∪ {b} = {b} ∪ {a} — Yes - Idempotent? {a} ∪ {a} = {a} — Yes ⇒ A CRDT

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} {1} {1} {1}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4 5 6}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6} {1 3 4 5 6}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 2 3 4 5 6} {1 3 4 5 6}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 2 3 4 5 6} {1 3 4 5 6} {1 2 3 4 5 6}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 2 3 4 5 6}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 2 3 4 5 6}

@ peterbourgon · Fastly In fi nite Parallel Universes · Context {1} 4 2 3 5 3 {1} {1} {1} 4 6 {1 2 3} {1 3 4} {1 4 5 6} Join semilattice {1 2 3 4} {1 2 3 4 5 6} {1 3 4 5 6} {1 2 3 4 5 6}

@ peterbourgon · Fastly In fi nite Parallel Universes Context Architecture Protocol Complications Conclusions

@ peterbourgon · Fastly In fi nite Parallel Universes · Architecture YVR AMS HND JHB AKL EZE as of 31 Dec 2019 · https://www.fastly.com/network-map

@ peterbourgon · Fastly In fi nite Parallel Universes · Architecture 1. Single write primary Write primary ··· AKL AMS YVR Read replica Read replica Read replica Write Familiar. Consistent. Slow. Doesn't satisfy local read/write criteria.

@ peterbourgon · Fastly In fi nite Parallel Universes · Architecture 2. Gossip network ··· AKL AMS YVR Replica Replica Replica Write Lots of connections. Where do objects live? How do objects propagate? etc.

@ peterbourgon · Fastly In fi nite Parallel Universes · Architecture 3. Hub-and-spoke Upstream ··· AKL AMS YVR Write Replica Replica Replica Fewer connections. Objects live in root. Replica like LRU cache.

@ peterbourgon · Fastly In fi nite Parallel Universes Context Architecture Protocol Complications Conclusions

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol B A C Protocol (i) - How do objects get from A to B? - Track objects that receive any request - Batch them into epochs - Regularly emit batches

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol B A C Protocol (ii) - How do objects get from B to C? - Bad answer: push from B to C - Key insight: sites know which objects have been requested - Better answer: pull relevant objects to C from B — sync

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Site Site State Interest 1. Write K1=V1 {K1:V1 {K1 2. Read K2: miss K2 3. Write K3=V3 K3:V3} K3}

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Site Sync step 1 State Interest SyncRequest + = After some time, {K1:V1 {K1 {K1:V1 make a SyncRequest K2 K2:-- with all interesting K3:V3} K3} K3:V3} keys and values.

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Site Upstream Sync step 2 Send the SyncRequest SyncRequest to the upstream. {K1:V1 K2:-- K3:V3}

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Upstream Sync step 3 Upstream merges New state State SyncRequest = ⋃ incoming data to {K1:V5 ⋃ V1=V8 {K1:V5 {K1:V1 its own data. K2:V6 K2:-- K2:V6 K3:V3} K3:V3 K4:V7} K4:V7}

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Upstream Site Sync step 4 Upstream returns a SyncRequest SyncResponse to the {K1:V1 site, with latest value K2:-- K3:V3} for each requested key. SyncResponse {K1:V8 K2:V6 K3:V3}

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Site Sync step 5 Site merges the New state State SyncResponse = ⋃ SyncResponse to {K1:V1 ⋃ V8=V8 {K1:V1 {K1:V8 its own state, and K2:V6 K2:V6 resets the interest K3:V3} K3:V3} K3:V3 ⋃ V3=V3} set.

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Site Upstream Properties of sync (i) - Synchronous - The only way data moves State State - Bandwidth minimized {K1:V8 {K1:V8 - Data sets minimized K2:V6 K2:V6 - Authoritative upstream K3:V3 K3:V3} K4:V7}

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Site Upstream Properties of sync (ii) - Schedule is fl exible - Missed syncs impact liveness, State State not correctness {K1:V8 {K1:V8 - Eventually consistent K2:V6 K2:V6 - Quiet by default K3:V3 K3:V3} K4:V7}

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Upstream Site Site Site

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Upstream Site Site

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Incoming user requests - Requests are hashed and proxied to N replicas - Response is the union of all returned CRDTs - Compare union with individual responses - Discrepant replicas easily identi fi ed, fi xed

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Upstream ? Site Site

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Upstream Outgoing sync requests - Each instance has di ff erent set of objects - But CRDTs tolerate over-merging - Instances may sync totally independently - Upstream state converges to stability

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol ? ? ? ?

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Incoming sync requests - User request = 1 key · Op - Sync = N keys · Merge - Otherwise identical - Choose any upstream replica

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Fractal design - N-ary tree - Can be hierarchical - May insulate against regional connectivity issues - Tradeo ff between liveness and capacity

@ peterbourgon · Fastly In fi nite Parallel Universes · Protocol Smart primitives, simple systems - One operation for moving data - No coördination between - Faults need no handling besides retry ⇒ All from CRDT properties

@ peterbourgon · Fastly In fi nite Parallel Universes Context Architecture Protocol Complications Conclusions