Declarative, Secure, Convergent Edge Computation Christopher - PowerPoint PPT Presentation

Declarative, Secure, Convergent Edge Computation Christopher Meiklejohn Université catholique de Louvain, Belgium 1

Internet of Things 2

Internet of Things but, more generally… 2

Edge Computation • Logical extremes 
 Pushing both computation and data to the logical extremes of the network 3

Edge Computation • Logical extremes 
 Pushing both computation and data to the logical extremes of the network • Arbitrary computation 
 Support arbitrary computations regardless of location of data in the network 3

Edge Computation • Logical extremes 
 Pushing both computation and data to the logical extremes of the network • Arbitrary computation 
 Support arbitrary computations regardless of location of data in the network • Self-organizing, resilient 
 Directed diffusion, Cornell circa-1990; self-organizing systems that coordinate to complete computations 3

Example Application Hospital Refrigerators 4

Hospital Refrigerators Typical Topology 5

6

Internet 7

Internet HDFS 8

Client Hadoop Internet Client Internet HDFS 9

Internet HDFS Client Spark Internet Client 10

Hospital Refrigerators Ideal Execution 11

Spark Internet HDFS 12

Spark Internet HDFS 13

Spark Internet HDFS 14

Spark Internet HDFS 15

Spark Internet HDFS 16

Spark Client Internet HDFS 17

Problem Connectivity 18

Spark Internet HDFS 19

Spark Internet HDFS 20

Spark Internet HDFS 21

Spark Internet HDFS 22

Spark Internet HDFS 23

Spark Internet HDFS 24

Spark Internet HDFS 25

Solution Local Decisions 26

Spark Internet HDFS 27

Spark Internet HDFS 28

Spark Internet HDFS 29

Spark Internet HDFS 30

Spark Internet HDFS 31

Local Decisions • Not new for backup (80s-90s) 
 Backup communication mechanisms for critical systems; POTS backup for ISDN, etc. 32

Local Decisions • Not new for backup (80s-90s) 
 Backup communication mechanisms for critical systems; POTS backup for ISDN, etc. • Not new for storage (90s-00s) 
 EMC’s “phone home” via POTS when disks failed in NAS devices to signal for replacement unit 32

Solution Transitive Dissemination 33

Spark Internet HDFS 34

Spark Internet HDFS 35

Spark Internet HDFS 36

Spark Internet HDFS 37

Spark Internet HDFS 38

Spark Client Internet HDFS 39

Problem State Transmission 40

Internet 41

Internet 42

Internet 43

Internet 44

Solution Aggregate Dissemination 45

Internet 46

Internet 47

Internet 48

Internet HDFS 49

Internet HDFS = = 50

Internet ? = = 51

Internet ? = = 52

Local Computation • Reduce state transmission 
 Perform some local computation to reduce transmitted state on the wire 53

Local Computation • Reduce state transmission 
 Perform some local computation to reduce transmitted state on the wire • Make local decisions 
 Make decisions based on results of local computation 53

Databases Consistency Models 54

Databases Strong Consistency 55

R1 C1 C2 56

R1 C1 C2 57

R1 Read C1 C2 58

R1 Read C1 C2 59

R1 C1 C2 60

R1 CAS C1 C2 61

R1 C1 C2 62

R1 CAS C1 C2 63

I won’t diagram the Paxos protocol 64

Value 2 R1 R2 R3 Paxos C1 C2 Value 2 Value 1 65

Databases Eventual Consistency 66

R1 R2 R3 C1 C2 67

R1 R2 R3 C1 C2 68

R1 R2 R3 Read C1 C2 69

R1 R2 R3 Read C1 C2 70

R1 R2 R3 Write C1 C2 71

R1 R2 R3 Write C1 C2 72

R1 R2 R3 Write C1 C1 C2 73

R1 R2 R3 Read C1 C2 74

R1 R2 R3 Write C1 C2 75

Eventual Consistency As The Model 76

Clients Own Their Data 77

78

79

80

Computations Mergeability & Provenance 81

A C A B 82

A F D’ C A F D’’ B 83

A F D’ C ≤ ≤ D’ D’’ D A F D’’ B 84

A F D’ D C ≤ ≤ D’ D’’ D Merge A F D’’ D B 85

Example Application Preliminary Results 86

Preliminary Results • Conflict-Free Replicated Data Types 
 Distributed data structures designed for convergence 
 [Shapiro et al., 2011] 87

Preliminary Results • Conflict-Free Replicated Data Types 
 Distributed data structures designed for convergence 
 [Shapiro et al., 2011] • Lattice Processing 
 Make decisions based on results of local computation 
 [Meiklejohn & Van Roy, 2015] 87

Preliminary Results • Conflict-Free Replicated Data Types 
 Distributed data structures designed for convergence 
 [Shapiro et al., 2011] • Lattice Processing 
 Make decisions based on results of local computation 
 [Meiklejohn & Van Roy, 2015] • Selective Hearing 
 Scalable, epidemic broadcast based runtime system 
 [Meiklejohn & Van Roy, 2015/2016] 87

Conflict-Free 
 Replicated Data Types • Collection of types 
 Sets, counters, registers, flags, maps 88

Conflict-Free 
 Replicated Data Types • Collection of types 
 Sets, counters, registers, flags, maps • Strong Eventual Consistency (SEC) 
 Objects that receive the same updates, regardless of order, will reach equivalent state 88

RA RB RC

add(1) RA {1} (1, {a}, {}) RB RC

add(1) RA {1} (1, {a}, {}) RB add(1) RC {1} (1, {c}, {})

add(1) RA {1} (1, {a}, {}) RB add(1) remove(1) RC {1} {} (1, {c}, {c}) (1, {c}, {})