Abstractions to Build Permissionless Systems Soutenance d’habilitation à diriger des recherches Emmanuelle Anceaume ( CNRS/IRISA ) CIDRE research team Amr El Abbadi Antonio Fernández Anta Nicolas Hanusse Fernando Pedone Yvonne-Anne Pignolet Francois Taiani Philippas Tzigas December 18-th, 2019 1
Distributed computing ◮ Solving a common problem with distinct entities ◮ Unpredictability due to failures, asynchrony, and local view ◮ Encapsulating the difficulty of robust cooperation within abstractions no yes ? no yes no 2
Goal of distributing computing ◮ Identifying problems that are abstractions of those that arise in a variety of distributed situations ◮ Stating precisely these abstractions in terms of properties ◮ Designing and analyzing efficient distributed algorithms to solve them 3
Abstractions 4
Abstractions Reliable broadscast Causal broadcast R/W register Atomic broadcast 5
Abstractions Reliable broadscast Clock synchronization Causal broadcast Snapshot R/W register Mutual exclusion Atomic broadcast 6
Abstractions Consensus Reliable broadscast Clock synchronization Leader election Causal broadcast Snapshot Lattice agreement Replicated state machine R/W register Mutual exclusion Atomic broadcast 7
Implementing abstractions Asynchronous Consensus Reliable broadscast Timing model Clock synchronization Leader election Causal broadcast Snapshot Lattice agreement Replicated state machine Synchronous R/W register Mutual exclusion Atomic broadcast 8
Implementing abstractions Asynchronous Consensus Reliable broadscast Timing model Clock synchronization Leader election Causal broadcast Snapshot Lattice agreement Replicated state machine Synchronous R/W register Mutual exclusion Atomic broadcast Failure model Byzantine Crash 9
Implementing abstractions Asynchronous g n i s s a p - e g a Consensus s s e M Communication model Reliable broadscast Timing model Clock synchronization Leader election Causal broadcast Snapshot Lattice agreement Replicated state machine Shared memory Synchronous R/W register Mutual exclusion Atomic broadcast Failure model Byzantine Crash 10
Implementation in « classic » distributed systems ◮ Known and fixed size set of participants ◮ Participants are authorized nodes ◮ Complete communication graph 6 5 1 4 2 3 11
Permissionless systems Unbounded number of participants Open membership Connected graph 12
Permissionless enlarges adversarial strategies spectrum ◮ Churn strategies ◮ Join and leave at just the worst possible time ◮ Byzantine strategies ◮ Sybil attacks : generation of numerous fake identities ◮ Eclipse attacks : manipulation of routing tables ◮ Targeted attacks : isolation of nodes ◮ Rational / irrational behaviors ◮ Free-ride ◮ Make the service sustainable 13
Abstractions implementation : Scalability issues ◮ For both safety and liveness guarantees ◮ Implementations rely on (Byzantine) quorums ◮ Quorum size in O ( n ) , where n is the size of the system |Q 1 |= Q correct faulty |Q 2 |= Q n nodes among which f are Byzantine ◮ Round structure of algorithms ◮ Message complexity in O ( n 2 ) , where n is the size of the system 14
Outline of the presentation ◮ Focus on the implementation of distributed abstractions in permissionless systems ◮ Reliable broadcast abstraction ◮ Consensus abstraction ◮ Replicated state machine abstraction 15
Reliable broadcast abstraction ◮ Allows any node in the system to disseminate a message to all the other nodes ◮ Liveness and safety properties ◮ Termination ◮ Uniqueness ◮ Integrity 16
Broadcast abstraction ◮ Randomized rumor spreading ◮ Nodes pick random neighbors and exchange information (rumor) with them ◮ Local, tolerant to churn, scalable ◮ ex. Y t = number of nodes that have received the rumor after t interactions ◮ ∀ δ ∈ ( 0 , 1 ) and t ≥ n (ln( n ) − ln( δ / 2 )) , P { Y t = n } ≥ 1 − δ 𝜐 (n, δ =10 -3 ) Simulations with δ =10 -3 𝜐 (n, δ =10 -2 ) Simulations with δ =10 -2 Parallel time to propagate a rumor 𝜐 (n, δ =10 -1 ) Simulations with δ =10 -1 Number of nodes 17
Broadcast abstraction ◮ Randomness is fundamental ◮ Uniformity : For any discrete time t ≥ 0, for any nodes i and j in the system P { routing_table i ( t ) = j } = 1 / n . ◮ Node sampling service ◮ Local to nodes ◮ Returns the identifier of a random node stream of node ids S (k) i Sampling memory Γ i ◮ Making this service robust to eclipse attacks ◮ The adversary injects many corrupted node identifiers to bias sampling ! 18
Node sampling : omniscient strategy ◮ Suppose that ◮ Each node id j received in the input stream is tagged with its occurrence probability p j in the full input stream ◮ Let q = min j p j ◮ Omniscient strategy : ◮ Pick id j with probability a j = q / p j ◮ Whatever the frequency of ids in the stream, they are output uniformly Sampling memory Input stream of node identifiers p 13 Output stream a 13 61 16 24 14 13 1 56 16 14 14 2 13 14 98 56 11 11 16 Get_node() Constant size 19
Node sampling : Knowledge-free strategy ◮ Need to estimate the frequency of each identifier in the input stream ◮ Rely again on randomness ! ◮ Cormode’s Count Min sketch data structure ◮ Ingredients ◮ t hash functions and an array of t × k counters h 1 :[N]-> [ 𝑙 ] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h 2 :[N]-> [ 𝑙 ] 0 0 0 0 0 0 0 0 𝑢 = log(1/ δ) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 h t :[N]-> [ 𝑙 ] 0 0 0 0 0 0 0 0 𝑙 = 𝑓/𝜁 20
Node sampling : Knowledge-free strategy ◮ Need to estimate the frequency of each identifier in the input stream ◮ Rely again on randomness ! ◮ Cormode’s Count Min sketch data structure +1 0 0 0 0 0 0 0 0 0 0 0 +1 0 0 0 Stream of node identifiers 0 0 0 0 0 0 +1 0 13 22 16 21 14 14 21 13 21 61 16 21 14 0 0 0 +1 0 0 0 0 +1 0 0 0 0 0 0 0 0 0 +1 0 0 0 0 0 21
Node sampling : Knowledge-free strategy ◮ By taking ˆ f j = min t h t [ j ] then ◮ ˆ f j ≥ f j , and ◮ ∀ ε ∈ ( 0 , 1 ) , ∀ δ ∈ ( 0 , 1 ) , P { ˆ f j ≤ f j + ε m } ≥ 1 − δ 0 0 1 0 0 0 1 0 0 0 0 0 2 0 0 0 Stream of node identifiers 0 0 0 0 0 1 1 0 1 21 14 13 22 16 21 14 14 21 21 61 16 0 0 1 1 0 0 0 0 0 0 0 0 0 1 0 1 0 0 2 0 0 0 0 0 22
Node sampling robust to eclipse attacks 23
Reliable broadcast abstraction ◮ Lessons learnt from this study ◮ Randomization is an essential ingredient ◮ Adversarial strategies can be defeated by increasing memory footprint ◮ Adversarial strategies do not impact scalability ◮ Message complexity : O (log n ) ◮ Mode details in [OPODIS 2011, DSN 2013, IJOC 2019] 24
Consensus abstraction ◮ One of the most important abstraction in fault tolerant distributed computing ◮ Guarantee a unique decision among a set of input values Liveness and safety properties ◮ Termination ◮ Agreement ◮ Validity 25
Randomly chosen committees General idea : ◮ Execute consensus algorithms on O ( log n ) nodes ◮ Committee C made of random nodes → µ C = µ system ◮ How electing random nodes ? ◮ How this election is resilient to adversarial behaviors ? 26
PeerCube : Nodes self-organize within committees ◮ Size of committees is lower and upper bounded ◮ Enforce role separation at committee level : core and spare sets ◮ Only core sets are visible from the outside 0110 spare set size max=O(log N) 0111 1110 0010 1111 0011 1010 1100 1101 1000 1001 core set 0101 constant size=C 0100 0000 0001 0000101000 27
PeerCube : Significantly reduces churn impact 1000 No core/spare classification Number of routing tables updates per node PeerCube 100 10 1 0.1 0.01 0.001 5000 10000 15000 20000 25000 30000 Simulation time - every 500 time units, up to 500 nodes join or leave the system 28
PeerCube : Resistance to adversarial strategies Scalability ◮ Byzantine algorithms within constant size committees ◮ Randomization scheme to elect these committees Resistance to targeted attacks ◮ Move ! ! ◮ Induced churn : pushing nodes to random positions within the system 29
PeerCube : Resistance to an adaptive adversary Impact of induced churn on committees state µ system = 0 % µ system = 10 % d 0 . 90 0 . 99 0 . 999 0 . 90 0 . 99 0 . 999 safe state 100 % 100 % 100 % 99 . 17 % 82 . 28 % 0 . 78 % polluted state 0 % 0 % 0 % 0 . 82 % 17 . 71 % 99 . 21 % µ system = 20 % µ system = 30 % d 0 . 90 0 . 99 0 . 999 0 . 90 0 . 99 0 . 999 safe state 95 . 36 % 1 . 66 % ∼ 0 . 0 % 87 . 36 % 0 . 09 % ∼ 0 . 0 % polluted state 4 . 75 % 98 . 33 % ∼ 100 % 12 . 63 % 99 , 90 % ∼ 100 % ◮ A pinch of induced churn is sufficient to defend against targeted attacks 30
Consensus Abstraction ◮ Lessons learnt from this study ◮ Clustering makes possible the design of robust operations ◮ Induced churn significantly reduces adversarial targeted attacks ◮ Only a small amount of these ingredients is necessary and sufficient ◮ More details in [MCAP 2013, SIGMETRICS P.E.R 2012, IJFCS 2011, DSN 2011, ICC 2010, SSS 2009, SASO 2008] 31
Recommend
More recommend
