Zyzzyva : Speculative Byzantine Fault Tolerance R. Kotla, L. - PowerPoint PPT Presentation

Zyzzyva : 
 Speculative Byzantine Fault Tolerance R. Kotla, L. Alvisi, M.Dahlin, A. Clement, E. Wong Sajjad Rahnama, November 1st 1

Agenda • Introduction • Zyzzyva System Model • Protocol Overview • Node State and Checkpoints • Agreement Protocol • View Change • Correctness • Safety • Liveness 2

Introduction Byzantine Fault State Machine Replication Byzantine Fault Tolerant State Machine Replication 3

Introduction PBFT Practical Byzantine Fault Tolerant Protocol • 3F+1 node • Can Tolerate f faulty node • 3 Phase • Pre-Prepare, Prepare, Commit • 4 One-way messages 4

Introduction PBFT Practical Byzantine Fault Tolerant Protocol Make sure that I didn’t Everyone know that nobody I know That nobody receive two same receive two same sequence receive two same sequence number number sequence number 5

Introduction Zyzzyva “A protocol that uses Speculation to reduce the cost and Simplify the design of BFT state machine replication” 6

Introduction Zyzzyva • Speculative Execution • Replies to the client contain Sufficient history yes Client uses the reply Speculative Response History and response are Stable? History Wait until No converge 7

Introduction Zyzzyva • Challenge is ensuring that response to the client become stable • Move output Commit to the client • Clients act on request in one or two phases 8

Introduction Why Zyzzyva? Cost PBFT Zyzzyva Total Replicas 3f+1 3f+1 Replica with application state 2f+1 2f+1 Critical path 1-way Latency 4 3 9

System Model Assumptions • Faulty nodes may behave Arbitrarily • Faulty nodes cannot break cryptographic signs • Messages may fail to deliver or delay 10

Protocol Overview Subprotocols Agreement View Change Checkpoint 11

Protocol Overview Principles and Challenges • Safety property as they are observed by client • Replicas can be temporarily inconsistent • Client detect them, drive them to convergence • Client rely on consistent responses • Replicas execute the orders before its Order 
 Fully Stablished 12

Protocol Overview Safety f If a request with sequence number n and history h n completes, then any request that completes with a higher sequence number n ′ ≥ n has a history h n ′ that includes h n as a prefix. Liveness Any request issued by a correct client eventually completes. 13

Protocol Overview Protocol Communication Client Send Request to the Primary 14

Protocol Overview Protocol Communication • Primary Forwards the Request to all replicas • Replicas Executes the request 15

Protocol Overview Protocol Communication Gracious execution 3f+1 • Replicas Send Response with history to the client • 3f+1 mutually consistent response then it is done 16

Protocol Overview Protocol Communication Faulty nodes 2f+1 • Some of nodes are faulty • Client Receive between 2f+1 and 3f+1 response 17

Protocol Overview Protocol Communication Faulty nodes 2f+1 2f+1 • Client Gather 2f+1 response and make Commit Certificate • Send’s commit certificate to all nodes 18

Protocol Overview Protocol Communication Faulty nodes 2f+1 2f+1 2f+1 • Client Respond to CC and acknowledge to the Client • Once 2f+1 acknowledgments received client act on request 19

Node State and Checkpoint Ordered History History of executed requests Max Commit Certificate CC seen by node with the largest seq number Committed History History up to seq number of max commit certificate Speculative History History follows the committed history 20

Node State and Checkpoint Checkpoint • A replica constructs a checkpoint every CP_INTERVAL requests. • Similar to other BFT protocols like PBFT Reach Sign and send CP 1) Highest #seq of requests Collect f+1 CP checkpoint message to all 2) digest of current CP message and done interval replicas 21

Node State and Checkpoint Replica State 22

1 Agreement Protocol 2 Step 1 3 • Client Sends Request to the Primary 4 4a • o: operation 4b • t: timestamp 4c • c: client Id 4d 23

1 Agreement Protocol 2 Step 2 3 • Primary receive request and assign seq number 4 • Forward ordered request to all primary 4a 4b v: view number d: H(m) • • h n : H(h n-1 ,d) • • n: sequence number 4c ND: application values • m: client message • 4d 24

1 Agreement Protocol 2 Step 3 3 • Replica receive ordered Request • Check that: 4 • m is wellformed and d is correct digest 4a • n = max n +1 4b • h n = H(h n-1 ,d) • Execute the request and create Spec-Response 4c 4d 25

1 Agreement Protocol 2 Step 3 3 4 • r: reply to the operation • i: replica id 4a • OR: order request 4b 4c Question :What will happen to out of order Sequence numbers? 4d 26

1 Agreement Protocol 2 Step 3 Out of order Sequence numbers: 3 n <= max n +1 Discard the request 4 n > max n +1 The replica has some gap in its history 4a • Replica send Fill-Hole message to the primary 4b 4c • Primary respond with order request for k ≤ n ′ ≤ n 4d Question :What will happen if primary doesn’t answer? 27

1 Agreement Protocol 2 Step 3 If primary doesn’t answer to Fill-Hole Message: 3 4 • After replica timer for fill-hole message expires replica broadcast Fill-Hole message to all replicas 4a Start view change timer • Replicas which receive Fill-Hole message, will forward 
 • 4b Order-Req of corresponding holes to sender if they already have 4c If timer expires and still replica doesn’t receive Order-Reqs it • will initiate view change 4d 28

1 Agreement Protocol 2 Step 4 3 Client Gathers Speculative Responses 4 • Spec-Response messages must mach following properties: v: view number • 4a n: sequence number • c: client id • 4b • H(r): reply digest h n : H(h n-1 ,d) • t: request timestamps • 4c 4d Based on number of speculative response and OR four case could happen 29

1 Agreement Protocol 2 Step 4a 3 • Client Receive 3f+1 matching response 4 • It assumes that request is completed 4a • No acknowledgement will send to replicas 4b • Replicas cannot determine that request is committed 4c 4d 30

1 Agreement Protocol 2 Step 4b 3 When some of nodes are faulty: 4 • Client Receive between 2f+1 and 3f+1 matching response • It assembles 2f+1 response as a Commit-Certificate 4a • Send commit message with CC to all replicas 4b 4c CC is the list of all 2f+1 matching speculative responses 4d 31

1 Agreement Protocol 2 Step 4b-1 3 • Replica receive a commit message from a client containing CC 4 • Replica acknowledge to the client with Local-Commit message • Send CC to all replicas 4a 1) It already has executed request Send Commit Local 4b 2) It hasn’t execute request Update max sequence number and execute operations and send Commit local message 4c 3) Replica has holes in its history Fill the hole as previously discussed 4d 32

1 Agreement Protocol 2 Step 4b-2 3 • Client Receive a Local Commit from a 2f+1 replica 4 • Assume that request is completed • Send CC to all replicas 4a Question :What will happen if doesn’t receive 2f+1 local-commit? 4b • It starts timer when send commit message 4c • If timer expires before 2f+1 one local-commit message 4d then it will act same as 4c step 33

1 Agreement Protocol 2 Step 4c 3 • Client Receive fewer than 2f+1 matching Spec-Response 4 • It Resend the its request to all Replicas • Replicas will forward client request to the primary 4a • A non-primary replica which receive client request 4b • 1) If it has cached response it will send that to client 4c • 2) if the sequence number is new then send 
 Confirm Message to the primary 4d 34

1 Agreement Protocol 2 Step 4c 3 • Replica send Confirm-Message to primary and ask for 4 Order-Request 4a • m is client request 4b • Replica start timer after sending Confirm-Message • If primary accepts then it send response to client 4c • If timer expires then it will initiate view change 4d 35

1 Agreement Protocol 2 Step 4d 3 • Client receive response indicating inconsistent ordering by primary 4 • It sends Proof of Misbehaver to all replicas • They will initiate view change 4a 4b • Inconsistent Ordering: two spec response with valid OR and view number and different sequence number 4c 4d Proof of Misbehavior message 36

View Change View Change Sub Protocol Elect new primary • Must guarantee no change will happen in committed history • The View Change sub protocol is like previous BFT’s ones • 37

View Change View Change step 1 1 • Replica Initiate view change by sending accusation to all replicas 2 3 In previous protocols, this message would indicate that replica is no • 4 longer participating in the current view This message is only a hint that a replica would like to change views • 5 38