Increasing Performance in Byzantine Fault-Tolerant Systems with - PowerPoint PPT Presentation

Increasing Performance in Byzantine Fault-Tolerant Systems with On-Demand Replica Consistency Tobias Distler and R¨ udiger Kapitza System Software Group Friedrich-Alexander University Erlangen-Nuremberg EuroSys April 11, 2011 supported by

Byzantine Fault Tolerance (BFT) Agreement-based Byzantine fault tolerance Replica 0 3 f + 1 replicas to tolerate f faults BFT agreement protocol Client-side voting Replica 1 Client Drawbacks Replica 2 High resource usage Performance overhead for agreement Replica 3 [Castro et al., Practical Byzantine fault tolerance, OSDI ’99] REFIT Project Research Goal Resource-efficient BFT systems Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 2

Making BFT Systems More Resource-Efficient REFIT Project Research Goal Resource-efficient BFT systems Optimizing Resource Usage Optimizing Performance Reduced number of replicas Default number of replicas Same performance Increased performance Recent examples ⇒ ODRC SPARE [Distler et al., NDSS ’11] ZZ [Wood et al., EuroSys ’11] Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 3

Where to Optimize? Agreement 0 Execution 0 Agreement stage BFT agreement protocol Sequence of agreed requests Agreement 1 Execution 1 Client Execution stage Agreement 2 Execution 2 Service application Request processing Agreement 3 Execution 3 [Yin et al., Separating agreement from execution Observations for Byzantine fault tolerant services, SOSP ’03] Past optimizations have significantly reduced agreement overhead Non-trivial services: response times are dominated by execution stage ODRC Approach Reducing the load on the execution stage Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 4

Basic Approach Traditional BFT systems All 3 f + 1 replicas process all requests Client waits for f + 1 identical replies Insight In the absence of faults , a client only needs f + 1 replies to make progress ODRC Each request is processed by only f + 1 replicas Load distribution across replicas Additional replicas process the request in case of faults Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 5

Talk Outline ODRC Selective Request Execution O n- D emand R eplica C onsistency Evaluation Conclusion Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 6

Talk Outline ODRC Selective Request Execution O n- D emand R eplica C onsistency Evaluation Conclusion Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 7

Architecture Selector Selects requests for execution Stores requests that have not been selected Agreement 0 Selector 0 Execution 0 Agreement 1 Selector 1 Execution 1 Client Agreement 2 Selector 2 Execution 2 Agreement 3 Selector 3 Execution 3 Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 8

Application State Set of objects Examples: files, directories, ... Assumption: requests carry information about object access Agreement 0 Selector 0 Execution 0 Agreement 1 Selector 1 Execution 1 Client Agreement 2 Selector 2 Execution 2 Agreement 3 Selector 3 Execution 3 State Objects Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 9

Application State Distribution Object distribution scheme Each object is maintained on f + 1 replicas, unmaintained on others State of unmaintained objects may be outdated Selector 0 Agreement 0 Execution 0 Selector 1 Agreement 1 Execution 1 Client Selector 2 Agreement 2 Execution 2 Selector 3 Agreement 3 Execution 3 Maintained Objects Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 10

Selective Request Execution in Action Normal-case operation Selector 0 0 0 Agreement 0 Execution 0 0 0 Selector 1 0 0 Agreement 1 Execution 1 0 0 Client Selector 2 0 0 Agreement 2 Execution 2 0 0 Selector 3 0 0 Agreement 3 Execution 3 0 0 Garbage collection of stored requests Periodic object checkpoints of maintained objects Stable checkpoint: f + 1 identical checkpoints Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 11

Selective Request Execution in Action Normal-case operation Selector 0 0 0 inc() Agreement 0 Execution 0 0 0 Selector 1 0 0 inc() Agreement 1 Execution 1 0 0 Client Selector 2 0 0 inc() Agreement 2 Execution 2 0 0 Selector 3 0 0 inc() Agreement 3 Execution 3 0 0 Garbage collection of stored requests Periodic object checkpoints of maintained objects Stable checkpoint: f + 1 identical checkpoints Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 11

Selective Request Execution in Action Normal-case operation Selector 0 inc() 0 0 Agreement 0 Execution 0 0 0 Selector 1 inc() 0 0 Agreement 1 Execution 1 0 0 Client Selector 2 0 0 inc() Agreement 2 Execution 2 0 0 Selector 3 0 0 inc() Agreement 3 Execution 3 0 0 Garbage collection of stored requests Periodic object checkpoints of maintained objects Stable checkpoint: f + 1 identical checkpoints Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 11

Selective Request Execution in Action Normal-case operation Selector 0 inc() 0 0 Agreement 0 Execution 0 0 0 Selector 1 inc() 0 0 Agreement 1 Execution 1 0 0 Client Selector 2 0 1 Agreement 2 Execution 2 0 0 Selector 3 0 1 Agreement 3 Execution 3 0 0 Garbage collection of stored requests Periodic object checkpoints of maintained objects Stable checkpoint: f + 1 identical checkpoints Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 11

Selective Request Execution in Action Normal-case operation Selector 0 inc() 0 0 inc() Agreement 0 Execution 0 0 0 Selector 1 inc() 0 0 inc() Agreement 1 Execution 1 0 0 Client Selector 2 0 1 inc() Agreement 2 Execution 2 0 0 Selector 3 0 1 inc() Agreement 3 Execution 3 0 0 Garbage collection of stored requests Periodic object checkpoints of maintained objects Stable checkpoint: f + 1 identical checkpoints Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 11

Selective Request Execution in Action Normal-case operation Selector 0 inc() 0 0 inc() Agreement 0 Execution 0 0 0 Selector 1 inc() 0 0 inc() Agreement 1 Execution 1 0 0 Client Selector 2 inc() 0 1 Agreement 2 Execution 2 0 0 Selector 3 inc() 0 1 Agreement 3 Execution 3 0 0 Garbage collection of stored requests Periodic object checkpoints of maintained objects Stable checkpoint: f + 1 identical checkpoints Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 11

Selective Request Execution in Action Normal-case operation Selector 0 inc() 1 0 Agreement 0 Execution 0 0 0 Selector 1 inc() 1 0 Agreement 1 Execution 1 0 0 Client Selector 2 inc() 0 1 Agreement 2 Execution 2 0 0 Selector 3 inc() 0 1 Agreement 3 Execution 3 0 0 Garbage collection of stored requests Periodic object checkpoints of maintained objects Stable checkpoint: f + 1 identical checkpoints Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 11

Talk Outline ODRC Selective Request Execution O n- D emand R eplica C onsistency Evaluation Conclusion Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 12

Multi-Object Operations Access of multiple objects Only unmaintained objects ⇒ store request At least one maintained object Update unmaintained objects Process request Selector 0 inc() 1 0 Agreement 0 Execution 0 0 0 Selector 1 inc() 1 0 Agreement 1 Execution 1 0 0 Client Selector 2 inc() 0 1 Agreement 2 Execution 2 0 0 Selector 3 inc() 0 1 Agreement 3 Execution 3 0 0 Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 13

Multi-Object Operations Access of multiple objects Only unmaintained objects ⇒ store request At least one maintained object Update unmaintained objects Process request inc() Selector 0 inc() 1 0 Agreement 0 Execution 0 0 0 inc() Selector 1 inc() 1 0 Agreement 1 Execution 1 0 0 Client inc() Selector 2 inc() 0 1 Agreement 2 Execution 2 0 0 inc() Selector 3 inc() 0 1 Agreement 3 Execution 3 0 0 Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 13

Multi-Object Operations Access of multiple objects Only unmaintained objects ⇒ store request At least one maintained object Update unmaintained objects Process request inc() Selector 0 inc() 1 0 Agreement 0 Execution 0 0 0 inc() Selector 1 inc() 1 0 Agreement 1 Execution 1 0 0 Client inc() Selector 2 inc() 0 1 Agreement 2 Execution 2 0 0 inc() Selector 3 inc() 0 1 Agreement 3 Execution 3 0 0 Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 13

Multi-Object Operations Access of multiple objects Only unmaintained objects ⇒ store request At least one maintained object Update unmaintained objects Process request inc() Selector 0 inc() 1 0 Agreement 0 Execution 0 0 0 inc() Selector 1 inc() 1 0 Agreement 1 Execution 1 0 0 Client inc() Selector 2 inc() 0 1 Agreement 2 Execution 2 0 0 Selector 3 inc() inc() 0 1 Agreement 3 Execution 3 0 0 Tobias Distler ( distler@cs.fau.de ) Increasing Performance in BFT Systems with ODRC 13