Evan Tschannen 1 Evan Tschannen Worked on FoundationDB for 8 - PowerPoint PPT Presentation

Evan Tschannen � 1

Evan Tschannen • Worked on FoundationDB for 8 years • Touched every core component of the database • Recently added multi-region asynchronous replication

• https://www.foundationdb.org • https://github.com/apple/foundationdb • Open Source, Apache License (v2)

Why are there so many databases? � 4

Why are there so many databases? • Tension between performance, scalability, and consistency • A wide variety of data models � 5

FoundationDB Performance • Organizes many instances of a single process database into a single database • Read and write throughput 90% of the aggregate individual read and write throughput • Single hop read latencies • Four hop write latencies

FoundationDB is a… • Highly performant, • Scalable, • Ordered key value store, • That supports ACID transactions.

Servers A Writer Reader Wait for 2 of 3 responses Wait for 2 of 3 responses B C � 8

Servers A Writer Reader Wait for 2 of 3 responses Wait for 2 of 3 responses B C � 9

Servers A Writer Reader Wait for 3 of 3 responses Wait for 1 of 3 responses B C � 10

Servers A Writer Reader Wait for 3 of 3 responses Wait for 1 of 3 responses B C � 11

0: A,B,C 101: B,C,D Versions (Stored in a di ff erent database) Servers A Writer Reader Wait for 3 of 3 responses Wait for 1 of 3 responses B C D � 12

Coordinators ZooKeeper (Failure Handling) Storage Servers Transaction Logs SQLite (Reads) FoundationDB (Writes) � 13

Coordinators Cluster Controller Storage Servers Transaction Logs � 14

Coordinators Cluster Controller Storage Servers Transaction Logs � 15

FoundationDB API Versions A : apple 100 Get Read Version (200) 200 Read (A, 200) Write (B, banana) Commit 400 � 16

FoundationDB API Versions A : apple 100 Get Read Version (200) 200 Read (A, 200) 300 A : orange Write (B, banana) Commit 400 � 17

Coordinators Cluster Controller Master Storage Servers Proxies Resolvers Transaction Logs � 18

Coordinators Get Read Version Reads Cluster Controller Writes Commit Master Storage Servers Proxies Resolvers Transaction Logs � 19

Coordinators Get Read Version (200) Reads (A, 200) Cluster Controller Writes (B : banana) Commit Master Data Distribution Storage Servers Proxies Resolvers Transaction Logs � 20

Coordinators Get Read Version (200) Reads (A, 200) Cluster Controller Writes (B : banana) Commit Master (400) Storage Servers Proxies Resolvers Transaction Logs � 21

Coordinators Get Read Version (200) Reads (A, 200) Cluster Controller Writes (B : banana) Commit Master (400) Storage Servers Proxies Resolvers Transaction Logs � 22

Coordinators Get Read Version (200) Reads (A, 200) Cluster Controller Writes (B : banana) Commit Master Storage Servers Proxies Resolvers Transaction Logs � 23

Coordinators Get Read Version (200) Reads (A, 200) Cluster Controller Writes (B : banana) Commit (400) Master Storage Servers Proxies Resolvers Transaction Logs � 24

Coordinators Get Read Version Reads Cluster Controller Writes Commit Master Storage Servers Proxies Resolvers Transaction Logs � 25

Coordinators Get Read Version Reads Cluster Controller Writes Commit Master Storage Servers Proxies Resolvers Transaction Logs (400) (300) (200) � 26

Coordinators Get Read Version (400) Reads Cluster Controller Writes Commit Master Storage Servers Proxies Resolvers Transaction Logs (400) (300) (200) � 27

Coordinators Get Read Version Reads Cluster Controller Writes Commit Master Storage Servers Proxies Resolvers Transaction Logs � 28

Coordinators Get Read Version Reads Cluster Controller Writes Commit Master Storage Servers Proxies Resolvers Transaction Logs � 29

Coordinators Get Read Version Reads Cluster Controller Writes Commit Master Storage Servers Proxies Resolvers Transaction Logs � 30

Coordinators Get Read Version Reads Cluster Controller Writes Commit Master Data Distribution Storage Servers Proxies Resolvers Transaction Logs � 31

Coordinators Get Read Version Reads Cluster Controller Writes Commit Master Storage Servers Proxies Resolvers Transaction Logs � 32

FoundationDB Performance • Organizes many instances of a single process database into a single database • Read and write throughput 90% of the aggregate individual read and write throughput • Single hop read latencies • Four hop write latencies

Does it work? • Run the entire distributed database in a single process • Inject random failures • Check correctness • Reproduce errors

https://www.foundationdb.org � 35