Aspect Ratio Test Slide APACHE COUCHDB SYNC DEEP DIVE by Jan - PowerPoint PPT Presentation

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia CA7BEA04… Chickleston Cecilia BE2EBD0B… Burrows Bertram DFFCFABC… Dunkington David CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David By-Sequence Update Sequence Doc ID 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B…

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David By-Sequence Update Sequence Doc ID 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B…

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David By-Sequence Update Sequence Doc ID 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B…

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David By-Sequence Update Sequence Doc ID High Watermark: 4 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B…

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255… Doe John By-Sequence Update Sequence Doc ID High Watermark: 4 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 5 E328E255…

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255… Doe John E328E255… Doe John By-Sequence Update Sequence Doc ID High Watermark: 5 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 5 E328E255… Next: Updates

UPDATES

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255…* Esterhase John E328E255… Doe John By-Sequence Update Sequence Doc ID High Watermark: 5 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 5 E328E255… 6 E328E255…*

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255…* Esterhase John E328E255…* Esterhase John By-Sequence Update Sequence Doc ID High Watermark: 6 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 5 E328E255… 6 E328E255…* Now we go back and make a typo

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255…** Esterhose John E328E255… Doe John By-Sequence Update Sequence Doc ID High Watermark: 5 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 5 E328E255… 6 E328E255…* 7 E328E255…**

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255…** Esterhose John E328E255…* Esterhase John By-Sequence Update Sequence Doc ID High Watermark: 6 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 5 E328E255… 6 E328E255…* 7 E328E255…** Wouldn’t it be more e ffi cient to just send update 7?

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255…** Esterhose John E328E255…** Esterhose John By-Sequence Update Sequence Doc ID High Watermark: 7 1 CA7BEA04… 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 7 E328E255…** Why yes it would -> make the sequence table unique for doc ids Next: deletes

DELETES

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255…** Esterhose John E328E255…** Esterhose John By-Sequence Update Sequence Doc ID High Watermark: 7 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 7 E328E255…** 8 CA7BEA04… Wouldn’t it be more e ffi cient to just send update 7? Why yes it would -> make the sequence table unique for doc ids

Database A Database B ID Last Name First Name ID Last Name First Name A0D31890… Abblesworth Antonia A0D31890… Abblesworth Antonia BE2EBD0B… Burrows Bertram BE2EBD0B… Burrows Bertram CA7BEA04… Chickleston Cecilia CA7BEA04… Chickleston Cecilia DFFCFABC… Dunkington David DFFCFABC… Dunkington David E328E255…** Esterhose John E328E255…** Esterhose John By-Sequence Update Sequence Doc ID High Watermark: 8 2 DFFCFABC… 3 A0D31890… 4 BE2EBD0B… 7 E328E255…** 8 CA7BEA04… Wouldn’t it be more e ffi cient to just send update 7? Why yes it would -> make the sequence table unique for doc ids

SYNC INGREDIENTS ➤ Identity ➤ What happened since?

QUESTIONS?

VERSIONS * ** business, what’s up with that?

AUTO INCREMENT * ** business, what’s up with that?

VERSIONS: AUTO INCREMENT? Database A Database B ID Version Last Name First Name ID Version Last Name First Name A0D31890 1 Abblesworth Antonia …

VERSIONS: AUTO INCREMENT? Database A Database B ID Version Last Name First Name ID Version Last Name First Name A0D31890 A0D31890 1 Abblesworth Antonia 1 Abblesworth Antonia … …

VERSIONS: AUTO INCREMENT? Database A Database B ID Version Last Name First Name ID Version Last Name First Name A0D31890 A0D31890 2 Bobble sworth Antonia 2 Wibble sworth Antonia … … What is it about the auto-increment that is so appealing: strictly ordered in math terms: monotonically increasing what else is monotonically increasing?

VERSIONS: AUTO INCREMENT? Database A Database B ID Version Last Name First Name ID Version Last Name First Name A0D31890 A0D31890 2 Bobble sworth Antonia 2 Wibble sworth Antonia … … 👏 👏 What is it about the auto-increment that is so appealing: strictly ordered in math terms: monotonically increasing what else is monotonically increasing?

TIME! From: Falsehoods programmers believe about time, and the sequel. ALL OF THESE STATEMENTS ARE FALSE http://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time http://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time

THE SYSTEM CLOCK WILL ALWAYS BE SET TO THE CORRECT LOCAL TIME.

THE SYSTEM CLOCK WILL ALWAYS BE SET TO A TIME THAT IS NOT WILDLY DIFFERENT FROM THE CORRECT LOCAL TIME.

IF THE SYSTEM CLOCK IS INCORRECT, IT WILL AT LEAST ALWAYS BE OFF BY A CONSISTENT NUMBER OF SECONDS.

THE SERVER CLOCK AND THE CLIENT CLOCK WILL ALWAYS BE SET TO THE SAME TIME.

THE SERVER CLOCK AND THE CLIENT CLOCK WILL ALWAYS BE SET TO AROUND THE SAME TIME.

OK, BUT THE TIME ON THE SERVER CLOCK AND TIME ON THE CLIENT CLOCK WOULD NEVER BE DIFFERENT BY A MATTER OF DECADES.

IF THE SERVER CLOCK AND THE CLIENT CLOCK ARE NOT IN SYNC, THEY WILL AT LEAST ALWAYS BE OUT OF SYNC BY A CONSISTENT NUMBER OF SECONDS.

THE SERVER CLOCK AND THE CLIENT CLOCK WILL USE THE SAME TIME ZONE.

THE SYSTEM CLOCK WILL NEVER BE SET TO A TIME THAT IS IN THE DISTANT PAST OR THE FAR FUTURE.

ONE MINUTE ON THE SYSTEM CLOCK HAS EXACTLY THE SAME DURATION AS ONE MINUTE ON ANY OTHER CLOCK

OK, BUT THE DURATION OF ONE MINUTE ON THE SYSTEM CLOCK WILL BE PRETTY CLOSE TO THE DURATION OF ONE MINUTE ON MOST OTHER CLOCKS.

FINE, BUT THE DURATION OF ONE MINUTE ON THE SYSTEM CLOCK WOULD NEVER BE MORE THAN AN HOUR.

A TIME STAMP OF SUFFICIENT PRECISION CAN SAFELY BE CONSIDERED UNIQUE.

IT’S POSSIBLE TO ESTABLISH A TOTAL ORDERING ON TIMESTAMPS THAT IS USEFUL OUTSIDE YOUR SYSTEM.

TIMESTAMPS ALWAYS ADVANCE MONOTONICALLY.

MY SOFTWARE IS ONLY USED INTERNALLY/LOCALLY, SO I DON’T HAVE TO WORRY ABOUT TIMEZONES

MY SOFTWARE STACK WILL HANDLE TIMEZONE WITHOUT ME NEEDING TO DO ANYTHING SPECIAL

ALL MEASUREMENTS OF TIME ON A GIVEN CLOCK WILL OCCUR WITHIN THE SAME FRAME OF REFERENCE. In other words

TIME PASSES AT THE SAME SPEED ON TOP OF A MOUNTAIN AND AT THE BOTTOM OF A VALLEY.

TIMESTAMPS? NO Spanner 2FA Whatever the exact scenario, this is plausible and has documented occurrences in small and large-scale systems: you can’t rely on timestamps to guarantee the order of two items, even if the timestamps were generated on the same device as they lead to data loss and/or duplication. This is what happens under the hood when you suddenly have all your notes or contacts twice, after syncing your phone and your desktop. Or why that one contact always gets deleted when you try to sync from phone to desktop (but not the other way around) Ok. How can we improve on timestamps? Before we find out, we need to introduce one more new concept: conflicts.

DISTRIBUTED SYSTEMS INTERLUDE