applying the saga pattern
play

Applying the Saga Pattern Caitie McCafffrey Caitie McCaffrey - PowerPoint PPT Presentation

Jun 19, 2023 •606 likes •1.53k views

Applying the Saga Pattern Caitie McCafffrey Caitie McCaffrey Distributed Systems Engineer @Caitie CaitieM.com Why Sagas? Sagas Paper Distributed Sagas Sagas in Halo 4 Systems Used to Be Simple Serializability & ACID

  1. Applying the Saga Pattern Caitie McCafffrey

  2. Caitie McCaffrey � Distributed Systems Engineer @Caitie CaitieM.com

  4. • Why Sagas? • Sagas Paper • Distributed Sagas • Sagas in Halo 4

  5. Systems Used to Be Simple

  6. Serializability & ACID

  7. SOA/Microservices

  8. Two Phase Commit

  9. 2PC: Prepare Phase

  10. 2PC: Commit Phase

  11. 2PC Doesn’t Scale • O(n^2) Messages • Coordinator: Single Point of Failure • Reduced Throughput

  12. Spanner Google’s Globally-Distributed Database Corbett et. al.

  13. “Spanner is Google’s scalable, multi-version, globally distributed , and synchronously- replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed transactions .” –Corbett et al.

  14. “The key enabler of these properties is a new TrueTime API and its implementation…using multiple modern clock references ( GPS and atomic clocks ).” –Corbett et al.

  15. Spanner is Expensive & Proprietary • Custom Hardware & Data-Centers • Synchronization Not Solved

  16. Distributed Transactions are Hard & Expensive

  17. Can We Do Better?

  18. Can We Do Better?

  19. Sagas Hector Garcia-Molina, Kenneth Salem Princeton University 1987

  20. Sagas are Long Lived Transactions

  21. “A Saga is a Long Lived Transaction that can be written as a sequence of transactions that can be interleaved. All transactions in the sequence complete successfully or compensating transactions are ran to amend a partial execution.”

  22. A Saga is a Collection of Sub-Transactions T 1 , T 2 … T n

  23. Each Sub-Transaction has a Compensating Transaction C 1 , C 2 … C n

  24. C n Semantically Undoes T n

  25. Saga Guarantee Either • T 1 , T 2 … T n or • T 1 , T 2 … T j, C j , … C 2 , C 1

  26. Trade-Off: Atomicity for Availability

  27. Sagas are a Failure Management Pattern

  28. Large Single Transaction

  29. Sagas • Book Hotel (T 1 ) • Cancel Hotel (C 1 ) • Book Car (T 2 ) • Cancel Car (C 2 ) • Book Flight (T 3 ) • Cancel Flight (C 3 )

  30. Saga Execution Coordinator (SEC)

  31. Saga Log • Begin T i • Begin Saga • End T i • End Saga • Begin C i • Abort Saga • End C i

  32. Successful Saga Begin Saga � Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  33. Successful Saga Begin Saga Start Book Hotel (T 1 ) � End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  34. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) � Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  35. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) � End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  36. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) � Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  37. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) � End Book Flight (T 3 ) End Saga

  38. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) � End Saga

  39. Successful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) End Book Car Rental (T 2 ) Start Book Flight (T 3 ) End Book Flight (T 3 ) End Saga

  40. Unsuccessful Saga Backwards Recovery

  41. Unsuccessful Saga Begin Saga � Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  42. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) � End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  43. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) � Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  44. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) � Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  45. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga � Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  46. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) � End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  47. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) � Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  48. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) � End Compensate Book Hotel (C 1 ) End Saga

  49. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) � End Saga

  50. Unsuccessful Saga Begin Saga Start Book Hotel (T 1 ) End Book Hotel (T 1 ) Start Book Car Rental (T 2 ) Abort Saga Start Compensate Car Rental (C 2 ) End Compensate Car Rental (C 2 ) Start Compensate Book Hotel (C 1 ) End Compensate Book Hotel (C 1 ) End Saga

  51. Sagas in Distributed Systems “Due to space limitations, we only discuss Sagas in a centralized System, although clearly they can be implemented in a distributed database system.” –Molina et. al

  52. SOA/Microservices

  53. Requests instead of Transactions \ • Book Hotel (T 1 ) • Cancel Hotel (C 1 ) • Book Car (T 2 ) • Cancel Car (C 2 ) • Book Flight (T 3 ) • Cancel Flight (C 3 )

  54. A Distributed Saga is a Collection of Sub-Requests T 1 , T 2 … T n Each Sub-Request has a Compensating Request C 1 , C 2 … C n

  55. Successful Distributed Saga Begin Saga Start Book Hotel Request (T 1 ) End Book Hotel Request (T 1 ) Start Book Car Rental Request (T 2 ) End Book Car Rental Request (T 2 ) Start Book Flight Request (T 3 ) End Book Flight Request (T 3 ) End Saga

  56. Saga Log Durable & Distributed

  57. Saga Execution Coordinator (SEC) • Interprets & Writes to Saga Log • Applies Saga Sub-Requests • Applies Saga Compensating Requests when Necessary

  67. Apply Compensating Requests • Aborted Saga Response • Start Request Fails • SEC Crashes (non-safe state)

  71. What Happens when Compensating Requests Fail?

  72. Compensating Requests Must Be Idempotent

  73. What Happens when SEC Fails?

  74. Safe States • All Executed Sub-Requests are Complete (Start T i & End T i both logged) • Saga has been Aborted, Proceed with Compensating Transactions

  75. Un-Safe State • Start T i logged, no End T i logged Abort Saga Start Compensating Requests

  76. Request Messaging Semantics • Sub-Requests (T i ): At Most Once • Compensating Requests (C i ): At Least Once

  77. Distributed Saga Guarantee Either • T 1 , T 2 … T n or • T 1 , T 2 … T j, C j , … C 2 , C 1

  78. Distributed Sagas • Distributed/Durable Saga Log • SEC Process • Compensating Requests: Idempotent

  80. Halo Statistics

  81. Halo Statistics Service

  82. Halo Statistics Service with Sagas

  83. Forward-Recovery

  84. Game Grain / SEC Store Stats Store Stats Store Stats Store Stats Player One Player Two Player Three Player Four

  85. Forward Recovery Sub-Requests Must Also Be Idempotent

  86. Game Grain / SEC Store Stats Store Stats Store Stats Store Stats Player One Player Two Player Three Player Four

  87. Game Grain / SEC Store Stats Store Stats Store Stats Store Stats Player One Player Two Player Three Player Four

Recommend

Introduction to the SAGA API Outline SAGA Standardization API Structure and Scope (C++)

Introduction to the SAGA API Outline SAGA Standardization API Structure and Scope (C++)

Introduction to the SAGA API Outline SAGA Standardization API Structure and Scope (C++) API Walkthrough SAGA SoftwareComponents Command Line Tools Python API bindings C++ API bindings [ Java API bindings ] SAGA:

1.26k views • 87 slides
Interoperabilty: The SAGA Approach and Experience Shantenu Jha, Andre Merzky, Ole Weidner &

Interoperabilty: The SAGA Approach and Experience Shantenu Jha, Andre Merzky, Ole Weidner &

Interoperabilty: The SAGA Approach and Experience Shantenu Jha, Andre Merzky, Ole Weidner & * Collaborators http://saga.cct.lsu.edu Outline Introduction to SAGA: Why SAGA for Interoperability? Use of a standards-based approach for

390 views • 27 slides
C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS

C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS

C# Design Patterns: Proxy APPLYING THE PROXY PATTERN Steve Smith FORCE MULTIPLIER FOR DEV TEAMS @ardalis | ardalis.com | weeklydevtips.com t h s What problems does proxy solve? Objectives How is the proxy pattern structured? Apply the

779 views • 23 slides
Applying Architectural Patterns for the Cloud: Lessons Learned During Pattern Mining and

Applying Architectural Patterns for the Cloud: Lessons Learned During Pattern Mining and

Applying Architectural Patterns for the Cloud: Lessons Learned During Pattern Mining and Application Ralph Retter (Daimler TSS GmbH) ralph.retter@daimler.com Christoph Fehling (University of Stuttgart, Germany) Problem We need to do Cloud

282 views • 25 slides
Today you are going to: Watch a video Learn what a mini saga is and how to write one

Today you are going to: Watch a video Learn what a mini saga is and how to write one

Today you are going to: Watch a video Learn what a mini saga is and how to write one Generate ideas to write about Create a character, setting and plot Write a mini saga for Young Writers Ridiculous Writers mini saga

528 views • 13 slides
A Pattern Pattern Taxonomy Creational Behavioral Structural Pattern Pattern Pattern

A Pattern Pattern Taxonomy Creational Behavioral Structural Pattern Pattern Pattern

Podcast Ch08-15 Title : Template Design Pattern Description : Generic class diagram; examples; template vs. inheritance Participants : Barry Kurtz (instructor); Brandon Winters, Sara Hyde, Cheng Vue, Dan Baehr (students) Textbook

315 views • 3 slides
Projet Raw-data La Saga le dveloppement de cet t Le Rsultat Readers Analyst &

Projet Raw-data La Saga le dveloppement de cet t Le Rsultat Readers Analyst &

Projet Raw-data La Saga le dveloppement de cet t Le Rsultat Readers Analyst & Xcalibur 15/12/2008 La Saga La source : Les API des constructeurs L'interface : OLE/COM Le software : Dveloppement d'un reader

522 views • 26 slides
Building and Installing All material from this tutorial can be found at:

Building and Installing All material from this tutorial can be found at:

A Brief Introduction to SAGA Building and Installing All material from this tutorial can be found at: https://svn.cct.lsu.edu/repos/saga-projects/tutorial/EGI-2011 General Information and Documentation General information http://saga.

220 views • 9 slides
Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern

Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern

CSC209 Fall 2001 What is AWK? Awk, Awk Pattern matching and processing language Looks for pattern in file If pattern matches, do something Many details handled automatically very easy to one off (write and throw away)

673 views • 12 slides
saga PRODUCTION UNIT RAW MATERIAL PRODUCT APPLICATIONS PRODUCTION PROCESS BALES

saga PRODUCTION UNIT RAW MATERIAL PRODUCT APPLICATIONS PRODUCTION PROCESS BALES

SICEM SAGA CMP FOR PAPER INDUSTRY www. sicemsaga.com sicem saga PRODUCTION UNIT RAW MATERIAL PRODUCT APPLICATIONS PRODUCTION PROCESS BALES AND TRASPORT UNITS QUALITY CONTROL AND R&D ENERGY MANAGEMENT AND

746 views • 20 slides
Dr. Michelle Black Map of Japan I lived on Kyushu island in Saga Prefecture ( ,

Dr. Michelle Black Map of Japan I lived on Kyushu island in Saga Prefecture ( ,

Dr. Michelle Black Map of Japan I lived on Kyushu island in Saga Prefecture ( , Saga-ken ) for 3 years Saga Prefecture is a prefecture in the northwest part of the island of Kyushu, Japan. [ It touches both the Sea of Japan and

367 views • 36 slides
The SAGA Project: Satellites Around Milky Way-like Galaxies Marla Geha (Yale) Risa Wechsler

The SAGA Project: Satellites Around Milky Way-like Galaxies Marla Geha (Yale) Risa Wechsler

The SAGA Project: Satellites Around Milky Way-like Galaxies Marla Geha (Yale) Risa Wechsler (Stanford), Yao-Yuan Mao (UPitt/Rutgers), Erik Tollerud (STScI), Ben Weiner (UA), Nitya Kallivayalil (UA) and the SAGA team Satellites Around a Milky

335 views • 17 slides
Travel Insurance Niall Palmer Saga Insurance Overview About Saga Types of Travel

Travel Insurance Niall Palmer Saga Insurance Overview About Saga Types of Travel

Travel Insurance Niall Palmer Saga Insurance Overview About Saga Types of Travel Insurance What Travel Insurance covers you for Preparing to travel Where to buy Travel Insurance Dos and don'ts Case

662 views • 25 slides
Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern

Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern

Pattern Matching a b a c a a b 1 a b a c a b 4 3 2 a b a c a b Pattern Matching 1 Outline and Reading Strings (11.1) Pattern matching algorithms Brute-force algorithm (11.2.1) Boyer-Moore algorithm (11.2.2)

612 views • 14 slides
Pattern Review Pattern Name and Classification: A descriptive and unique name that helps in

Pattern Review Pattern Name and Classification: A descriptive and unique name that helps in

Pattern Review Pattern Name and Classification: A descriptive and unique name that helps in identifying and referring to the pattern. Intent: A description of the goal behind the pattern and the reason for using it. Also Known As: Other names for

352 views • 4 slides
Today you are going to: Watch a video Learn what a mini saga is and Learn what a

Today you are going to: Watch a video Learn what a mini saga is and Learn what a

Today you are going to: Watch a video Learn what a mini saga is and Learn what a mini saga is and is and how to write one is and how to write one Take part in a group discussion discussion Generate ideas to write about

604 views • 16 slides
Part 5 pattern recognition pattern recognition track pattern recognition: associate hits

Part 5 pattern recognition pattern recognition track pattern recognition: associate hits

Part 5 pattern recognition pattern recognition track pattern recognition: associate hits that belong to one particle nature or GEANT track finding + fitting will discuss concepts and some examples if you are interested in this,

631 views • 25 slides
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 1: INTRODUCTION Pattern Recogniton Pattern: Any

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 1: INTRODUCTION Pattern Recogniton Pattern: Any

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 1: INTRODUCTION Pattern Recogniton Pattern: Any regularity in data X Pattern Recognition: Discovery of any regularity in data, through computer algorithms and takes actions (Such as

1.11k views • 62 slides
KNOWLEDGE FACTORY User generated content in OpenCms WHO AM I? WHAT AM I DOING HERE? My name

KNOWLEDGE FACTORY User generated content in OpenCms WHO AM I? WHAT AM I DOING HERE? My name

KNOWLEDGE FACTORY User generated content in OpenCms WHO AM I? WHAT AM I DOING HERE? My name is Ramn Gavira, SAGA Soluciones CEO I come from Spain SAGA ERP Solutions Integration BAS ERP GEINPROD ERP PRIMAVERA BBS ERP

513 views • 35 slides
Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p

Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p

Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p 1 p n and text t = t 1 t m Output: All positions 1< i < ( m n + 1) such that the n - letter substring of t starting at i matches p

486 views • 24 slides
Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p

Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p

Exact Pattern Matching p t Goal: Find all occurrences of a pattern in a text Input: Pattern p = p 1 p n and text t = t 1 t m Output: All positions 1< i < ( m n + 1) such that the n - letter substring of t starting at i matches p

657 views • 19 slides
Tinker: The Saga Continues PC 03 0 A B 0 IR OUTPUT N 1 AC 001 Mode:

Tinker: The Saga Continues PC 03 0 A B 0 IR OUTPUT N 1 AC 001 Mode:

When last we left our heroine CPU Registers Process A Process B Process C Run Queue Tinker: The Saga Continues PC 03 0 A B 0 IR OUTPUT N 1 AC 001 Mode: USER Main Memory

127 views • 9 slides
The Singleton Pattern Design Patterns In Java Bob Tarr The Singleton Pattern The Singleton

The Singleton Pattern Design Patterns In Java Bob Tarr The Singleton Pattern The Singleton

The Singleton Pattern Design Patterns In Java Bob Tarr The Singleton Pattern The Singleton Pattern G Intent Ensure a class only has one instance, and provide a global point of access to it G Motivation Sometimes we want just a single

436 views • 9 slides
The SAGA of the french apeNEXTs To fight for decent computing power it sometimes needs

The SAGA of the french apeNEXTs To fight for decent computing power it sometimes needs

The SAGA of the french apeNEXTs To fight for decent computing power it sometimes needs Don Quixotes stubbornness and Sancho Panzas patience Don Quixote and Sancho Panza by Honor Daumier. Dulcinea del Toboso We discovered a

365 views • 21 slides

More recommend