Saga Pattern

In a monolithic system, a database transaction should ensure consistent query results after a successful request. Otherwise, it should rollback all the commits that were made to the previous state (Atomicity). Atomic transaction is common and easily achievable in a single database system.

However, in a distributed system you may need to handle this differently ensuring data consistency across multiple services and where each service have its own database. So traditional ACID approach wouldn't work.

Problem

For example, we have three services: Payment service, Invoicing service and Debtor service.

Imagine a debtor initiate a full payment to an invoice and we want to capture that transaction through the payment service then set the invoice status into "paid" and update the debtor outstanding balance.

Solution

One of the most common approach to achieve transaction that spans multiple services is by using the Saga Pattern.

A Saga is a sequence of local transactions and each transaction will publish messages or events that trigger the next local transaction.

If something goes wrong to a participating microservice, there should be a compensating transaction to undo the changes that were made by the preceding local transactions.

Now there are two popular Saga based approaches.

1. Choreography-based Saga
2. Orchestration-based Saga

Choreography-based Saga Example

It is when microservices publish a message/event from a local transaction and trigger subscribers or participating microservices for the next local transaction. See image below as an example that solves the given Problem:

1. The Payment Service will create a payment transaction and produce an Invoice Update Event.
2. The Invoice Service listens to this event to update the given invoice status to paid.
3. This will also produce Debtor Update Event signaling the Debtor Service for the next operation.
4. The Debtor Service listens to this event and do the outstanding calculation update.
5. It will then produce an Outstanding Deducted Event that will communicate back to the Payment Service.
6. The Payment Service receives an updated that a successful debtor outstanding balance was reflected ending the payment transaction flow.

Compensating (Rollback) Transaction

1. The Payment Service will create a payment transaction and produce an Invoice Update Event.
2. The Invoice Service listens to this event to update the given invoice status to paid.
3. This will also produce Debtor Update Event signaling the Debtor Service for the next operation.
4. The Debtor Service listens to this event and do the outstanding calculation update. However, this operation failed to update debtor outstanding balance for some reason e.g. a validation error.
5. The Debtor Service will trigger a new event called Invoice Revert Event to instruct the Invoice Service of the next operation.
6. The Invoice Service receives the event and will revert the invoice status back to created.
7. It will then produce a Payment Revert Event to signal the Payment Service.
8. The Payment Service receives the event and ending the transaction with a failed payment process.

Benefits and drawbacks

• No additional complexity for coordination logic.
• Very simple and easy to implement with smaller workflows.
• Hard to maintain when it grows.
• Cyclic dependency between participating services.
• Hard to tests as you will need to run all applications that are involved.

Orchestration-based Saga Example

It is when microservices have an orchestrator to command what participating microservices should trigger the next local transaction. The participating microservices should send a reply back to the local orchestrator before triggering the next service.

Below is the illustration of the workflow solution of the given Problem above.

1. Payment Service creates a payment transaction and triggers to start the Payment Service Orchestrator.
2. Payment Service Orchestrator produces an Invoice Update Event that will tell Invoice Service what to do.
3. The Invoice Service receives the event and updates the invoice status to paid.
4. This will produce a succesful invoice update response back to Payment Orchestration Event Reply.
5. Payment Service Orchestrator receives the event of a successful invoice update operation.
6. After a successful invoice status update, it will trigger a Debtor Update Event.
7. The Debtor Service receives the event to update a debtor outstanding balance.
8. And will produce a successful response of debtor outstanding update to the Payment Orchestration Event Reply.
9. This will signal Payment Service Orchestrator of a successful debtor update operation.
10. Then Payment Service Orchestrator will tell the Payment Service that a payment process was successful.

Compensating (Rollback) Transaction

1. Payment Service creates a payment transaction and triggers to start the Payment Service Orchestrator.
2. Payment Service Orchestrator produces an Invoice Update Event that will tell Invoice Service what to do.
3. The Invoice Service receives the event and updates the invoice status to paid.
4. This will produce a succesful invoice update response back to Payment Orchestration Event Reply.
5. Payment Service Orchestrator receives the event of a successful invoice update operation.
6. After a successful invoice status update, it will trigger a Debtor Update Event.
7. That will call Debtor Service to update a debtor outstanding balance. However, this operation failed to update debtor outstanding balance for some reason e.g. a validation error.
8. The Debtor Service will produce a response to Payment Orchestration Event Reply that a debtor update failed.
9. The Payment Orchestration Event Reply will signal Payment Service Orchestrator of the failed debtor update operation.
10. Then Payment Service Orchestrator will trigger a Invoice Revert Event.
11. To instruct Invoice Service to revert invoice status back to the initial state e.g. created.
12. Then the Invoice Service will return a response to the Payment Orchestration Event Reply of a successful invoice status revert.
13. This signals the Payment Service Orchestrator of a successful invoice state rollback.
14. And will tell Payment Service that a payment process did not push through and every state of data is still the same like nothing happened.

Benefits and drawbacks

• Complex saga orchestration implementation.
• No cyclic dependency.
• Distribution of transactions are centralized.
• Testable Orchestrator commands.
• Handle rollback transaction better.