Distributed Transactions with Sagas

You're building an e-commerce system. Placing an order needs to: (1) reserve inventory, (2) charge payment, (3) create shipment. Each operation is in a different microservice with its own database.

What if payment fails after inventory is reserved? You can't just rollback - there's no distributed transaction across services.

The Problem: No Distributed ACID

Enter the Saga Pattern

A Saga is a sequence of local transactions. Each step has a compensating transaction that undoes its effects if a later step fails.

Order Saga: Place Order
═══════════════════════════════════════════════════════════════════

Step 1: Reserve Inventory     ←──── Compensate: Release Inventory
           ↓
Step 2: Charge Payment        ←──── Compensate: Refund Payment
           ↓
Step 3: Create Shipment       ←──── Compensate: Cancel Shipment
           ↓
        SUCCESS!

If Step 2 fails:
───────────────────
Step 1: Reserve Inventory ✓
Step 2: Charge Payment ✗ (card declined)
  └─→ Run compensation for Step 1: Release Inventory
Result: System back to consistent state

Saga Execution: Orchestration vs Choreography

Orchestrator
     │
     ├──▶ Inventory: "Reserve"
     │         ↓ (done)
     ├──▶ Payment: "Charge"
     │         ↓ (done)
     └──▶ Shipping: "Create"

Order Created
     │
Inventory ──▶ InventoryReserved
     │
Payment ────▶ PaymentCharged
     │
Shipping ───▶ ShipmentCreated

Implementing an Orchestrated Saga

class PlaceOrderSaga {
  private steps: SagaStep[] = [
    {
      name: 'Reserve Inventory',
      execute: (ctx) => this.inventoryService.reserve(ctx.orderId, ctx.items),
      compensate: (ctx) => this.inventoryService.release(ctx.orderId)
    },
    {
      name: 'Charge Payment',
      execute: (ctx) => this.paymentService.charge(ctx.orderId, ctx.amount),
      compensate: (ctx) => this.paymentService.refund(ctx.orderId)
    },
    {
      name: 'Create Shipment',
      execute: (ctx) => this.shippingService.create(ctx.orderId, ctx.address),
      compensate: (ctx) => this.shippingService.cancel(ctx.orderId)
    }
  ];

  async execute(context: OrderContext): Promise<SagaResult> {
    const completedSteps: SagaStep[] = [];

    for (const step of this.steps) {
      try {
        await step.execute(context);
        completedSteps.push(step);
      } catch (error) {
        // Step failed! Run compensations in reverse order
        console.log(`Step "${step.name}" failed. Compensating...`);

        for (const completed of completedSteps.reverse()) {
          try {
            await completed.compensate(context);
          } catch (compError) {
            // Log and alert - compensation failed!
            console.error(`Compensation for "${completed.name}" failed!`);
          }
        }

        return { success: false, error };
      }
    }

    return { success: true };
  }
}

Implementing a Choreographed Saga

// Inventory Service
class InventoryService {
  @EventHandler('OrderCreated')
  async onOrderCreated(event: OrderCreated) {
    try {
      await this.reserveInventory(event.orderId, event.items);
      await this.publish(new InventoryReserved(event.orderId));
    } catch (error) {
      await this.publish(new InventoryReservationFailed(event.orderId, error));
    }
  }

  @EventHandler('PaymentFailed')
  async onPaymentFailed(event: PaymentFailed) {
    // Compensate: release the reserved inventory
    await this.releaseInventory(event.orderId);
    await this.publish(new InventoryReleased(event.orderId));
  }
}

// Payment Service
class PaymentService {
  @EventHandler('InventoryReserved')
  async onInventoryReserved(event: InventoryReserved) {
    try {
      await this.chargePayment(event.orderId);
      await this.publish(new PaymentCharged(event.orderId));
    } catch (error) {
      await this.publish(new PaymentFailed(event.orderId, error));
    }
  }
}

// Shipping Service
class ShippingService {
  @EventHandler('PaymentCharged')
  async onPaymentCharged(event: PaymentCharged) {
    await this.createShipment(event.orderId);
    await this.publish(new ShipmentCreated(event.orderId));
  }
}

Saga State Machine

Sagas often model as state machines, making failure handling explicit:

                                    ┌─────────────────┐
                    ┌───────────────│     STARTED     │
                    │               └────────┬────────┘
                    │                        │
                    │                        ▼
                    │               ┌─────────────────┐
    InventoryFailed │               │   RESERVING     │
                    │               │   INVENTORY     │
                    │               └────────┬────────┘
                    │                        │
                    ▼                        ▼ InventoryReserved
           ┌─────────────────┐      ┌─────────────────┐
           │     FAILED      │◀─────│    CHARGING     │
           │  (compensating) │      │    PAYMENT      │
           └─────────────────┘      └────────┬────────┘
                    ▲                        │
                    │ PaymentFailed          ▼ PaymentCharged
                    │               ┌─────────────────┐
                    └───────────────│    CREATING     │
                                    │    SHIPMENT     │
                                    └────────┬────────┘
                                             │
                                             ▼ ShipmentCreated
                                    ┌─────────────────┐
                                    │    COMPLETED    │
                                    └─────────────────┘

Handling Edge Cases

Idempotency is Critical

What if a message is delivered twice? Your handlers must be idempotent.

class PaymentService {
  async chargePayment(orderId: string, amount: number) {
    // Check if already charged (idempotency)
    const existing = await this.db.findPayment(orderId);
    if (existing) {
      console.log(`Payment for ${orderId} already exists, skipping`);
      return existing;
    }

    // Process payment
    const payment = await this.processPayment(orderId, amount);
    await this.db.savePayment(payment);
    return payment;
  }
}

Timeouts and Retries

Real-World: Uber's Saga Implementation

Saga vs Two-Phase Commit

Orchestration Frameworks

Key Takeaways

Next up: Assessment Quiz - Test your understanding of CQRS and Event Sourcing patterns.