Building Read Models with Projections

You have an event stream with thousands of OrderPlaced, OrderShipped, and OrderDelivered events. But your dashboard needs to show "orders by region this month" and your API needs to return "order details by ID". How do you serve these different queries efficiently?

What is a Projection?

A projection is a function that:

• Subscribes to events from the event stream
• Transforms each event into updates for a read model
• Maintains a denormalized, query-optimized view

Event Stream                    Projections                 Read Models
─────────────                   ───────────                 ───────────
│ OrderPlaced    │
│ PaymentReceived│─────────────▶ OrderDetails ────────────▶ [Order lookup by ID]
│ OrderShipped   │              Projection
│ OrderDelivered │
│ ...            │─────────────▶ SalesDashboard ──────────▶ [Revenue by region]
│                │              Projection
│                │
│                │─────────────▶ CustomerHistory ─────────▶ [Orders by customer]
                                Projection

Multiple Views, Same Events

{
  orderId: "ORD-123",
  customerName: "John",
  items: [...],
  status: "shipped",
  tracking: "1Z999"
}

{
  region: "US-West",
  month: "2024-01",
  orderCount: 1547,
  revenue: 234567.89
}

{
  customerId: "CUST-456",
  totalOrders: 23,
  lifetimeValue: 4567.89,
  recentOrders: [...]
}

Building a Projection

// The projection handler processes events and updates the read model
class OrderDetailsProjection {
  private readDb: ReadDatabase;

  // Handle each event type
  async handle(event: OrderEvent) {
    switch (event.type) {
      case 'OrderPlaced':
        await this.readDb.insert('order_details', {
          orderId: event.orderId,
          customerId: event.customerId,
          customerName: event.customerName,  // Denormalized!
          items: event.items,
          totalAmount: event.totalAmount,
          status: 'pending',
          placedAt: event.timestamp
        });
        break;

      case 'PaymentReceived':
        await this.readDb.update('order_details',
          { orderId: event.orderId },
          { status: 'paid', paidAt: event.timestamp }
        );
        break;

      case 'OrderShipped':
        await this.readDb.update('order_details',
          { orderId: event.orderId },
          {
            status: 'shipped',
            trackingNumber: event.trackingNumber,
            carrier: event.carrier,
            shippedAt: event.timestamp
          }
        );
        break;

      case 'OrderDelivered':
        await this.readDb.update('order_details',
          { orderId: event.orderId },
          { status: 'delivered', deliveredAt: event.timestamp }
        );
        break;
    }
  }
}

Projection Lifecycle

┌─────────────────────────────────────────────────────────────────────┐
│                     PROJECTION LIFECYCLE                            │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  1. SUBSCRIBE        2. PROCESS           3. STORE                  │
│  ───────────────     ──────────────       ─────────────             │
│  │ Event Stream │───▶│ Projection  │─────▶│ Read Model │           │
│  │              │    │ Handler     │      │ Database   │           │
│  └──────────────┘    └─────────────┘      └────────────┘           │
│                                                                     │
│  4. QUERY (Fast!)                                                   │
│  ─────────────────────────────────────────────────────              │
│  │ API Request │───▶│ Read Model │───▶│ Response │                 │
│  └─────────────┘    └────────────┘    └──────────┘                 │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

The Power of Rebuild

One of the most powerful features of projections: you can rebuild them from scratch.

class ProjectionRebuilder {
  async rebuild(projection: Projection) {
    // 1. Clear existing read model
    await projection.clear();

    // 2. Get all events from the beginning
    const events = await this.eventStore.getAllEvents();

    // 3. Process each event through the projection
    for (const event of events) {
      await projection.handle(event);
    }

    console.log(`Rebuilt projection with ${events.length} events`);
  }
}

Handling Eventual Consistency

Projections are eventually consistent - there's a delay between an event being written and the read model being updated.

Real-World: Event Store Subscriptions

// Using EventStoreDB's subscription API
const subscription = client.subscribeToAll({
  fromPosition: lastProcessedPosition
});

for await (const event of subscription) {
  // Route to appropriate projection(s)
  await orderDetailsProjection.handle(event);
  await salesDashboardProjection.handle(event);
  await customerHistoryProjection.handle(event);

  // Track position for restart recovery
  await saveCheckpoint(event.position);
}

Projection Patterns

Real-World: Netflix Billing

Key Takeaways

Next up: Event Store - How to persist and replay your events reliably.