Outbox Pattern: Reliable Messaging in Distributed Systems

The Outbox Pattern solves a common problem: how do you update a database AND publish a message reliably? Without it, you risk losing messages or creating duplicates.

The Problem

// This is NOT reliable
await _db.SaveChangesAsync();  // What if this succeeds...
await _messageBus.PublishAsync(orderCreatedEvent);  // ...but this fails?

If the publish fails, your database has the order but no event was sent. Downstream systems never know about it.

The Solution

Write events to an outbox table in the same transaction as your data. A separate process reads and publishes them.

using var transaction = await _db.Database.BeginTransactionAsync();

// Save order
_db.Orders.Add(order);

// Save event to outbox (same transaction)
_db.OutboxMessages.Add(new OutboxMessage
{
    Id = Guid.NewGuid(),
    Type = "OrderCreated",
    Payload = JsonSerializer.Serialize(orderCreatedEvent),
    CreatedAt = DateTime.UtcNow
});

await _db.SaveChangesAsync();
await transaction.CommitAsync();

The Publisher

// Background service
public class OutboxPublisher : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken ct)
    {
        while (!ct.IsCancellationRequested)
        {
            var messages = await _db.OutboxMessages
                .Where(m => m.ProcessedAt == null)
                .OrderBy(m => m.CreatedAt)
                .Take(100)
                .ToListAsync();

            foreach (var message in messages)
            {
                await _messageBus.PublishAsync(message.Type, message.Payload);
                message.ProcessedAt = DateTime.UtcNow;
            }

            await _db.SaveChangesAsync();
            await Task.Delay(TimeSpan.FromSeconds(5), ct);
        }
    }
}

Why This Works

• Atomicity: Data and event saved in same transaction
• Durability: Events survive crashes (they’re in the DB)
• At-least-once: Publisher retries until confirmed

Considerations

• Consumers must be idempotent (handle duplicates)
• Clean up old processed messages
• Consider using libraries like MassTransit or NServiceBus

References

• Transactional Outbox Pattern