Event Sourcing with Kafka: A Practical Guide for Backend Engineers

Event sourcing stores every state change as an immutable event. Instead of updating a row in place, you append an event to a log. The current state is derived by replaying events. Kafka, with its durable append-only log, is a natural fit for this pattern. But the gap between the concept and a production-ready implementation is wider than most tutorials suggest.

Why Event Sourcing?

Traditional CRUD systems store current state. Event sourcing stores the history of how you got there. This gives you:

• Complete audit trail: every change is recorded
• Temporal queries: reconstruct state at any point in time
• Decoupled read/write models: optimize reads and writes independently (CQRS)
• Event-driven integration: other services can consume the event stream

The tradeoffs are real: increased complexity, eventual consistency for read models, and operational overhead. I've worked on systems where event sourcing was the right call (a financial ledger where every state change needed to be auditable) and systems where it was premature complexity (a simple inventory tracker that would have been better served by a CRUD API with an audit log table).

Kafka as an Event Store

Kafka topics act as the event log. Each aggregate (e.g., an Order, an Account) gets its own partition key to ensure ordered delivery of events for that aggregate.

ProducerRecord<String, OrderEvent> record = new ProducerRecord<>(
    "order-events",
    order.getId(),
    new OrderCreatedEvent(order.getId(), order.getItems(), order.getTotal())
);
producer.send(record).get();

Key Design Decisions

• One topic per aggregate type vs. one topic for all events: per-aggregate topics are simpler to reason about. A single topic requires consumer-side filtering but makes it easier to build cross-aggregate projections.
• Partition key: use the aggregate ID so all events for one aggregate land on the same partition, preserving order.
• Retention: set retention to infinite (or very long) for event sourcing. Kafka's log.retention.ms = -1 keeps all events.
• Number of partitions: choose carefully. You can increase partitions later, but it breaks ordering guarantees for existing keys. Start with enough partitions to handle your peak throughput (one partition can handle ~1MB/s of writes).

Event Design

The quality of your events determines how maintainable the system is long-term. Some principles:

• Name events in past tense: OrderCreated, PaymentProcessed, InventoryReserved. Events represent facts that have already happened.
• Include enough context: each event should carry the data needed to understand the state change without querying other systems. If OrderCreated only contains an order ID, every consumer needs to call back to the order service for details. Include the relevant fields.
• Avoid command-style events: UpdateInventory is a command, not an event. InventoryReserved is an event. The distinction matters because events are facts (immutable, undeniable), while commands are requests (can be rejected).
• Version your events: include an eventVersion field from day one. When you need to add fields or change the schema, bump the version rather than modifying the existing structure.

public record OrderCreatedEvent(
    String eventId,
    String eventVersion,
    Instant occurredAt,
    String orderId,
    String customerId,
    List<OrderItem> items,
    BigDecimal total,
    String currency
) implements OrderEvent {}

Projections: Building Read Models

Events are the source of truth, but you don't query them directly. Projections consume the event stream and build read-optimized views:

@KafkaListener(topics = "order-events", groupId = "order-projection")
public void handle(OrderEvent event) {
    switch (event) {
        case OrderCreatedEvent e -> orderView.insert(e.orderId(), e.items(), e.total(), "CREATED");
        case OrderPaidEvent e -> orderView.updateStatus(e.orderId(), "PAID");
        case OrderShippedEvent e -> orderView.updateStatus(e.orderId(), "SHIPPED");
        case OrderCancelledEvent e -> orderView.updateStatus(e.orderId(), "CANCELLED");
    }
}

Projections are disposable: if the read model is corrupted or the schema changes, reset the consumer group offset and replay from the beginning.

Making Projections Idempotent

Kafka guarantees at-least-once delivery by default, so your projection consumers will process some events more than once. Every projection must handle duplicates gracefully (see Designing for Idempotency for a detailed treatment of deduplication strategies):

• Track the last processed event offset or sequence number per aggregate
• Use INSERT ... ON CONFLICT DO NOTHING or conditional updates
• For counter-based projections (e.g., "total revenue"), store per-event contributions rather than incrementing a running total

A projection that increments a counter on every event will produce incorrect results when events are replayed or redelivered. A projection that sets the counter to a value derived from the event's content will produce the correct result regardless of how many times the event is processed.

Projection Lag

There's always a delay between an event being written and the read model being updated. This is eventual consistency in action. For most use cases, the lag is measured in milliseconds to low seconds. But it creates a UX challenge: a user creates an order, then immediately views their order list, and the new order isn't there yet.

Common mitigations:

• Write-through cache: after publishing the event, also write to the read model in the same request. This is technically cheating, but it solves the immediate-read problem.
• Optimistic UI: the frontend assumes the write succeeded and shows the expected state without waiting for the read model.
• Polling with backoff: the client retries the read until the projection catches up.

Snapshots: Avoiding Full Replay

As the event count grows, replaying all events to reconstruct state becomes expensive. Snapshots periodically capture the aggregate's current state:

1. Every N events (e.g., every 100), save a snapshot
2. To load state: load the latest snapshot, then replay only events after it
3. Store snapshots in a separate store (database, S3) indexed by aggregate ID and sequence number

Snapshots are an optimization, not a replacement. The event log remains the source of truth. If a snapshot is corrupted, you can always delete it and rebuild from events. This is also useful during schema migrations: generate new snapshots from the event log after deploying new event-handling code.

Operational Realities

Schema Evolution

Events are immutable, but your schema will change. Options:

• Upcasting: transform old events to the new schema at read time. Simple but adds processing overhead and complexity to consumers.
• Versioned events: include a version field and handle each version explicitly. The consumer switches on the version and applies the appropriate logic.
• Avro/Protobuf with schema registry: enforce backward compatibility at the serialization layer. The schema registry rejects breaking changes at publish time, which is the strongest guarantee you can get.

In practice, I recommend Avro with a schema registry for any system beyond a prototype. It catches breaking changes before they reach production and makes cross-team event contracts explicit.

Compaction vs. Retention

Kafka log compaction keeps only the latest value per key, which conflicts with event sourcing (you need the full history). Use retention-based topics, not compacted topics, for event stores.

Exactly-Once Semantics

Kafka supports exactly-once semantics with idempotent producers and transactional writes. For event sourcing, this means:

• Enable enable.idempotence=true on the producer
• Use transactions when publishing multiple events atomically
• Consumers use read_committed isolation level

Debugging Event-Sourced Systems

When something goes wrong, the event log is your best debugging tool. You can reconstruct exactly what happened by replaying events for a specific aggregate:

kafka-console-consumer --bootstrap-server localhost:9092 \
  --topic order-events \
  --property print.key=true \
  --property key.separator="|" \
  | grep "ord-789"

This gives you a complete timeline of every state change for that order. No guessing, no "I think someone updated this record manually." The event log is an authoritative record.

Build tooling that makes this easy. A simple CLI or admin UI that takes an aggregate ID and shows the full event history (with pretty-printed payloads and timestamps) will save hours of debugging time.

When to Use Event Sourcing

• Audit-critical domains: finance, healthcare, compliance
• Complex business logic: where understanding "how we got here" matters
• CQRS architectures: when read and write models need different optimization
• Event-driven integration: when other services need to react to state changes

When Not to Use It

• Simple CRUD: the overhead isn't justified for straightforward data management
• Strong consistency requirements: read models are eventually consistent
• Small teams: the operational complexity requires engineering investment
• High-throughput, low-latency reads: the projection lag may be unacceptable without careful tuning

Key Takeaways

• Event sourcing stores state changes, not current state. Kafka's append-only log is a natural fit.
• Use one topic per aggregate type with the aggregate ID as partition key.
• Design events as immutable facts in past tense, with enough context for consumers to be self-sufficient.
• Build read models via projections. They're disposable and rebuildable.
• Make every projection idempotent; Kafka's at-least-once delivery means duplicates will happen.
• Use snapshots to avoid replaying the full event history.
• Plan for schema evolution from day one. Avro with a schema registry is the safest approach.
• Build tooling for event replay and aggregate inspection. Your future self will thank you.
• Event sourcing adds complexity. Use it when the audit trail, temporal queries, or decoupled models justify the investment.