I’ve been thinking a lot about how systems can stay responsive under load while keeping services decoupled. That’s what brought me to event-driven architectures. When services communicate through events rather than direct calls, they gain independence—scaling becomes easier, failures affect less, and development accelerates. Today I’ll show you how to build such a system using Spring Cloud Stream, Apache Kafka, and Testcontainers. Stick with me, and you’ll see how these tools work together to create resilient microservices.

First, let’s talk about our setup. We’re creating an order processing flow with three services: orders, inventory, and notifications. The order service emits events when orders are placed. The inventory service listens and adjusts stock. The notification service keeps users informed. All communicate via Kafka topics.

You’ll need Java 17+, Maven, and Docker installed. We’ll use Spring Boot as our foundation. Spring Cloud Stream abstracts messaging details, so we focus on business logic. Kafka handles message delivery. Testcontainers create real Kafka instances for tests.

We start with a shared module for events. Why? Because all services must understand the same event structures. Here’s our base event class:

// BaseEvent.java
public abstract class BaseEvent {
    private String eventId;
    @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss")
    private LocalDateTime timestamp;
    private String correlationId;

    protected BaseEvent() {
        this.eventId = UUID.randomUUID().toString();
        this.timestamp = LocalDateTime.now();
    }
    
    // Getters and setters...
}

Specific events like OrderCreatedEvent extend this. Now, the order service. It creates orders and publishes events. Notice how Spring Cloud Stream simplifies binding to Kafka:

// OrderChannels.java
public interface OrderChannels {
    String ORDERS_OUT = "orders-out";

    @Output(ORDERS_OUT)
    MessageChannel ordersOut();
}

When placing an order, we send an event:

// OrderController.java
@PostMapping("/orders")
public ResponseEntity<String> createOrder(@RequestBody Order order) {
    OrderCreatedEvent event = new OrderCreatedEvent(order.getId(), order.getItems());
    channels.ordersOut().send(MessageBuilder.withPayload(event).build());
    return ResponseEntity.accepted().body("Order processing started");
}

See how we return immediately? That’s event-driven: accept fast, process later. But what if inventory is low? The inventory service listens:

// InventoryListener.java
@StreamListener(InventoryChannels.ORDERS_IN)
public void handleOrderCreated(OrderCreatedEvent event) {
    try {
        inventory.updateStockFor(event.getItems());
        // Publish StockUpdatedEvent...
    } catch (InsufficientStockException e) {
        // Handle failure...
    }
}

When processing fails, we need recovery. Kafka’s dead-letter queues (DLQ) save failed messages:

# application.yml
spring:
  cloud:
    stream:
      bindings:
        orders-in:
          destination: orders
          group: inventory-group
          consumer:
            maxAttempts: 3
            backOffInitialInterval: 2000
            backOffMultiplier: 2.0

After three retries, messages move to a DLQ topic. We can monitor and reprocess them later.

Notifications work similarly. The service listens for multiple events:

// NotificationListener.java
@StreamListener("notifications-in")
public void notifyUser(StockUpdatedEvent event) {
    // Send stock update alert
}

@StreamListener("notifications-in")
public void notifyUser(OrderConfirmedEvent event) {
    // Send order confirmation
}

Now, how do we test this without mock headaches? Testcontainers run real Kafka in Docker:

// OrderServiceIntegrationTest.java
@SpringBootTest
@Testcontainers
class OrderServiceIntegrationTest {
    @Container
    static KafkaContainer kafka = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:7.0.0"));

    @DynamicPropertySource
    static void kafkaProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.kafka.bootstrap-servers", kafka::getBootstrapServers);
    }

    @Test
    void whenOrderCreated_thenEventPublished() {
        // Test logic using KafkaTemplate...
    }
}

This test starts a Kafka container, connects our app, and verifies event publishing. It’s production-like without complex setups.

For observability, we add tracing. Propagate a correlation ID in event headers:

MessageBuilder.withPayload(event)
    .setHeader("correlationId", correlationId)
    .build()

Then use Spring Cloud Sleuth to trace events across services. Monitor consumer lag with Kafka metrics. How? Expose them via Actuator and scrape with Prometheus.

In production, consider partitioning. Partition keys ensure order events go to the same partition:

spring:
  cloud:
    stream:
      bindings:
        orders-out:
          producer:
            partition-key-expression: payload.orderId

This preserves event order per order ID. Also, tune Kafka for durability: set acks=all and min.insync.replicas=2.

Common pitfalls? Serialization mismatches. Use shared schemas or schema registries. Also, handle duplicate events—make services idempotent. Check event IDs before processing.

Alternatives? RabbitMQ works instead of Kafka. Or use Spring Kafka directly. But Spring Cloud Stream’s abstraction simplifies setup. For complex workflows, consider sagas.

Event-driven architectures change how systems interact. Services become autonomous, systems more resilient. Have you tried similar patterns? What challenges did you face? I’d love to hear your experiences. If this guide helped, share it with others who might benefit.