I’ve spent the last few years building systems that need to handle thousands of requests per second, and I kept hitting walls with traditional architectures. That’s what led me to event-driven design with Apache Kafka and Spring Boot. Today, I want to share how you can build systems that scale effortlessly and handle failures gracefully. Why does this matter? Because in today’s world, users expect instant responses and zero downtime, and event-driven patterns deliver exactly that.
Let me show you how to get started. First, we need a running Kafka instance. I prefer using Docker Compose because it mirrors production setups. Here’s a configuration I use daily:
services:
zookeeper:
image: confluentinc/cp-zookeeper:7.4.0
ports: ["2181:2181"]
environment:
ZOOKEEPER_CLIENT_PORT: 2181
kafka:
image: confluentinc/cp-kafka:7.4.0
ports: ["9092:9092"]
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092Run docker-compose up -d, and you have Kafka ready. Ever wondered how to ensure your services don’t become tightly coupled? Events are the answer. In Spring Boot, start by adding Kafka dependencies to your pom.xml:
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>Now, let’s create an event producer. Imagine an order service where placing an order publishes an event. Here’s how I handle it:
@Service
public class OrderService {
@Autowired
private KafkaTemplate<String, Object> kafkaTemplate;
public void placeOrder(Order order) {
OrderEvent event = new OrderEvent(order.getId(), "CREATED", order.getDetails());
kafkaTemplate.send("orders-topic", event);
}
}What happens if a consumer fails to process an event? This is where dead letter queues save the day. In your consumer, you can configure retries and error handling:
@KafkaListener(topics = "orders-topic")
public void handleOrderEvent(OrderEvent event) {
try {
inventoryService.reserveItems(event.getOrderId());
} catch (Exception e) {
// Send to dead letter queue for manual review
kafkaTemplate.send("orders-dlt", event);
}
}Testing event-driven systems can be tricky. I use TestContainers to spin up a real Kafka instance in tests. Here’s a snippet from my test suite:
@Test
void shouldPublishOrderEvent() {
Order order = new Order("123", "Pending");
orderService.placeOrder(order);
// Verify event was sent
await().atMost(10, SECONDS).untilAsserted(() -> {
ConsumerRecord<String, OrderEvent> record =
kafkaConsumer.poll(Duration.ofMillis(100)).iterator().next();
assertThat(record.value().getStatus()).isEqualTo("CREATED");
});
}How do you monitor events in production? Spring Boot Actuator and Micrometer provide metrics out of the box. Expose them via Prometheus, and you can track event rates and consumer lag. I’ve found this essential for diagnosing performance issues before they affect users.
When configuring Kafka, pay attention to partitions and replication. More partitions allow higher throughput, but require careful planning. I typically start with 3 partitions per topic and adjust based on load. Remember, the key is to keep services independent. If one service goes down, others should continue processing events.
What patterns have I seen work best? Event sourcing combined with CQRS lets you rebuild state from events, making audits straightforward. For sagas, use choreographed events where each service reacts to previous steps. This avoids central coordination and reduces bottlenecks.
In my experience, the biggest pitfall is not planning for duplicate events. Make your consumers idempotent by checking if an event was already processed. Use database constraints or a dedicated idempotency store.
I hope this gives you a solid foundation. Building with events transformed how I design systems, making them more resilient and scalable. If you found this helpful, please like and share this article. I’d love to hear your experiences in the comments—what challenges have you faced with event-driven systems?
