Building Event-Driven Microservices with Spring Boot, Apache Kafka, and Avro Schema Registry

I’ve been thinking about how modern systems handle constant change while maintaining reliability. Traditional request-response architectures often struggle with scalability and resilience. That’s why I’ve turned to event-driven microservices using Spring Boot, Apache Kafka, and Avro Schema Registry. This combination offers robust solutions for distributed systems that need to evolve without breaking.

Let’s start with our environment. We’ll use Docker Compose to spin up Kafka, Zookeeper, and Schema Registry in one command:

# docker-compose.yml
version: '3.8'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.4.0
    # ... (environment variables omitted for brevity)
  
  kafka:
    image: confluentinc/cp-kafka:7.4.0
    ports: ["9092:9092"]
    # ... (depends_on and environment config)

  schema-registry:
    image: confluentinc/cp-schema-registry:7.4.0
    ports: ["8081:8081"]
    # ... (dependency and listener config)

Run docker-compose up -d, and we have our foundation ready. Did you notice how this setup handles both message brokering and schema management?

For our e-commerce system, we need shared schemas. Avro provides strong typing and evolution capabilities. Here’s how we define an order event:

{
  "type": "record",
  "name": "OrderCreatedEvent",
  "fields": [
    {"name": "orderId", "type": "string"},
    {"name": "customerId", "type": "string"},
    {"name": "orderItems", "type": { /* array of items */ }},
    {"name": "totalAmount", "type": "double"},
    {"name": "timestamp", "type": "long", "logicalType": "timestamp-millis"}
  ]
}

Notice the logicalType for timestamps? This ensures consistent time handling across services.

In the Order Service, we configure Kafka producers with schema support:

@Configuration
public class KafkaProducerConfig {
  
  @Bean
  public ProducerFactory<String, Object> producerFactory() {
    Map<String, Object> props = new HashMap<>();
    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, KafkaAvroSerializer.class);
    props.put("schema.registry.url", "http://localhost:8081");
    return new DefaultKafkaProducerFactory<>(props);
  }
}

When creating an order, we publish events like this:

@Service
public class OrderService {
  
  @Autowired
  private KafkaTemplate<String, Object> kafkaTemplate;

  public void createOrder(Order order) {
    OrderCreatedEvent event = convertToEvent(order);
    kafkaTemplate.send("orders.topic", event.getOrderId(), event);
  }
}

How might we handle schema changes when adding new fields? The Schema Registry manages backward compatibility. Let’s say we add a paymentMethod field later. As long as we set default values, existing consumers won’t break:

{
  "name": "paymentMethod",
  "type": "string",
  "default": "CREDIT_CARD"
}

For the Inventory Service, we implement consumers with error handling:

@KafkaListener(topics = "orders.topic")
public void listen(OrderCreatedEvent event, 
                   @Header(KafkaHeaders.RECEIVED_KEY) String key) {
  try {
    reserveInventory(event);
  } catch (Exception ex) {
    handleFailure(event, ex); // Send to dead letter queue
  }
}

What happens when a service goes offline? Kafka’s offset management ensures no messages are lost. Consumers pick up where they left after restart.

We add resilience with retries and dead letter queues:

spring:
  kafka:
    listener:
      ack-mode: manual
    consumer:
      enable-auto-commit: false
    properties:
      max.poll.interval.ms: 300000

For monitoring, Spring Boot Actuator with Micrometer provides tracing:

@Bean
public MicrometerTracingCustomizer tracingCustomizer(MicrometerTracing tracing) {
  return (template) -> template.setObservationEnabled(true);
}

This traces events across services, showing call flows in tools like Zipkin.

Some key practices I’ve found essential:

• Always set schema compatibility to BACKWARD
• Use dead letter topics for unprocessable messages
• Monitor consumer lag with Kafka metrics
• Test schema evolution locally before deployment
• Separate public and internal event schemas

Implementing this pattern has transformed how I build systems. The loose coupling allows teams to deploy independently, while schema enforcement prevents data corruption.

What challenges have you faced with distributed systems? Share your experiences below! If this approach resonates with you, please like and share this with others who might benefit. Your comments help shape future content.