Event-Driven Architecture in Java with Apache Kafka enables scalable, loosely coupled systems by using events for asynchronous communication. 1. Producers publish events to topics, and consumers react without direct dependencies. 2. Kafka provides durable, partitioned logs with replayability and high throughput. 3. Use the kafka-clients library in Java with KafkaProducer and KafkaConsumer for manual control. 4. Producers send structured events (preferably JSON or Avro) to topics using ProducerRecord with callback handling. 5. Consumers poll messages in a loop, process them, and commit offsets, optionally using consumer groups for horizontal scaling. 6. Best practices include meaningful topic naming, event versioning, error handling, avoiding blocking operations, monitoring consumer lag, and ensuring idempotency. 7. Optionally, use Spring Kafka with @KafkaListener and KafkaTemplate for simplified, annotation-driven development. By starting with a single workflow like order processing and leveraging Kafka’s reliability, Java-based distributed systems can achieve real-time responsiveness, resilience, and maintainability through event-driven design.

Event-Driven Architecture (EDA) in Java with Apache Kafka is a powerful combination for building scalable, resilient, and loosely coupled systems. Instead of components calling each other directly, they communicate through events—something happened, and others react. Apache Kafka excels as the backbone for such architectures due to its high throughput, durability, and real-time processing capabilities.

Here’s how you can implement EDA in Java using Kafka effectively.

1. Core Concepts of Event-Driven Architecture

In EDA:

• Producers publish events (messages) to topics.
• Consumers subscribe to topics and react to events.
• Communication is asynchronous and decoupled.
• Components don’t need to know about each other—only about the events.

With Kafka:

• Topics are durable, partitioned logs.
• Events are stored for a configurable time, allowing replay.
• Multiple consumers can read the same event stream independently.

This makes Kafka ideal for microservices, real-time analytics, audit logging, and more.

2. Setting Up Kafka in a Java Project

Use Apache Kafka Clients library in your Maven or Gradle project.

Maven dependency:

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>3.7.0</version>
</dependency>

You’ll mainly use:

• KafkaProducer – to send events
• KafkaConsumer – to receive and process events

Also consider Spring Kafka if you're using Spring Boot—it simplifies configuration and annotation-driven development.

3. Producing Events in Java

To publish an event, create a KafkaProducer and send messages to a topic.

import org.apache.kafka.clients.producer.*;
import java.util.Properties;

public class OrderProducer {
    private final Producer<String, String> producer;

    public OrderProducer() {
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        this.producer = new KafkaProducer<>(props);
    }

    public void sendOrderEvent(String orderId, String status) {
        String topic = "order-events";
        ProducerRecord<String, String> record = 
            new ProducerRecord<>(topic, orderId, "Order "   orderId   " is "   status);

        producer.send(record, (metadata, exception) -> {
            if (exception != null) {
                System.err.println("Failed to send message: "   exception.getMessage());
            } else {
                System.out.println("Sent to partition "   metadata.partition()   
                                   " with offset "   metadata.offset());
            }
        });
    }

    public void close() {
        producer.close();
    }
}

Tip: Use JSON or Avro for structured events instead of plain strings in production.

4. Consuming Events with Kafka Consumer

Create a consumer that polls events from a topic and processes them.

import org.apache.kafka.clients.consumer.*;
import java.time.Duration;
import java.util.Collections;
import java.util.Properties;

public class OrderConsumer {
    private final KafkaConsumer<String, String> consumer;
    private final String topic = "order-events";

    public OrderConsumer() {
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("group.id", "order-processing-group");
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("auto.offset.reset", "earliest"); // or 'latest'

        this.consumer = new KafkaConsumer<>(props);
    }

    public void start() {
        consumer.subscribe(Collections.singletonList(topic));

        try {
            while (true) {
                ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(1000));
                for (ConsumerRecord<String, String> record : records) {
                    System.out.printf("Received: key=%s, value=%s, partition=%d, offset=%d%n",
                            record.key(), record.value(), record.partition(), record.offset());

                    // Process the event
                    processOrderEvent(record.value());
                }
                // Commit offset manually if needed
                consumer.commitSync();
            }
        } finally {
            consumer.close();
        }
    }

    private void processOrderEvent(String value) {
        // Business logic: update DB, notify user, trigger next step, etc.
        System.out.println("Processing event: "   value);
    }
}

Use consumer groups to scale horizontally—each instance in a group reads from different partitions.

5. Best Practices for EDA with Kafka in Java

• Use meaningful topic names: e.g., user-signed-up, payment-failed, inventory-updated.
• Version your events: Include schema version in event payload or use Confluent Schema Registry.
• Handle errors gracefully: Don’t crash on bad messages—log, retry, or send to a dead-letter topic.
• Avoid long blocking operations in consumers: Offload work to threads or message queues if needed.
• Monitor lag: Track consumer group lag to detect processing delays.
• Use idempotent consumers: Ensure processing an event twice doesn’t cause side effects.

6. Optional: Use Spring Kafka for Simplicity

With Spring Boot, you can use annotations like @KafkaListener and @SendTo.

@Service
public class KafkaConsumerService {

    @KafkaListener(topics = "order-events", groupId = "order-group")
    public void listen(String message) {
        System.out.println("Received via Spring Kafka: "   message);
        // Handle event
    }
}

And produce easily:

@Autowired
private KafkaTemplate<String, String> kafkaTemplate;

public void sendEvent(String topic, String key, String payload) {
    kafkaTemplate.send(topic, key, payload);
}

Spring Kafka handles serialization, threading, and error handling out of the box.

Summary

Building an event-driven system in Java with Apache Kafka lets you:

• Decouple services
• Scale independently
• React in real time
• Replay events for recovery or analytics

Start small: pick one workflow (e.g., order processing), model it as events, and connect producers and consumers. Then expand as needed.

With Kafka’s reliability and Java’s ecosystem, EDA becomes not just feasible, but maintainable and performant at scale.

Basically, if you're building distributed systems in Java, Kafka is one of the best tools to go event-driven.

The above is the detailed content of Event-Driven Architecture in Java with Apache Kafka. For more information, please follow other related articles on the PHP Chinese website!