Kafka # Consumer & Producer Problem

 🏗 Project Setup

Add Kafka dependencies in pom.xml:

<dependency>

    <groupId>org.springframework.kafka</groupId>

    <artifactId>spring-kafka</artifactId>

</dependency>

 

⚙️ Application Properties (application.yml)

Configure Kafka broker and topic:

spring:

  kafka:

    bootstrap-servers: localhost:9092

    producer:

      key-serializer: org.apache.kafka.common.serialization.StringSerializer

      value-serializer: org.apache.kafka.common.serialization.StringSerializer

    consumer:

      group-id: order-consumer-group

      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer

      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer

 

🚀 Producer Example

Here we send messages with orderId as the key. Kafka uses the key to decide the partition.

import org.springframework.kafka.core.KafkaTemplate;

import org.springframework.stereotype.Service;

 

@Service

public class OrderProducer {

 

    private final KafkaTemplate<String, String> kafkaTemplate;

 

    public OrderProducer(KafkaTemplate<String, String> kafkaTemplate) {

        this.kafkaTemplate = kafkaTemplate;

    }

 

    public void sendOrderEvent(String orderId, String event) {

        // orderId is the partition key

        kafkaTemplate.send("orders-topic", orderId, event);

        System.out.println("Sent event: " + event + " for orderId: " + orderId);

    }

}

 

📥 Consumer Example

Consumer will receive messages in order per partition.

import org.springframework.kafka.annotation.KafkaListener;

import org.springframework.stereotype.Service;

 

@Service

public class OrderConsumer {

 

    @KafkaListener(topics = "orders-topic", groupId = "order-consumer-group")

    public void consumeOrderEvent(String message) {

        System.out.println("Consumed message: " + message);

        // process sequentially per partition

    }

}

 

🔑 Key Points

• Partition key (orderId) ensures all events for the same order go to the same partition.
• Kafka guarantees ordering within a partition.
• Multiple consumers in the same group → each partition is assigned to exactly one consumer, so order is preserved per partition.
• If you need global ordering (all messages sequential), use a single partition (but that limits scalability).

 

🧩 Example Usage

// Somewhere in your service layer

orderProducer.sendOrderEvent("ORD123", "Order Created");

orderProducer.sendOrderEvent("ORD123", "Payment Processed");

orderProducer.sendOrderEvent("ORD123", "Order Shipped");

Consumer output:

Consumed message: Order Created

Consumed message: Payment Processed

Consumed message: Order Shipped

---