Kafka - ISR (In-Sync Replicas)

 In Kafka, ISR (In-Sync Replicas) refers to the set of replicas of a partition that are fully caught up with the leader replica. These replicas have the latest data and are considered reliable for maintaining consistency and fault tolerance.

 

🔑 Key Details About ISR in Kafka

• Replication in Kafka
  Each topic partition in Kafka can have multiple replicas across different brokers. Replication ensures that if one broker fails, another replica can take over without data loss.
• Leader and Followers
• One replica of a partition is elected as the leader.
• Other replicas act as followers, fetching data from the leader.
• Only replicas that are fully synchronized with the leader are part of the ISR set.
• Definition of ISR
• ISR = Leader + all followers that are caught up with the leader’s log.
• If a follower falls behind (due to network issues, slow performance, or broker failure), it is removed from the ISR until it catches up again.
• Why ISR Matters
• Data durability: Ensures that committed messages are stored on multiple brokers.
• High availability: If the leader fails, a new leader is elected from the ISR, guaranteeing no data loss.
• Consistency: Producers can be configured to wait for acknowledgments from all ISR replicas before considering a message "committed".
• Dynamic Membership
• ISR membership is dynamic. Replicas can join or leave the ISR depending on their synchronization status.
• Kafka continuously monitors replicas to maintain the ISR set.

 

⚙️ Example Scenario

Suppose you have a topic partition with 3 replicas:

• Broker 1 → Leader
• Broker 2 → Follower
• Broker 3 → Follower

If all brokers are caught up, ISR = {Broker 1, Broker 2, Broker 3}.
If Broker 3 lags behind, ISR = {Broker 1, Broker 2}.
If the leader (Broker 1) fails, Kafka elects a new leader from the ISR set (Broker 2 in this case).

 

📌 Best Practices

• Configure acks=all in producers to ensure messages are acknowledged by all ISR replicas before being considered committed.
• Monitor ISR shrinkage (replicas dropping out) as it may indicate performance or network issues.
• Use tools like Kafka’s metrics and monitoring dashboards to track ISR health for production reliability.

 

################################################################################

Here’s the cleanest and interview-ready explanation of Kafka ISR (In-Sync Replicas) 👇

---

✅ What is ISR in Kafka? (Simple Definition)

ISR (In-Sync Replicas) is the set of replicas (brokers) that are fully caught up with the leader’s log.
These replicas have all the committed messages and are synchronized with the leader.

---

🔥 Interview Definition

“ISR is the set of replicas that are fully caught up with the leader, meaning they contain all the messages the leader has committed. Only ISR members can become the next leader if the current leader fails.”

---

🧠 Why ISR Matters

• Provides fault tolerance

• Ensures high availability

• Guarantees no data loss, if acks=all

---

🧩 How ISR Works

1. A partition has:

• Leader replica (handles reads/writes)

• Follower replicas

2. If a follower keeps up with the leader →

➡ It stays in ISR

3. If a follower falls behind (slow, down, network issue) →

❌ Kafka removes it from ISR
(It becomes an OSR – Out-of-Sync Replica)

4. Only replicas in the ISR can be elected as leader if the current leader fails.

---

⚡ When does a replica drop out of ISR?

A follower is removed from ISR when it lags behind more than the configured threshold, mainly:

• replica.lag.max.messages

• replica.lag.time.max.ms

If it can’t catch up → Removed from ISR.

---

🏁 If producer uses acks=all

Kafka waits for all ISR replicas to acknowledge before considering the message committed.
This gives strong durability.

---

🎤 One-liner interview answer (Senior Level)

“ISR stands for In-Sync Replicas — the set of replicas that are fully caught up with the leader’s log and eligible for leader election. Kafka commits data only after all ISR replicas acknowledge it (when acks=all).”

---