A Consumer Group in Kafka is a collection of one or more consumer instances (processes/threads) that work together to consume messages from one or more topics.

• Each consumer in the group reads from a different partition of a topic.

• Together, the group ensures that each message in a partition is processed by only one consumer in that group.

👉 This provides scalability (parallel consumption) and fault tolerance.

---

🔹 Key Properties

1. Group ID

   • Each consumer group has a unique group.id.

   • Example: billing-consumers, analytics-consumers.

2. Partition Assignment

   • Kafka automatically divides partitions among consumers in the group.

   • If number of consumers > partitions → some consumers remain idle.

   • If consumers < partitions → some consumers process multiple partitions.

3. Offset Management

   • Each group tracks its offsets (position in partition).

   • Stored in Kafka’s internal topic __consumer_offsets.

   • Allows groups to restart from the last committed offset.

---

🔹 Example Scenario

Topic orders with 4 partitions:

P0   P1   P2   P3

Case 1: One Consumer (Group A)

• Consumer-1 reads all partitions (P0–P3).

Case 2: Two Consumers (Group A)

• Consumer-1 → P0 + P1

• Consumer-2 → P2 + P3

Case 3: Four Consumers (Group A)

• Each consumer reads exactly one partition.

Case 4: More Consumers than Partitions

• One or more consumers stay idle (not assigned any partition).

---

🔹 Multiple Groups Example

• Group A (Billing Service) consumes topic orders.

• Group B (Analytics Service) also consumes topic orders.

👉 Each group gets a full copy of the data → enabling fan-out to different applications.

---

🔹 Benefits of Consumer Groups

✅ Scalability → distribute load across multiple consumers.
✅ Fault tolerance → if a consumer crashes, partitions are reassigned to remaining consumers.
✅ Parallelism → multiple consumers process partitions in parallel.
✅ Isolation → multiple groups can consume the same topic independently.

---

🔹 In Short

A Consumer Group = a team of consumers with a shared group ID, consuming from a topic together:

• Each partition → assigned to only one consumer in the group.

• Offsets tracked per group.

• Enables scalable, fault-tolerant, parallel consumption.