Diagnose Kafka Consumer Lag and a Stuck Partition

Start with the group and topic, then look at every partition. A total lag number hides the useful shape: one partition can be stalled while the others are nearly caught up.

Compare committed position, partition end, and owner

In the Kafka consumer group analyzer, take a snapshot and watch it for several samples. A committed offset is the next position the group intends to read. The lag shown here is the offset gap between that position and the current partition end. It is not always an exact count of live messages when retention, compaction, or aborted transactions leave gaps. The Kafka consumer design guide explains committed positions and group recovery.

Tell a slow consumer from a stuck one

If the commit advances but the end advances faster, the consumer is falling behind. Look at processing time, downstream dependencies, retry behavior, and partition distribution. If the commit stays unchanged while the end moves, inspect the next available record at the committed position and the previous retained record. A malformed or unexpected message may be a poison pill, but an unavailable database or a paused application can produce the same offset pattern.

The analyzer compares two snapshots and marks stalled or growing lag. Its Compare action reads nearby records through the local bridge. It does not claim to know the last message your application processed successfully.

Check who joined the consumer group

Group IDs are shared identifiers. Another deployment using the same ID can take some partitions. Enter expected client ID or host patterns, such as checkout-processor* or 10.4.*, and compare the group’s members and assignments. A mismatch is a lead: verify deployment metadata before blaming an unfamiliar IP alone.

Check assignment balance and in-sync replicas too. One member owning more partitions can be legitimate when topic subscriptions differ; missing in-sync replicas point to a separate broker or network issue.

Preview an offset reset before applying it

Stop the group’s consumers first. Select exact offset, earliest, latest, current, shift, UTC time, duration, or a CSV file. The preview shows the old and proposed offset for each selected partition. Apply only after reviewing the data that will be reprocessed or skipped. The bridge checks the group again, verifies that offsets have not changed since preview, and reports the offsets Kafka stores. Apache Kafka’s consumer group operations document the same reset scenarios.

Export the partition table before a change. That gives you a record of the previous positions if you need to explain the intervention later. Share a redacted snapshot for collaboration; member hosts and connection credentials are omitted from the link.