Updated October 2026: We have expanded this glossary and updated it for Apache Kafka 4.x, including KRaft controllers, the new consumer rebalance protocol, share groups and tiered storage.
Kafka terminology can be confusing, especially when the same idea has two names, such as message and record. This page explains the terms you need to read Kafka documentation, configure clients and talk to your platform team, grouped by topic area.
What are the core Apache Kafka terms?
Term
Definition
Cluster
A group of Kafka servers that work together to store and serve data. Production clusters run from three to hundreds of brokers.
Broker
A single Kafka server that stores partitions and serves reads and writes from clients.
Topic
A named stream of records, logically similar to a table in a database. You group related data in a topic, for example orders or payments.
Partition
A topic is split into partitions so it can scale across brokers. Records within a partition are strictly ordered, but there is no ordering across partitions.
Record (message)
A single piece of data stored in Kafka, made up of an optional key, a value, a timestamp and optional headers. The words record and message mean the same thing.
Key
Part of a record used to choose its partition. Records with the same key go to the same partition, so they stay in order.
Header
Key-value metadata attached to a record, often used for tracing IDs or content types.
Offset
The position of a record within a partition. Offsets increase by one for each new record and never change.
Log
The ordered, append-only data structure that stores a partition’s records on disk.
Segment
A log is stored as a series of segment files. Kafka deletes or compacts whole segments, not single records.
What do Kafka replication terms mean?
Term
Definition
Replica
A copy of a partition stored on a broker. Kafka keeps several replicas on different brokers for fault tolerance and durability.
Replication factor
The number of replicas for each partition of a topic. Three is the common choice for production.
Leader
The replica that handles all writes for a partition. By default, consumers also read from the leader.
Follower
A replica that copies data from the leader. If the leader fails, an in-sync follower can be elected as the new leader.
In-sync replicas (ISR)
The set of replicas that are fully caught up with the leader. Only these replicas are eligible to become leader without data loss.
min.insync.replicas
The minimum number of in-sync replicas that must acknowledge a write when a producer uses acks=all. With a replication factor of 3, a value of 2 is common.
Controller
The component that manages cluster metadata, such as which broker leads each partition. In Kafka 4.x this is a quorum of controller nodes, with one active controller at a time.
KRaft
Kafka’s built-in metadata system, based on the Raft consensus protocol. It replaced ZooKeeper, which Kafka 4.0 removed.
What are producers, consumers and consumer groups?
Term
Definition
Producer
An application that uses a Kafka client library to write records to topics.
acks
A producer setting that controls how many replicas must confirm a write. With acks=all, the leader waits for all in-sync replicas.
Idempotent producer
A producer mode, on by default since Kafka 3.0, that prevents duplicate records when the producer retries a send.
Transactions
A way to write to several partitions atomically, used with consumer offsets for exactly-once processing.
Consumer
An application that uses a Kafka client library to read records from topics.
Consumer group
A set of consumers that share the work of reading a topic. Each partition is assigned to exactly one consumer in the group.
Group coordinator
The broker that manages a consumer group’s membership, partition assignments and committed offsets.
Rebalance
The process of reassigning partitions when consumers join or leave a group. Kafka 4.0 made the new consumer rebalance protocol (KIP-848) generally available, which makes rebalances incremental and broker-driven.
Committed offset
The offset a consumer group has saved for a partition, so it can resume from that point after a restart.
Consumer lag
The difference between the latest offset in a partition and the group’s committed offset. Rising lag means consumers are falling behind.
Share group
A newer group type for queue-style consumption, where several consumers can read from the same partition and acknowledge records one by one. It was introduced as early access in Kafka 4.0 (KIP-932).
What do Kafka storage and retention terms mean?
Term
Definition
Retention
How long, or how much data, Kafka keeps in a topic before deleting old segments. It is set per topic by time or by size.
Log compaction
A cleanup policy that keeps at least the latest record for each key instead of deleting by age. It suits changelog and state topics.
Tombstone
A record with a key and a null value. In a compacted topic, it marks the key for deletion.
Tiered storage
A feature that moves older log segments from broker disks to object storage, such as Amazon S3. It became production-ready in Kafka 3.9 (KIP-405).
What are the main Kafka ecosystem terms?
Term
Definition
Kafka Connect
A framework, part of Apache Kafka, for moving data between Kafka and external systems without writing custom code.
Connector
A Kafka Connect plugin. Source connectors bring data into Kafka and sink connectors send data out.
Kafka Streams
A Java library, part of Apache Kafka, for building stream processing applications that read from and write to Kafka.
Schema Registry
A separate service that stores record schemas, such as Avro, Protobuf or JSON Schema, and checks compatibility. It is not part of Apache Kafka: Confluent Schema Registry, Apicurio and Karapace are common options.
MirrorMaker 2
A tool built on Kafka Connect that replicates topics between Kafka clusters, often for disaster recovery or migration.
What are the key takeaways from this Kafka glossary?
Most Kafka concepts build on a few core terms. Once you understand topics, partitions, replicas, offsets and consumer groups, the rest of the documentation is much easier to follow.
Key takeaway: Records live in topics, topics are split into ordered partitions, and partitions are replicated across brokers with one leader each. Consumers in a group share partitions and track progress with committed offsets, while a KRaft controller quorum manages metadata in Kafka 4.x.
Frequently asked questions
What is the difference between a topic and a partition in Kafka?
A topic is a named stream of related records. A partition is one ordered slice of a topic. Kafka splits each topic into partitions so it can spread data and load across brokers and let several consumers read in parallel.
What is a broker in Kafka?
A broker is a single Kafka server. It stores partition replicas on disk and handles produce and fetch requests from clients. A Kafka cluster is a group of brokers working together.
What is an offset in Kafka?
An offset is the position of a record within a partition. Each new record gets the next offset. Consumer groups commit offsets so they can resume reading from the right place after a restart.
Is a Kafka message the same as a record?
Yes. Message and record mean the same thing in Kafka. Current documentation and client APIs mostly use record, for example ProducerRecord and ConsumerRecord.
Does Kafka still use ZooKeeper?
No. Apache Kafka 4.0 removed ZooKeeper. Kafka now stores cluster metadata in KRaft, a Raft-based quorum of controllers. Clusters that still use ZooKeeper must migrate to KRaft on Kafka 3.9 or earlier before upgrading.