blog by OSO

Apache Kafka – Glossary

Sion Smith
Sion Smith 16 January 2025
OSO Apache Kafka glossary of key terms from brokers to KRaft, with white title text on a navy panel and a lime Blogs label.

Updated October 2026: We have expanded this glossary and updated it for Apache Kafka 4.x, including KRaft controllers, the new consumer rebalance protocol, share groups and tiered storage.

Kafka terminology can be confusing, especially when the same idea has two names, such as message and record. This page explains the terms you need to read Kafka documentation, configure clients and talk to your platform team, grouped by topic area.

What are the core Apache Kafka terms?

TermDefinition
ClusterA group of Kafka servers that work together to store and serve data. Production clusters run from three to hundreds of brokers.
BrokerA single Kafka server that stores partitions and serves reads and writes from clients.
TopicA named stream of records, logically similar to a table in a database. You group related data in a topic, for example orders or payments.
PartitionA topic is split into partitions so it can scale across brokers. Records within a partition are strictly ordered, but there is no ordering across partitions.
Record (message)A single piece of data stored in Kafka, made up of an optional key, a value, a timestamp and optional headers. The words record and message mean the same thing.
KeyPart of a record used to choose its partition. Records with the same key go to the same partition, so they stay in order.
HeaderKey-value metadata attached to a record, often used for tracing IDs or content types.
OffsetThe position of a record within a partition. Offsets increase by one for each new record and never change.
LogThe ordered, append-only data structure that stores a partition’s records on disk.
SegmentA log is stored as a series of segment files. Kafka deletes or compacts whole segments, not single records.

What do Kafka replication terms mean?

TermDefinition
ReplicaA copy of a partition stored on a broker. Kafka keeps several replicas on different brokers for fault tolerance and durability.
Replication factorThe number of replicas for each partition of a topic. Three is the common choice for production.
LeaderThe replica that handles all writes for a partition. By default, consumers also read from the leader.
FollowerA replica that copies data from the leader. If the leader fails, an in-sync follower can be elected as the new leader.
In-sync replicas (ISR)The set of replicas that are fully caught up with the leader. Only these replicas are eligible to become leader without data loss.
min.insync.replicasThe minimum number of in-sync replicas that must acknowledge a write when a producer uses acks=all. With a replication factor of 3, a value of 2 is common.
ControllerThe component that manages cluster metadata, such as which broker leads each partition. In Kafka 4.x this is a quorum of controller nodes, with one active controller at a time.
KRaftKafka’s built-in metadata system, based on the Raft consensus protocol. It replaced ZooKeeper, which Kafka 4.0 removed.

To learn more about running KRaft in production, read our guide to running Kafka without ZooKeeper.

What are producers, consumers and consumer groups?

TermDefinition
ProducerAn application that uses a Kafka client library to write records to topics.
acksA producer setting that controls how many replicas must confirm a write. With acks=all, the leader waits for all in-sync replicas.
Idempotent producerA producer mode, on by default since Kafka 3.0, that prevents duplicate records when the producer retries a send.
TransactionsA way to write to several partitions atomically, used with consumer offsets for exactly-once processing.
ConsumerAn application that uses a Kafka client library to read records from topics.
Consumer groupA set of consumers that share the work of reading a topic. Each partition is assigned to exactly one consumer in the group.
Group coordinatorThe broker that manages a consumer group’s membership, partition assignments and committed offsets.
RebalanceThe process of reassigning partitions when consumers join or leave a group. Kafka 4.0 made the new consumer rebalance protocol (KIP-848) generally available, which makes rebalances incremental and broker-driven.
Committed offsetThe offset a consumer group has saved for a partition, so it can resume from that point after a restart.
Consumer lagThe difference between the latest offset in a partition and the group’s committed offset. Rising lag means consumers are falling behind.
Share groupA newer group type for queue-style consumption, where several consumers can read from the same partition and acknowledge records one by one. It was introduced as early access in Kafka 4.0 (KIP-932).

What do Kafka storage and retention terms mean?

TermDefinition
RetentionHow long, or how much data, Kafka keeps in a topic before deleting old segments. It is set per topic by time or by size.
Log compactionA cleanup policy that keeps at least the latest record for each key instead of deleting by age. It suits changelog and state topics.
TombstoneA record with a key and a null value. In a compacted topic, it marks the key for deletion.
Tiered storageA feature that moves older log segments from broker disks to object storage, such as Amazon S3. It became production-ready in Kafka 3.9 (KIP-405).

What are the main Kafka ecosystem terms?

TermDefinition
Kafka ConnectA framework, part of Apache Kafka, for moving data between Kafka and external systems without writing custom code.
ConnectorA Kafka Connect plugin. Source connectors bring data into Kafka and sink connectors send data out.
Kafka StreamsA Java library, part of Apache Kafka, for building stream processing applications that read from and write to Kafka.
Schema RegistryA separate service that stores record schemas, such as Avro, Protobuf or JSON Schema, and checks compatibility. It is not part of Apache Kafka: Confluent Schema Registry, Apicurio and Karapace are common options.
MirrorMaker 2A tool built on Kafka Connect that replicates topics between Kafka clusters, often for disaster recovery or migration.

New to Kafka? Start with our introduction to what Apache Kafka is.

What are the key takeaways from this Kafka glossary?

Most Kafka concepts build on a few core terms. Once you understand topics, partitions, replicas, offsets and consumer groups, the rest of the documentation is much easier to follow.

Key takeaway: Records live in topics, topics are split into ordered partitions, and partitions are replicated across brokers with one leader each. Consumers in a group share partitions and track progress with committed offsets, while a KRaft controller quorum manages metadata in Kafka 4.x.

Frequently asked questions

What is the difference between a topic and a partition in Kafka?

A topic is a named stream of related records. A partition is one ordered slice of a topic. Kafka splits each topic into partitions so it can spread data and load across brokers and let several consumers read in parallel.

What is a broker in Kafka?

A broker is a single Kafka server. It stores partition replicas on disk and handles produce and fetch requests from clients. A Kafka cluster is a group of brokers working together.

What is an offset in Kafka?

An offset is the position of a record within a partition. Each new record gets the next offset. Consumer groups commit offsets so they can resume reading from the right place after a restart.

Is a Kafka message the same as a record?

Yes. Message and record mean the same thing in Kafka. Current documentation and client APIs mostly use record, for example ProducerRecord and ConsumerRecord.

Does Kafka still use ZooKeeper?

No. Apache Kafka 4.0 removed ZooKeeper. Kafka now stores cluster metadata in KRaft, a Raft-based quorum of controllers. Clusters that still use ZooKeeper must migrate to KRaft on Kafka 3.9 or earlier before upgrading.

Get expert help with Kafka

Talk to OSO's Kafka engineers about designing, running and upgrading Apache Kafka in production.

CONTACT US
OSO
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.