Kafka terminology can be confusing! Here is everything you need to know on one page!
blog by OSO
Apache Kafka – Glossary
Kafka terminology can be confusing! Here is everything you need to know on one page!
| Term | Definition |
| Broker | One Kafka server. Kafka is a distributed system, so there’s a couple of these (eg. three to hundreds). |
| Controller | A special Kafka server that has cluster management responsibilities (eg. elect partition leaders). |
| Consumer group | Multiple consumer applications form a logical group which works together to split the work they have to do (read from different partitions). |
| Topic | A topic is basically what you group your data in – logically similar to a table in a database. |
| Partition | A topic gets sharded into multiple partitions. Each partition is ordered and is its own entity, part of the topic. Multiple producers & consumers can write/read from a topic. |
| Replica | Multiple brokers hold the exact same copy of a partition’s data for fault tolerance & durability. This is a replica. Partitions usually have three. |
| Leader | Out of the 3 replicas, you can write to only one – that is called the leader replica, or the broker that’s leading the partition. |
| Follower | The replicas that aren’t leaders are called followers. They follow the leader by replicating all its data. Followers are eligible to be promoted to a leader role. |
| Producer | An application using the Kafka Clients library to save messages into Kafka. |
| Consumer | An application using the Kafka Clients library to read messages into Kafka. |
| Message/Record | The data you store in Kafka via the Producer. The words message and record are used interchangeably. |
| Log | The simple ordered, immutable data structure used to store messages on the broker. |
| Offset | The unique index of the message, denoting its position in the log |
Have a conversation with one of our Kafka experts to discover how we can help.
CONTACT US