Updated October 2026: We wrote this post when KRaft was new and running Kafka without ZooKeeper was optional. Kafka 4.0 removed ZooKeeper mode entirely, so we have updated it against the current Apache Kafka documentation. We corrected config and tool names, the replica placement behaviour, the migration steps and what metadata downgrades are supported.
Running Kafka without ZooKeeper is no longer a future option. It is the only way to run current Kafka releases. This guide covers what changes when ZooKeeper goes away, how KRaft handles replica placement and version skew, and how to troubleshoot and monitor a KRaft cluster in production.
Why does Kafka run without ZooKeeper now?
For most of its life, Kafka stored its cluster metadata in Apache ZooKeeper. ZooKeeper keeps data in znodes, arranged in a filesystem-like tree. Each broker found the ensemble through the zookeeper.connect setting and registered itself under a znode named after its unique broker.id. A single controller broker read metadata from ZooKeeper and pushed it to the other brokers.
KRaft (KIP-500) replaces that design. Metadata is stored as an event log in an internal topic, __cluster_metadata, and replicated by a quorum of controllers using the Raft consensus protocol. Brokers fetch updates from the log instead of waiting for the controller to push them.
Kafka version
ZooKeeper status
2.8
KRaft available as early access
3.3
KRaft production-ready for new clusters
3.6 to 3.9
ZooKeeper to KRaft migration available (3.9 is the last bridge release)
4.0 and later
ZooKeeper mode removed: KRaft only
How does KRaft change replica placement?
In ZooKeeper mode, a broker that is down loses its registration, so the controller cannot place replicas on it. In a three-broker cluster, that meant you could not create a topic with a replication factor of 3 while one broker was offline, for example during a rolling restart. Many teams ran four brokers to avoid this.
In KRaft mode, a broker that stops heartbeating is fenced rather than removed. The controller still prefers unfenced brokers, but it can place follower replicas on a fenced broker when there is no other choice. The leader of a new partition must always be an unfenced broker. As a result, a three-broker cluster can keep creating topics with a replication factor of 3 during a roll, so you do not need a fourth broker just for that.
How do you upgrade from ZooKeeper mode to KRaft mode?
Kafka 4.x cannot run in ZooKeeper mode, so a ZooKeeper cluster must migrate to KRaft on a bridge release before it can upgrade. The last bridge release is Kafka 3.9. The migration (KIP-866) works like this:
Upgrade to a bridge release. Move the cluster to Kafka 3.9, still in ZooKeeper mode.
Add KRaft controllers. Provision a new controller quorum with migration enabled and the ZooKeeper connection configured.
Roll the brokers with migration enabled. Each broker registers with the KRaft controllers. Once every broker has registered, the active KRaft controller takes over as the cluster controller and copies the metadata from ZooKeeper into the metadata log.
Run in dual-write mode. The KRaft controller now makes all metadata changes. It writes them to the KRaft log and also to ZooKeeper, and it sends the legacy controller requests to brokers that are still in ZooKeeper mode.
Roll the brokers into KRaft mode. Restart each broker as a KRaft broker without the ZooKeeper settings. At this point you can still roll back to ZooKeeper.
Finalise the migration. Roll the controllers with migration disabled. After this step, rollback to ZooKeeper is no longer possible.
Decommission ZooKeeper. Once the cluster runs entirely on KRaft, shut down the ensemble and then upgrade to Kafka 4.x.
During a rolling upgrade, some brokers run new software while others still run the old version. Kafka must not use a new RPC or record format until every node supports it. In ZooKeeper mode, the inter.broker.protocol.version setting controlled this, which had drawbacks:
Manual configuration: it was set by hand in each broker’s config, so it was easy to forget or get wrong.
Double roll: you upgraded the software on every broker, then changed the setting and rolled every broker again.
No supported downgrade: once raised, it could not be safely lowered.
KRaft replaces it with the metadata.version feature level, part of the cluster’s finalised features. You change it at runtime with kafka-features.sh, without a second roll, and the controller refuses an upgrade if any broker does not support the new level. For example, after rolling all nodes to Kafka 4.0:
KIP-778 designed both safe (lossless) and unsafe (lossy) metadata downgrades, but unsafe downgrades are not implemented. A downgrade only works when no metadata changes exist between the current and target versions. The Kafka 4.0 upgrade notes say a metadata downgrade from 4.0 is not supported, so test upgrades in a non-production cluster first.
How do you troubleshoot a Kafka cluster in KRaft mode?
In KRaft mode, the __cluster_metadata log is the source of truth, so you troubleshoot the log instead of ZooKeeper. Kafka ships three tools for this:
Tool
What it does
kafka-metadata-quorum.sh
Shows the quorum’s leader, epoch, high watermark, voters, observers and per-replica lag
kafka-dump-log.sh
Decodes metadata log segments and snapshots into readable records
kafka-metadata-shell.sh
Opens a metadata snapshot as a virtual filesystem you browse with ls and cat
The dump log tool needs the cluster metadata decoder to print records such as broker registrations, topic records and leader changes:
Start with the quorum tool. It calls the DescribeQuorum API on the controllers and reports the leader ID, leader epoch, high watermark and how far each voter and observer lags:
Then alert on the controller and Raft metrics over time:
What to watch
Why it matters
Active controller count
Exactly one controller in the cluster should be active
Raft state, leader and epoch
Frequent leader or epoch changes point to an unstable quorum
Metadata log end offset and high watermark
Shows whether the quorum is committing new metadata
Last applied record offset on brokers
A broker that falls behind is serving stale metadata
Metadata error counts
Any error loading or applying metadata needs investigation
For redundancy, run 3 controllers to tolerate one failure, or 5 to tolerate two. Use dedicated controller nodes in production: the documentation recommends against combined broker and controller mode for critical environments.
What are the key takeaways for running Kafka without ZooKeeper?
Kafka’s move to KRaft brings simpler operations, faster controller failover and better scalability, and it is now mandatory. Plan the ZooKeeper to KRaft migration on Kafka 3.9, then upgrade to 4.x and operate the cluster through the metadata log, feature levels and quorum tools.
Key takeaway: Kafka 4.0 removed ZooKeeper, so every cluster must run in KRaft mode. Migrate on the Kafka 3.9 bridge release, run 3 or 5 dedicated controllers, raise metadata.version with kafka-features.sh after each upgrade, and monitor the quorum with kafka-metadata-quorum.sh.
Frequently asked questions
Can Kafka run without ZooKeeper?
Yes. Kafka has been able to run without ZooKeeper in KRaft mode since version 2.8, and KRaft became production-ready in Kafka 3.3. From Kafka 4.0, ZooKeeper mode has been removed, so every cluster runs without ZooKeeper.
Which Kafka version removed ZooKeeper?
Apache Kafka 4.0, released in March 2025, removed ZooKeeper mode. Kafka 3.9 is the last release that supports ZooKeeper and the last bridge release for migrating a ZooKeeper cluster to KRaft.
Can I upgrade a ZooKeeper-based cluster straight to Kafka 4.0?
No. You must first migrate the cluster to KRaft mode on a bridge release, ideally Kafka 3.9. Once the migration is finalised and ZooKeeper is decommissioned, you can upgrade the KRaft cluster to Kafka 4.x.
How many KRaft controllers do I need?
Most production clusters run 3 controllers, which tolerate one failure, or 5, which tolerate two. A majority of controllers must be available, and dedicated controller nodes are recommended over combined broker and controller mode.
Can I downgrade metadata.version in KRaft mode?
Only in limited cases. A downgrade is possible only when there are no metadata changes between the current and target versions, and unsafe lossy downgrades are not implemented. Kafka 4.0 does not support a metadata downgrade, so test upgrades before you finalise them.
