Kafka Cruise Control, originally developed at LinkedIn, is a powerful open-source tool that helps manage and optimise Apache Kafka clusters. We have spent hundreds of hours building, configuring and running Cruise Control for clients, and in this guide we share the key features we rely on and how they can benefit your Kafka deployment.
The Cruise Control UI gives a single view of broker state, offline partitions and under-replicated partitions.
Updated October 2026: the project now lives at github.com/cruise-control-for-kafka/cruise-control (the old linkedin/cruise-control URLs redirect there). The main branch (2.5.x releases) supports Apache Kafka 2.5 and later, including KRaft-only Kafka 4.0 from release 2.5.144 and Kafka 4.3 from 2.5.146, and requires Java 17. Configuration names and security options below have been checked against the project’s wiki.
Feature
What it does
Where it is configured
Provisioner
Checks whether the cluster is under- or over-provisioned and can rightsize brokers or partitions
POST /kafkacruisecontrol/rightsize, pluggable provisioner class
Concurrency adjuster
Raises or lowers the number of concurrent partition and leadership movements during a rebalance based on broker health
concurrency.adjuster.* settings (off by default)
Goal-based rebalancing
Moves replicas and leaders to satisfy hard and soft goals such as rack awareness and capacity
rebalance endpoint and goal configuration
Security
Authentication, HTTPS and role-based authorisation for the REST API
webserver.security. and webserver.ssl. settings
How does Cruise Control’s Provisioner allocate resources?
When adding a new broker to your Kafka cluster, it is important to consider how resources will be allocated. Cruise Control offers a pluggable framework called the Provisioner, which allows you to define how resources should be allocated. This includes determining whether your cluster is under-provisioned or over-provisioned.
Under-provisioning means that your cluster lacks certain resources, while over-provisioning means that you are wasting resources. The Provisioner can help you identify these issues and take appropriate actions, such as adding or removing brokers or changing partition counts. The rightsize endpoint accepts a target broker count difference (num_brokers_to_add) or a target partition_count.
While Cruise Control provides an API for resource allocation, it does not ship an implementation that adds brokers on specific cloud providers such as AWS or Azure. You can write your own glue code that implements the Provisioner interface for your chosen provider and use the Provisioner API effectively.
How does the Concurrency Adjuster automate rebalancing?
Setting manual limits for rebalancing Kafka clusters can be challenging, especially when traffic fluctuates throughout the day. If the limit is set too high, it can overwhelm the cluster during traffic spikes. If it is set too low, rebalances can take so long that they never seem to end.
Cruise Control addresses this with its Concurrency Adjuster. During an ongoing reassignment, it compares the 99.9th percentile of broker metrics with configurable limits: log flush time, follower fetch local time, produce local time, consumer fetch local time and request queue size. It uses an additive increase, multiplicative decrease (AIMD) algorithm: while every metric is within its limit, it raises the number of allowed concurrent movements by a fixed step, and as soon as any broker exceeds a limit, it divides the cap (by 2 by default).
Setting
Default
Purpose
concurrency.adjuster.inter.broker.replica.enabled
false
Turn the adjuster on for inter-broker replica movements
Upper cap on concurrent partition movements per broker
concurrency.adjuster.limit.log.flush.time.ms
2000
99.9th percentile log flush time above which concurrency is reduced
concurrency.adjuster.limit.produce.local.time.ms
1000
99.9th percentile produce local time above which concurrency is reduced
The adjuster is disabled by default, so enable it explicitly in cruisecontrol.properties or per request with the enable_concurrency_adjuster_for parameter. This automated approach gives faster rebalances without causing latency spikes in the cluster, and it reduces the need for manual intervention because Cruise Control keeps adjusting the limits based on the observed metrics.
Rebalances can be run from the UI or REST API, with a dry run first to review the proposed movements.
How does Cruise Control secure its REST API?
Since the Log4j vulnerability incident, security has become a top concern for many organisations. Cruise Control’s security features protect its REST API, which can move data around your cluster. Authentication is enabled with webserver.security.enable=true and a pluggable webserver.security.provider:
HTTP Basic (the default BasicSecurityProvider): credentials and roles are read from a file set by webserver.auth.credentials.file. Use it together with HTTPS.
JSON Web Token (JWT): Cruise Control redirects users to a token provider such as Apache Knox and validates the token it issues.
SPNEGO: Kerberos-based authentication over HTTP, which suits services that already authenticate with Kerberos.
Trusted proxy: built on SPNEGO. A trusted service such as Apache Knox authenticates to Cruise Control and passes the real end user in the doAs HTTP parameter, so each request runs with that user’s privileges.
HTTPS for the web server, which is powered by Jetty, is configured with the webserver.ssl.* settings (keystore location, password and type). Cruise Control can also connect securely to Kafka and, for ZooKeeper-based clusters, to ZooKeeper over TLS.
For authorisation, Cruise Control defines three default roles:
Role
Access
VIEWER
The lightweight kafka_cluster_state, user_tasks and review_board endpoints
USER
All GET endpoints except bootstrap and train
ADMIN
All endpoints, including the POST endpoints that rebalance, add, remove or demote brokers
You can define custom roles by extending the SecurityProvider interface.
How do you manage multiple Kafka clusters with Cruise Control?
Each Cruise Control instance manages a single Kafka cluster, so run one instance per cluster, with its own capacity configuration and goals. The Cruise Control UI can then be configured to show several clusters, as in the cluster selector at the top of the screenshot above.
Kafka Cruise Control is a valuable tool for managing and optimising Kafka clusters. Its features, such as the Provisioner, the Concurrency Adjuster and its authentication and authorisation options, help streamline cluster management, improve performance and strengthen security.
By automating tasks like resource allocation and rebalancing, Cruise Control reduces manual effort and keeps cluster operations stable and efficient, while its security features protect your Kafka deployment from unauthorised changes.
If you are looking to optimise your Kafka deployment and improve its security, consider adding Kafka Cruise Control to your workflow.
Key takeaway: run one Cruise Control instance per Kafka cluster, enable the Concurrency Adjuster so rebalances speed up or slow down with broker health, and turn on authentication, HTTPS and role-based access before exposing its REST API.
Kafka Cruise Control is an open-source tool, originally built at LinkedIn, that monitors the workload of an Apache Kafka cluster and automatically rebalances replicas and leaders across brokers to meet goals such as rack awareness and disk, network and CPU capacity. It also adds and removes brokers and detects anomalies through a REST API.
Does Cruise Control work with KRaft and Kafka 4.0?
Yes. The project’s compatibility notes list support for Kafka 4.0, which runs only in KRaft mode, from Cruise Control 2.5.144, and Kafka 4.3 from 2.5.146. The project requires Java 17.
What is the Cruise Control Concurrency Adjuster?
It is a feature that automatically raises or lowers the number of concurrent partition and leadership movements during a rebalance. It watches broker metrics such as log flush time, produce and fetch local time and request queue size, and uses an additive increase, multiplicative decrease algorithm. It is disabled by default.
How do you secure Cruise Control?
Set webserver.security.enable=true and choose a security provider: HTTP Basic, JWT, SPNEGO or trusted proxy. Enable HTTPS with the webserver.ssl settings and assign users the VIEWER, USER or ADMIN role so only administrators can run rebalances and broker changes.
Can one Cruise Control instance manage several Kafka clusters?
No. Each Cruise Control instance manages one Kafka cluster, so you run one instance per cluster. The Cruise Control UI can be configured to switch between several Cruise Control instances.
