Securing a Kafka cluster comes down to two questions. Authentication confirms who a client or broker is. Authorisation decides what that identity is allowed to do once it is connected. Think of a restaurant: you show your ID to get through the door (authentication), and the host checks your reservation before seating you in the VIP area (authorisation). Kafka needs both, plus encryption in transit, before it is safe to expose to real workloads.
This guide covers each authentication option Kafka supports, how to configure mTLS and SASL/SCRAM, where OAuth fits, how to point the Kafka command-line tools at a secured cluster, and how to lock everything down with ACLs. Every configuration example is written for Apache Kafka 4.3 running in KRaft mode; property names can differ in older releases, so check the security documentation for your version.
How does authentication work in Kafka?
Kafka applies security per listener. Each listener is a network endpoint on a broker or controller, and listener.security.protocol.map assigns it one of four security protocols:
Security protocol
Encryption in transit
Client authentication
PLAINTEXT
None
None
SSL
TLS
Optional TLS client certificates (mTLS)
SASL_PLAINTEXT
None
SASL mechanism
SASL_SSL
TLS
SASL mechanism
SASL (Simple Authentication and Security Layer) is a pluggable framework, so a SASL listener can enable one or more mechanisms through sasl.enabled.mechanisms: GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. Clients choose the matching security.protocol and sasl.mechanism.
Whatever mechanism you use, a successful handshake produces a Kafka principal, such as User:alice. That principal is what ACLs are written against. Most production clusters use explicitly named listeners, for example a CLIENT listener on SASL_SSL for applications, a separate listener for broker-to-broker traffic (inter.broker.listener.name) and the controller listener named in controller.listener.names for KRaft quorum traffic. Brokers authenticate to each other and to the controllers too, so secure those listeners as well.
One limitation is worth knowing early: Kafka has no native support for certificate revocation lists (KAFKA-3700). To revoke access you rely on JVM-level mechanisms, short certificate lifetimes, or removing the principal’s ACLs.
Which authentication mechanism should you use?
The right mechanism depends on the identity systems you already run. The comparison below is a relative assessment from our production work, not a Kafka guarantee; your certificate tooling or identity provider can shift it.
Mechanism
Set-up effort
Credential rotation
Identity-provider fit
Use it when
mTLS (SSL)
Medium: you need a CA and per-client certificates
Reissue certificates before they expire; no native revocation
PKI or a service mesh
You already manage certificates, or want encryption and identity in one step
SASL/SCRAM
Low to medium
Change passwords with kafka-configs; new connections pick them up
Kafka stores credentials in cluster metadata
You want username and password auth without external systems
SASL/OAUTHBEARER
Medium to high
Short-lived tokens issued by your identity provider
Okta, Entra ID, Keycloak and other OIDC providers
You already run a central identity provider
SASL/GSSAPI (Kerberos)
High
Tickets expire and renew; revoke by disabling the principal
Active Directory or an MIT KDC
Your enterprise already standardises on Kerberos
SASL/PLAIN
Low
Depends on the callback handler or credential store you plug in
Custom callback handlers
You need to delegate password checks to an external store, always over TLS
A few practical notes from running these in production:
mTLS gives you encryption and authentication together. Forgetting to renew certificates is the classic outage, so automate issuance and alert on expiry.
SCRAM never sends the password over the wire and uses a salted challenge-response with a random nonce, which protects against replayed authentication messages. Use it only on SASL_SSL listeners.
Kerberos gives fine-grained control over expiry and revocation, but troubleshooting is harder, and exposing a Kerberos-secured cluster to the internet means exposing the KDC as well. Avoid that when Active Directory is your KDC.
PLAIN sends the password itself, protected only by TLS. Never enable it on a SASL_PLAINTEXT listener.
Key takeaway: Start with mTLS or SCRAM over TLS, use OAuth when you already run a central identity provider, never run PLAIN without TLS, and authorise every principal with ACLs.
How do you configure mTLS for Kafka?
mTLS means the broker verifies a certificate from every client as well as presenting its own. On the broker, enable an SSL listener and require client certificates (Apache Kafka 4.3):
When ssl.client.auth is required (or requested), the broker truststore must contain every CA that signs client certificates. Each client then needs its own keystore plus a truststore holding the broker CA:
Hostname verification is on by default for client and inter-broker connections. Leave ssl.endpoint.identification.algorithm at its default rather than setting it to an empty string to “just get it working”.
By default, a certificate principal is the full distinguished name, for example CN=producer,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown. ssl.principal.mapping.rules maps that to a short name. Rules are evaluated in order and the first match wins:
With the first rule, a certificate for CN=producer,OU=ServiceUsers,... maps to the principal User:producer. For anything a rule cannot express, set a custom principal.builder.class. Plan certificate rotation up front: Kafka can reload updated keystores through dynamic broker configuration with kafka-configs, or you can roll brokers one at a time, so document which method you use.
How do you configure SASL/SCRAM?
In KRaft mode, SCRAM credentials are stored in the cluster metadata log, not in ZooKeeper. Credentials that brokers use to talk to each other must exist before the brokers start, so add them when you format storage (Apache Kafka 4.3):
Run SCRAM only over SASL_SSL. On SASL_PLAINTEXT the handshake is protected but every message afterwards travels unencrypted.
How does OAuth (OAUTHBEARER) work with Kafka?
OAUTHBEARER lets clients authenticate with OAuth 2.0 bearer tokens. Kafka’s default OAUTHBEARER implementation creates and validates unsecured JSON Web Tokens, and the Kafka documentation says it is only suitable for non-production use. For production, KIP-768 (Kafka 3.1 onwards) added built-in OIDC support: clients fetch tokens from your identity provider, and brokers validate them against its JWKS endpoint.
A client using the client-credentials grant looks like this (Apache Kafka 4.3):
On the broker, set OAuthBearerValidatorCallbackHandler as the listener’s server callback handler and point sasl.oauthbearer.jwks.endpoint.url at your provider, ideally with sasl.oauthbearer.expected.issuer and sasl.oauthbearer.expected.audience set. OAuth property names have changed between releases, so match them to your Kafka version. For a full walkthrough with an identity provider, read our guide to Apache Kafka security using OAuth.
How do you use Kafka command-line tools with SASL and SSL clusters?
The Kafka CLI tools are ordinary Kafka clients, so they need the same security settings as your applications. Put them in a properties file and pass it to each tool. For a SCRAM cluster, client.properties looks like this:
If a tool hangs or reports disconnections, the usual causes are a wrong security.protocol for that port, a truststore missing the broker CA, or a principal with no ACLs on the resource. Keep these files readable only by their owner, because they contain credentials.
How do you authorise clients with ACLs?
Authentication proves identity; ACLs decide what each principal can do. In KRaft clusters, enable the built-in authoriser on every broker and controller:
An ACL reads as: principal P is allowed or denied operation O from host H on resources matching pattern R. Operations include Read, Write, Create, Delete, Alter and Describe; resources include topics, consumer groups, the cluster and transactional IDs. Once an authoriser is set, any principal without a matching allow ACL is denied, unless you set allow.everyone.if.no.acl.found=true, which we do not recommend. Deny rules take precedence over allow rules.
Make broker and admin principals super users so they do not need individual ACLs. super.users is separated by semicolons because certificate names contain commas. Then grant applications only what they need:
If you use full certificate names as principals, quote the whole principal: --allow-principal "User:CN=producer,OU=OSO,O=Kafka,L=London,ST=UK,C=UK". Mapping certificates to short names with ssl.principal.mapping.rules keeps ACLs far easier to read. Avoid IP-based identification on its own: it is not secure.
What about Schema Registry and Kafka Connect?
Kafka client authentication secures the broker protocol only. It does not secure HTTP endpoints such as the Kafka Connect REST API or Schema Registry. Anyone who can reach an unsecured Connect REST port can create connectors that read or write your data, using the worker’s own Kafka credentials.
Secure each REST layer separately with TLS and either basic authentication or mTLS, restrict network access to those ports, and give Connect workers and Schema Registry their own Kafka principals with ACLs scoped to their internal topics. For the schema side, see our guide to managing schemas in Kafka.
Kafka supports TLS client certificates (mTLS) and SASL with five mechanisms: GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 and OAUTHBEARER. You enable them per listener, and each listener can allow more than one SASL mechanism.
What is the difference between SSL and SASL_SSL in Kafka?
SSL encrypts the connection with TLS and can authenticate clients with certificates. SASL_SSL also encrypts with TLS, but authenticates clients with a SASL mechanism such as SCRAM or OAUTHBEARER, so clients need a username and password or a token instead of their own certificate.
How do I enable mTLS in Kafka?
Enable an SSL listener with a keystore and truststore on each broker, then set ssl.client.auth=required. Give every client its own certificate signed by a CA the broker trusts, and use ssl.principal.mapping.rules to turn certificate names into short principals for ACLs.
Does Kafka support OAuth?
Yes. Kafka supports OAuth 2.0 through the SASL/OAUTHBEARER mechanism, with built-in OIDC support since Kafka 3.1 (KIP-768). The default unsecured-token implementation is for development only; production clusters should validate tokens from a real identity provider.
How do I use Kafka CLI tools against a SASL_SSL cluster?
Put security.protocol, sasl.mechanism, sasl.jaas.config and your truststore settings in a properties file. Pass it with --command-config to admin tools such as kafka-topics.sh and kafka-acls.sh, and with --producer.config or --consumer.config to the console clients.
Do I need Kerberos for Kafka?
No. Kerberos (SASL/GSSAPI) is only worth it when your organisation already runs it, for example through Active Directory. Most teams get strong security with less effort from mTLS or SCRAM over TLS, or from OAuth if they already have an identity provider.
OSO engineers run Apache Kafka in production for banks, retailers and the public sector. Tell us what you are weighing up and we will give you a straight answer.
