Kafka Streams Memory Best Practices

Joining and aggregating streams is where Kafka Streams earns its keep, but stateful applications are also where memory problems show up. In this guide we explain how to run Kafka Streams on Kubernetes without unnecessary restarts: how to diagnose a pod that was killed for using too much memory, why the cause is usually RocksDB rather than a heap leak, and how to fix and monitor it.

When you run real-time processing applications, restarts are expensive. A restarted instance may have to restore its state stores from changelog topics before it can process again, which adds lag. If your Kafka Streams application exceeds its container memory limit, Kubernetes kills it, so getting memory right matters.

Updated October 2026: configuration names, RocksDB defaults and metrics below have been checked against the Apache Kafka 4.3 Streams documentation and the Kafka Streams source.

How do you diagnose a Kafka Streams pod killed for memory?

Run kubectl describe pod <pod-name> and look at the Last State of the container. If it shows Terminated with Reason OOMKilled (usually exit code 137), the container went over its memory limit and the kernel killed it. That is the problem this article helps you fix.

kubectl describe pod my-streams-app-0

An OOMKilled pod is different from a Java OutOfMemoryError. An OutOfMemoryError means the JVM heap is full. OOMKilled means the whole process, heap plus everything off-heap, used more memory than the container limit allows.

Why does Kafka Streams use more memory than the heap?

A common mistake is to assume a Kafka memory leak in your own code. Set resource requests and limits on the pod and chart memory usage in Grafana. You will often find that the JVM heap is stable and the growth is somewhere else: in how Kafka Streams uses native memory.

Kafka Streams keeps its record cache on the Java heap. It is sized by statestore.cache.max.bytes (10 MB by default, shared across all stream threads; it replaces the deprecated cache.max.bytes.buffering). Persistent state stores, however, are backed by RocksDB, an embedded C++ key-value store. Each RocksDB instance allocates off-heap memory for its block cache, index and filter blocks and memtables (write buffers). The JVM heap limit (-Xmx) does not cover this memory.

By default Kafka Streams gives every RocksDB instance its own 50 MB block cache and up to three 16 MB memtables. There is one instance per store per partition, and windowed and session stores use several segments, each its own instance. An application with a few stores over dozens of partitions can therefore need several gigabytes of native memory that nothing bounds, which is why the pod is killed even though the heap looks healthy.

Two other causes look like leaks. Store iterators such as KeyValueIterator must be closed (use try-with-resources), otherwise memory keeps growing until the process runs out. And the default glibc allocator can fragment RocksDB memory, so the Kafka documentation recommends running RocksDB with jemalloc.

How do you fix Kafka Streams RocksDB memory usage?

Bound the total off-heap memory RocksDB can use. Kafka Streams lets you plug in your own RocksDB options with the rocksdb.config.setter parameter:

Properties props = new Properties();
props.put(StreamsConfig.ROCKSDB_CONFIG_SETTER_CLASS_CONFIG,
          BoundedMemoryRocksDBConfig.class.getName());

The BoundedMemoryRocksDBConfig class below follows the example in the Kafka Streams memory management guide. It shares one block cache across every RocksDB instance in the application, caches index and filter blocks inside that cache and counts memtable memory against it through a shared WriteBufferManager, so the total is capped at TOTAL_OFF_HEAP_MEMORY:

public static class BoundedMemoryRocksDBConfig implements RocksDBConfigSetter {

    private static final long TOTAL_OFF_HEAP_MEMORY = 512 * 1024 * 1024L;
    private static final long TOTAL_MEMTABLE_MEMORY = 128 * 1024 * 1024L;
    private static final double INDEX_FILTER_BLOCK_RATIO = 0.1;

    private static final org.rocksdb.Cache cache =
        new org.rocksdb.LRUCache(TOTAL_OFF_HEAP_MEMORY, -1, false, INDEX_FILTER_BLOCK_RATIO);
    private static final org.rocksdb.WriteBufferManager writeBufferManager =
        new org.rocksdb.WriteBufferManager(TOTAL_MEMTABLE_MEMORY, cache);

    @Override
    public void setConfig(final String storeName, final Options options, final Map<String, Object> configs) {
        BlockBasedTableConfig tableConfig = (BlockBasedTableConfig) options.tableFormatConfig();
        tableConfig.setBlockCache(cache);
        tableConfig.setCacheIndexAndFilterBlocks(true);
        tableConfig.setCacheIndexAndFilterBlocksWithHighPriority(true);
        tableConfig.setPinTopLevelIndexAndFilter(true);
        options.setWriteBufferManager(writeBufferManager);
        options.setTableFormatConfig(tableConfig);
    }

    @Override
    public void close(final String storeName, final Options options) {
        // The cache and WriteBufferManager are shared by every store, so do not close them here.
    }
}

The right values depend on the number of state stores and partitions in your application and on your access pattern, so treat the numbers above as a starting point. The RocksDB memory usage guide and the Kafka Streams memory management guide explain each setting.

To switch RocksDB to jemalloc, install it in the container image and preload it before the JVM starts:

apt install -y libjemalloc-dev
export LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libjemalloc.so"

How do you calculate your Kafka Streams memory limit?

Work out the RocksDB budget from the container limit down:

1. Get the memory limit of the container, for example from the pod spec with kubectl get pod <pod-name> -o yaml.
2. Reserve headroom for the operating system and other processes in the container. Start with around 10% and adjust from what you measure.
3. Subtract the JVM’s non-heap memory: metaspace, code cache, thread stacks and direct buffers.
4. Subtract the maximum heap (-Xmx), which must cover your application objects, the record cache and client buffers. If you suspect a heap leak, profile the heap first.
5. Give what remains to RocksDB as TOTAL_OFF_HEAP_MEMORY.

Adjust these values to your application’s needs and test under realistic load, including state restoration after a restart, before you settle on limits.

How do you monitor Kafka Streams memory usage?

After applying the fixes, monitor the memory use of your Kafka Streams application. Kafka Streams exposes RocksDB properties-based metrics at the default info recording level, including block-cache-usage, block-cache-pinned-usage, size-all-mem-tables and estimate-table-readers-mem, under this MBean:

kafka.streams:type=stream-state-metrics

Chart them in Grafana next to container memory and JVM heap, so you can see which part is growing. A sample Kafka Streams dashboard is a useful starting point for your own.

Kafka Streams memory takeaways

Running many Kafka Streams applications can be operationally complex. In this article we showed how to diagnose and fix memory problems in Kafka Streams applications on Kubernetes. Bound RocksDB’s off-heap memory, use jemalloc, close your iterators and size the heap and container limit together, and your applications will run without unnecessary restarts. Test and profile your application to find the right limits for your use case.

Key takeaway: an OOMKilled Kafka Streams pod with a healthy heap is almost always unbounded RocksDB off-heap memory, so cap it with a shared block cache and WriteBufferManager and budget heap, non-heap and RocksDB within the container limit.

For more content:

• Productionise Kafka Streams state stores
• Event-driven architecture: a simple guide
• Apache Kafka common mistakes
• Kafka Cruise Control 101
• Kafka performance best practices for monitoring and alerting
• The new consumer rebalance protocol KIP-848

Frequently asked questions