blog by OSO

4 Part Guide to Apache Kafka

Sion Smith 28 March 2023
Newsletter 002 - The Kafka Report by OSO

Exciting developments have kicked off Spring 2023. Kafka is now updated to version 3.4.0, and Kafka Summit London 2023 just released its new agenda. We’ve also released two recent success stories featuring Marlow Navigation and the Department for Education. We hope you enjoy this month’s collection of guide to Apache Kafka, Kafka events, updates, success stories, guides, podcasts, and resources!

Guide to Apache Kafka: The latests Kafka updates

💥 Kafka 3.4.0 launched on February 7, 2023, offering the ability to migrate a Kafka cluster from ZooKeeper to KRaft mode. This KIP-866 provides more information about the ZK to KRaft migration. Another feature we found intriguing is KAFKA-13602. If you’ve used topic partitions to distribute work, the 13602 update lets you broadcast a control message to all workers at once. In addition to KAKFA-13602, Kafka 3.4.0 introduced three other noteworthy updates: 

  • [KAFKA-10360] – Disabling JmxReporter registration
  • [KAFKA-13715] – Adding a “generation” field to the consumer protocol
  • [KAFKA-14286] – Creating time-based cluster metadata snapshots.

Curious what these features mean for your company? Danica Fine, Senior Developer Advocate at Confluent, provides an overview of 3.4.0 and its Kafka Improvement Protocols in her video ‘Apache Kafka 3.4 – New Features and Improvements’. Watch her video to learn more. 

Guide to Apache Kafka: Success Stories

The tales we’ve come across as Kafka experts. 

✔️ Marlow Navigation sought to ensure that its Kafka architecture adhered to best practices and could scale with higher demand, prompting OSO to conduct a Kafka health check. Our comprehensive analysis spanned fifty pages and featured an 18-step checklist, providing Marlow with expert recommendations to help them establish a sound Kafka strategy for the future. Read the complete story on our website.

🚀 The UK’s Department for Education aimed to become a data-synchronous organisation and enlisted Sion Smith, CTO of OSO, to perform a data architecture analysis. With limited ability to make strategic decisions using their data, the department opted for Confluent Enterprise Platform to transition to an event-driven Kafka architecture. Discover more details in our case study. 

Subscribe now

Get the Kafka Report delivered directly to your inbox

We will only use your details for sending your our newsletter and occasional OSO marketing emails. You can unsubscribe at any time.

How to…


What we learned from others on the web! 

🤝 Explore this comprehensive four-part guide to Apache Kafka by Michael Noll, a product and technology strategy expert at Confluent. Written to help software engineers navigate Kafka, the series provides a primer on streams and decisions to make before detailing Kafka’s storage fundamentals, its processing features, and elasticity and fault tolerance. 

What every software engineer should know about Apache Kafka: Events, Streams, Tables, Storage, Processing, and more, by Michael Noll

  1. Streams and Tables in Apache Kafka: A Primer
  2. Topics, Partitions, and Storage Fundamentals
  3. Processing Fundamentals with Kafka Streams and ksqlDB
  4. Elasticity, Fault Tolerance, and other Advanced Concepts

🚀Check out this GitHub example of a Kafka / Confluent workflow for multi-environment deployments using Flux, Kustomize, Helm, and Confluent Operator. The example outlines the project’s repository structure and how OSO deployed base Flux components and Flux Sync to end up with this Confluent stack. It also provides code samples for local deployment and links to related projects

Kafka Community News

What all the Kafka players are up to, be it acquisitions or major project milestones. 

🎧 Tune into this InfoQ podcast episodein which Colin McCabe, and Wes Reisz, a Technical Principal at Thoughtworks, discuss the impact of KRaft mode on Apache Kafka, the differences between ZooKeeper and KRaft, and Colin’s thoughts on the latest Kafka implementation. They also explore the implications of KRaft with multi-cluster architecture and provide insights into Kafka’s roadmap.

People have been tremendously excited about seeing [KRaft mode] as production ready . . . we’ve gotten some great improvements in [the] number of partitions, and we are optimistic that we can hopefully 10X the number of partitions [—] at least.

Colin McCabe, discussing KRaft mode for Kafka

Events and Networking

💥 Kafka Summit London 2023 is almost here. The official agenda is now live, and it includes the event’s first Kafka Summit Hackathon. This highly-anticipated event is our largest gathering of event-streaming and Kafka community leaders of the year, featuring technical leaders from Zopa Bank, University of Cambridge, Wix, IBM, Confluent, and Amazon Web Services. OSO has already assembled a hackathon team, and we want you to join us as we take on this challenge. Come to ExCeL London on May 16th and 17th. 

In addition to the hackathon, Kafka Summit London offers a wide range of talks, and a chance to meet some of your fellow Kafka lovers. View the full agenda at Speakers | Kafka Summit London 2023

We’ll see you at Kafka Summit London 2023!

Kafka Summit London can’t be missed. From May 16th to 17th, join OSO’s developers at the ExCeL London as we showcase our commitment to event-driven architecture. We’re looking forward to connecting with our community and learning from technical leaders who have implemented Apache Kafka in their teams and organisations. What’s more, we want to provide support for first-time attendees. Ask us questions about Kafka before the event at

Until then, may your data be synchronous and your systems scalable. 
OSO, The Kafka Experts

See you at Kafka Summit London 2023, May 16-17

Get started with OSO professional services for Apache Kafka

Have a conversation with a Kafka expert to discover how we help your adopt of Apache Kafka in your business.

Contact Us