Exciting developments have kicked off Spring 2023. Kafka is now updated to version 3.4.0, and Kafka Summit London 2023 just released its new agenda. We’ve also released two recent success stories featuring Marlow Navigation and the Department for Education. We hope you enjoy this month’s collection of guide to Apache Kafka, Kafka events, updates, success stories, guides, podcasts, and resources!
Guide to Apache Kafka: The latests Kafka updates
Kafka 3.4.0 launched on February 7, 2023, offering the ability to migrate a Kafka cluster from ZooKeeper to KRaft mode. This KIP-866 provides more information about the ZK to KRaft migration. Another feature we found intriguing is KAFKA-13602. If you’ve used topic partitions to distribute work, the 13602 update lets you broadcast a control message to all workers at once. In addition to KAKFA-13602, Kafka 3.4.0 introduced three other noteworthy updates:
- [KAFKA-10360] – Disabling JmxReporter registration
- [KAFKA-13715] – Adding a “generation” field to the consumer protocol
- [KAFKA-14286] – Creating time-based cluster metadata snapshots.
Curious what these features mean for your company? Danica Fine, Senior Developer Advocate at Confluent, provides an overview of 3.4.0 and its Kafka Improvement Protocols in her video ‘Apache Kafka 3.4 – New Features and Improvements’. Watch her video to learn more.
Guide to Apache Kafka: Success Stories
The tales we’ve come across as Kafka experts.
Marlow Navigation sought to ensure that its Kafka architecture adhered to best practices and could scale with higher demand, prompting OSO to conduct a Kafka health check. Our comprehensive analysis spanned fifty pages and featured an 18-step checklist, providing Marlow with expert recommendations to help them establish a sound Kafka strategy for the future. Read the complete story on our website.
The UK’s Department for Education aimed to become a data-synchronous organisation and enlisted Sion Smith, CTO of OSO, to perform a data architecture analysis. With limited ability to make strategic decisions using their data, the department opted for Confluent Enterprise Platform to transition to an event-driven Kafka architecture. Discover more details in our case study.
What we learned from others on the web!
🤝 Explore this comprehensive four-part guide to Apache Kafka by Michael Noll, a product and technology strategy expert at Confluent. Written to help software engineers navigate Kafka, the series provides a primer on streams and decisions to make before detailing Kafka’s storage fundamentals, its processing features, and elasticity and fault tolerance.
What every software engineer should know about Apache Kafka: Events, Streams, Tables, Storage, Processing, and more, by Michael Noll
- Streams and Tables in Apache Kafka: A Primer
- Topics, Partitions, and Storage Fundamentals
- Processing Fundamentals with Kafka Streams and ksqlDB
- Elasticity, Fault Tolerance, and other Advanced Concepts
Check out this GitHub example of a Kafka / Confluent workflow for multi-environment deployments using Flux, Kustomize, Helm, and Confluent Operator. The example outlines the project’s repository structure and how OSO deployed base Flux components and Flux Sync to end up with this Confluent stack. It also provides code samples for local deployment and links to related projects.
What all the Kafka players are up to, be it acquisitions or major project milestones.
Tune into this InfoQ podcast episode, in which Colin McCabe, and Wes Reisz, a Technical Principal at Thoughtworks, discuss the impact of KRaft mode on Apache Kafka, the differences between ZooKeeper and KRaft, and Colin’s thoughts on the latest Kafka implementation. They also explore the implications of KRaft with multi-cluster architecture and provide insights into Kafka’s roadmap.
People have been tremendously excited about seeing [KRaft mode] as production ready . . . we’ve gotten some great improvements in [the] number of partitions, and we are optimistic that we can hopefully 10X the number of partitions [—] at least.
Colin McCabe, discussing KRaft mode for Kafka
Kafka Summit London 2023 is almost here. The official agenda is now live, and it includes the event’s first Kafka Summit Hackathon. This highly-anticipated event is our largest gathering of event-streaming and Kafka community leaders of the year, featuring technical leaders from Zopa Bank, University of Cambridge, Wix, IBM, Confluent, and Amazon Web Services. OSO has already assembled a hackathon team, and we want you to join us as we take on this challenge. Come to ExCeL London on May 16th and 17th.
In addition to the hackathon, Kafka Summit London offers a wide range of talks, and a chance to meet some of your fellow Kafka lovers. View the full agenda at Speakers | Kafka Summit London 2023.
We’ll see you at Kafka Summit London 2023!
Kafka Summit London can’t be missed. From May 16th to 17th, join OSO’s developers at the ExCeL London as we showcase our commitment to event-driven architecture. We’re looking forward to connecting with our community and learning from technical leaders who have implemented Apache Kafka in their teams and organisations. What’s more, we want to provide support for first-time attendees. Ask us questions about Kafka before the event at enquiries@oso.sh.
Until then, may your data be synchronous and your systems scalable.
OSO, The Kafka Experts
Kafka 3.4.0 launched on February 7, 2023, offering the ability to 
Marlow Navigation sought to ensure that its Kafka architecture adhered to best practices and could scale with higher demand, prompting OSO to conduct a Kafka health check. Our comprehensive analysis spanned fifty pages and featured an 18-step checklist, providing Marlow with expert recommendations to help them establish a sound Kafka strategy for the future. 
The UK’s Department for Education aimed to become a data-synchronous organisation and enlisted Sion Smith, CTO of OSO, to perform a data architecture analysis. With limited ability to make strategic decisions using their data, the department opted for Confluent Enterprise Platform to transition to an event-driven Kafka architecture. 
Tune into 
