blog by OSO

Looking back at Kafka Summit London 2023

Sion Smith 22 May 2023
OSO - Kafka Summit 2023 recap

Kafka Summit 2023: Key takeaways

Sion — We joined this event as a startup sponsor, organised by Confluent, one of OSO’s key technology partners. We brought our technical Kafka experts to explain the importance of Apache Kafka best practices, and outlined how OSO professional services can increase the success of Kafka adoption in your organisation. Over two days, over 1,500 Kafka enthusiasts attended a combination of workshops, presentations and networking sessions –  I’ll distil the opening keynote, outline what is currently happening and what is on the roadmap for the upcoming future.

Kafka Summit London 2023 kick off

Keynote: The incredible adoption of Apache Kafka across all industries

Opening Kafka Summit London 2023, Jay Kreps took to the stage to address the business trends for the need of real-time data. In an ever more competitive world, organisations are looking to data as their differentiating factor. Kreps cites hungry AI models as reasons why streaming has become essential for quickly processing high volumes of data. Machine learning models have proven to be a total game changer from personalised chatbots in e-commerce to increased security in financial services with intelligent fraud detection and Kafka is the engine powering all these.

Keynote Highlights:

  • The Data Streaming Platform is now real – Kafka at the core, with other layers now being built on top of the Kafka core, connectors, stream processing, and tools to govern streaming data.
  • Queues in Kafka are coming, is this a play to land-grab more use cases or will this actually be useful in day to day use cases? KIP-932
  • Managed Flink is finally coming to Confluent Cloud. AWS and Avien have been supporting the Flink community over the last few years where Confluent has been lagging behind with its KsqlDB. Flink is now fast becoming the de facto standard for streaming processing.
  • Stream sharing, first step in competing with AWS Data Exchange, where customers can consume / connect with over 3,500 unique data streams. Confluent Stream Sharing is the first step in simplifying the process of sharing data outside the organisation – think the ability to commercialise their data that is currently generating no revenue. Companies can now create completely new revenue streams based on integrated data from a combination of organisations.
  • Data Quality Rules which allow inbuilt validation without the need for complex schemas is also coming to the Confluent platform.


OSO - The Kafka Experts

OSO’s Insights from Breakout Sessions & Lightning Talks

A couple of sessions which caught my attention which provided practical insights included:

1. Apples and Oranges – Comparing Kafka Streams and Flink

An extremely popular session, focused on an interesting comparison between  Kafka Streams and Flink given by Bill Bejeck, DevX Engineer at Confluent. What stood out to me as the greatest take away, its nearly impossible to recommend one over another, it ultimately depends on the use-case and technical skills within the organisation – there’s really quite a lot of overlap between the two technologies. The notable differences have been captured in an easy to use Kafka Streams vs Flink checklist, which matches stream processing requirements against which framework best meets your needs.

2. Designing a Data Mesh With Kafka

Another super interesting session presented by the team at Saxo Bank around “Designing a Data Mesh With Kafka” Covering how the team has implemented data mesh principles, and that the term can mean different things to different organisations. Leveraging open source tooling, like the Data Hub Project, Confluent Cluster Linking and the Apache Kafka Connector ecosystem, they have focused on operational data to deliver embedded analytics to optimise operational overheads, making it more discoverable, addressable, and trustworthy. This also ties into the open source project SpecMesh which OSO are supporting.

A thriving developer lead community

The importance of a passionate community is for all to see at the end of each session. Questions, interactions and one-to-ones are all the things we have love about in-person conferences. I had the pleasure of meeting some of the key Kafka Summit committee members forging new lasting relationships. With OSO powered London Kafka Meetup now gaining in traction, please register for the next event in June.

The benefits of sponsoring Kafka Summit London 2023 

Sponsoring and having a physical space at Kafka Summit was a career highlight for the OSO team. The most significant benefit for us was putting faces to all the names and great people we have worked with over the last three years. Hearing stories about how we have helped so many teams share best practices was inspiring, and conversely the opportunities which organisations are still facing when building Kafka streams applications. One slightly negative was the layout and location of the partner stands, as the venue was bigger and split over multiple levels. The feedback we got was that attendees found it difficult traversing floors between breakout sessions and therefore struggled to speak to every vendor in-depth throughout the day. Hopefully Confluent will take onboard this feedback and tweak the setup of the venue for the next Kafka Summit in 2024.

Need Kafka support or advice?

We love helping companies get the most out of Kafka and event streaming. Check out the Kafka Services we offer and get in touch to find out how we can support your business.

Get started with OSO professional services for Apache Kafka

Have a conversation with a Kafka expert to discover how we help your adopt of Apache Kafka in your business.

Contact Us