# OSO > OSO is a UK-based software consultancy specialising in Apache Kafka, real-time data streaming, and cloud infrastructure. We help enterprises design, deploy, and operate production Kafka platforms — from migration and architecture through to 24/7 managed support. > OSO works with organisations including Formula 1 racing teams, Lloyds Banking Group, the UK Department for Education, Cambridge University Press, and Scania. Our services span Kafka architecture design, managed Kafka operations, Confluent and AWS MSK deployments, Kubernetes-based infrastructure, and open-source developer tooling. We are Confluent partners and AWS Advanced Consulting Partners. --- ## Pages - [Fully Managed Apache Kafka Service | Enterprise Ready](https://oso.sh/fully-managed-kafka/) → [Markdown](https://oso.sh/wp-content/uploads/llms_md/page-1.md): Deploy enterprise-grade Apache Kafka on your infrastructure with full management. Eliminate operational complexity while maintaining complete control. 24/7 expert support included. - [Download Kafka-as-a-service White Paper](https://oso.sh/download-kafka-as-a-service-white-paper/) - [Contact the OSO team](https://oso.sh/book-a-call/) - [Migration from Apache Kafka to Amazon MSK](https://oso.sh/technologies/apache-kafka-migration-to-amazon-msk/): Explore how OSO helps companies migrate their streaming data processing from Apache Kafka to Amazon MSK, to reduce TCO and manage clusters more efficiently. - [All flavours of Apache Kafka Support](https://oso.sh/kafka-support/) → [Markdown](https://oso.sh/wp-content/uploads/llms_md/page-4.md): Enterprise Kafka support from the experts of Apache Kafka - Scale, optimise and secure your data platform with OSO - [What is Apache Kafka?](https://oso.sh/technologies/kafka/) → [Markdown](https://oso.sh/wp-content/uploads/llms_md/page-3.md): Learn about Apache Kafka - a distributed streaming platform that enables companies to build scalable, fault-tolerant, and real-time data pipelines - [The Kafka Report](https://oso.sh/newsletter/) - [Kafka Services](https://oso.sh/kafka-services/) → [Markdown](https://oso.sh/wp-content/uploads/llms_md/page-2.md): Maximise the Value of Streaming Technologies with Our World-Leading Kafka Expert Consultants and Specialised Kafka Services - [Confluent and OSO](https://oso.sh/technologies/confluent/): We deliver Confluent advisory and implementation services to accelerate your data platform - [Technologies](https://oso.sh/technologies/) - [Pedigree #2](https://oso.sh/solutions/pedigree/) - [Dev tools](https://oso.sh/dev-tools/) - [Solutions](https://oso.sh/solutions/): We understand different industries have different requirements when it comes to real-time data. Different industries have different requirements when it comes to real-time data. We build sector specific solutions that meet your specific challenges. - [Tableau Deployment Library #2](https://oso.sh/solutions/tableau-deployment-library/) - [About](https://oso.sh/who-we-are/): OSO help teams for building event driven applications on Kafka to enable real-time access. - [Blog](https://oso.sh/blog/) - [Home](https://oso.sh/) → [Markdown](https://oso.sh/wp-content/uploads/llms_md/page.md): Oso has Kafka Experts help teams build event driven applications and provides Kafka services. Enable real-time access to your data. --- ## Case Studies - [How OSO helped Colliers design a real-time, event-driven CRM-to-ERP integration architecture across EMEA](https://oso.sh/case-studies/how-oso-helped-colliers-design-a-real-time-event-driven-crm-to-erp-integration-architecture-across-emea/): OSO provided Kafka architecture design, expert consultancy, and a comprehensive enterprise backup to support Colliers' most complex EMEA integration challenge. - [OSO helps Formula 1 racing team introduce Kafka architecture for 2026](https://oso.sh/case-studies/oso-helps-formula-1-racing-team-introduce-kafka-architecture-for-2026/): Our client, a leading Formula One racing team, scaled their deployment of Apache Kafka to analyse race data and enable split-second strategic decisions. - [Discover how enterprise event streaming accelerated STC Kuwait's digital transformation](https://oso.sh/case-studies/stc-kuwait-enterprise-event-streaming/): Explore how STC Kuwait integrated its billing and advanced analytics systems with real-time Kafka enterprise event streaming platform Confluent 6.0. - [Explore how a Kafka health check helped Tungsten Automation manage Kafka at scale](https://oso.sh/case-studies/explore-how-kafka-health-check-helped-tungsten/): Discover how Tungsten Automation successfully managed Apache Kafka at scale with the help of a Kafka health check. - [Discover how Kafka in app design helped Memrise launch a new feature and gamify the user experience](https://oso.sh/case-studies/how-kafka-in-app-design-helped-memrise-gamify/): With 24/7 Kafka in app design support, Memrise incorporated an agile, more resilient data architecture into its legacy system. Learn the details. - [Find out how Kafka architecture in manufacturing transformed Scania Group's battery assembly factory in Sweden](https://oso.sh/case-studies/kafka-architecture-in-manufacturing/): Explore how Kafka architecture in manufacturing transformed Scania Group's factory systems at its new battery operations plant in Södertälje, Sweden. - [Explore how Lloyds Banking Group designed a secure and scalable environment with Kafka 24/7 support](https://oso.sh/case-studies/how-we-helped-lloyds-banking-group-deploy-kafka-on-prem/): Production-grade Kafka platform build that helped Lloyds launch new banking products and enhance its customer experience - [Learn how OSO’s Kafka support helped Cambridge University Press migrate to a modern digital learning system](https://oso.sh/case-studies/cambridge-university-kafka-support/): A real-time data pipeline powered by Apache Kafka and submission service helped Cambridge University Press synchronise its data between content... - [Analyse how Kafka for building management helped Kugu migrate its legacy resources to AWS ](https://oso.sh/case-studies/kugu-kafka-for-building-management/): Kafka For Building Management. Discover how Kugu leveraged Kafka to automate its legacy resource migration to AWS. Kafka for Real Estate - [Investigate why a Kafka proof of concept helped Dufry prove the merits of its new data architecture](https://oso.sh/case-studies/dufry-apache-kafka-proof-of-concept-poc/): An Apache Kafka proof of concept helped Dufry's Head of Data, Chief Revenue Officer, and Chief Technical Officer perform better. - [Discover how designing the right Apache Kafka Architecture helped the Department for Education become a data-synchronous organisation](https://oso.sh/case-studies/department-for-education-kafka-architecture/): After carefully analysing its options, the DfE selected a Kafka architecture to improve its digital infrastructure and generate new value from its data - [Explore how Apache Kafka best practices helped Marlow Navigation pursue digital transformation](https://oso.sh/case-studies/apache-kafka-best-practices-marlow/): One of OSO's Apache Kafka Best Practices. See How OSO helped Marlow's undergoing digital transformation - [Explore how Terraform-based GitOps helped Drivvn automate Kafka](https://oso.sh/case-studies/automate-kafka-with-terraform-based-gitops/): Automating Kafka deployment with Terraform-based GitOps helped Drivvn's developers minimise errors and monitor their new technology. - [Learn how Kafka support helped Drivvn migrate from batch to real time streaming data](https://oso.sh/case-studies/drivvn-batch-to-real-time-streaming-with-kafka/): With Kafka support, Drivvn migrated from batch to real time streaming data and adopted real-time data streaming to enhance its platform. Discover how they did it. - [Uncover how secure deployment helped Detected’s clients avoid the $200 billion fraud problem in B2B digital commerce](https://oso.sh/case-studies/highly-secure-deployment-detected-oso/): The OSO team took a prototype and turned it into a productionised, highly secure deployment in just 10 weeks in Detected! - [Explore how AWS support helped 33N successfully enable real-time data visualisation at the NHS](https://oso.sh/case-studies/how-33n-enabled-nhs-real-time-data-visualisation/): With careful iteration and implementation, we provided 33N with a tool that will make it easier for real-time data visualisation - [Investigate how support services helped VAS Group complete its AWS migration journey](https://oso.sh/case-studies/vas-group-achieved-growth-with-aws-migration/): OSO helped VAS group achieve growth with AWS with AWS migration. See how VAS migrated their applications to AWS. - [Discover how support services helped M1 Finance launch a production-ready Tableau server solution in just 12 days](https://oso.sh/case-studies/how-oso-helped-m1-to-launch-a-production-ready-tableau-server-solution-in-12-days/): M1 Finance was growing quickly and storing a lot of various datasets inside of AWS. The team wanted a tool... --- ## Blog - [How to Eliminate the 5 Hidden Cost Traps in Your Kafka-to-Iceberg Pipeline](https://oso.sh/blog/how-to-reduce-kafka-to-iceberg-pipeline-costs/): Learn how to eliminate the five hidden cost traps in Kafka-to-Iceberg pipelines. Reduce cloud spend by removing connectors, duplicate data, and over-provisioned compute. - [How to Build a Real-Time Context Engine for AI Agents Using Event Driven Architectures](https://oso.sh/blog/how-to-build-real-time-context-engine-ai-agents-apache-kafka/): Learn how to build a real-time context engine for AI agents using Apache Kafka. Architecture, governance, MCP integration, and practical implementation guide. - [What Every Engineer Who Uses Apache Kafka Needs to Know for 2026](https://oso.sh/blog/what-every-kafka-engineer-needs-to-know-2026/): The Kafka ecosystem is shifting dramatically in 2026. OSO engineers break down what's changing, what it means for your architecture, and how to prepare. - [Kafka Protocol Internals: How Partition Remapping Proxies Transform Messages Without Breaking Clients](https://oso.sh/blog/kafka-protocol-internals-partition-remapping-proxy/): Learn how Kafka's binary protocol enables partition remapping proxies to cut managed Kafka costs by 90% whilst remaining completely transparent to applications. - [How to Back Up Your Kafka Cluster: A Guide to Point-in-Time Recovery](https://oso.sh/blog/how-to-back-up-kafka-cluster-point-in-time-recovery/): Learn how to back up your Kafka cluster properly with point-in-time recovery and consumer offset restoration. Discover why replication alone isn't enough. - [Current 2025: Watching Confluent Prepare for Sale in Real Time](https://oso.sh/blog/current-summit-new-orleans-2025-review/): Current Summit New Orleans 2025: A critical look at the biggest Kafka conference. What worked, what didn't, and where data streaming is heading. - [Why You Don't Need Apache Flink for Agentic AI (And Why Akka Is the Simpler Choice)](https://oso.sh/blog/flink-vs-akka-agentic-ai-comparison/): Apache Flink is overkill for AI agents. Learn why Akka's purpose-built framework delivers simpler, more maintainable agentic systems in production. - [Building Multi-Region Orchestration with Apache Kafka: A Pull-Based Architecture](https://oso.sh/blog/kafka-multi-region-orchestration-architecture-pull-based-workflows/): Discover how to build scalable multi-region orchestration using Apache Kafka's pull-based architecture. Real-world patterns from production systems. - [From Block to Object Storage: Engineering a Cost-Efficient Data Streaming Platform at Massive Scale](https://oso.sh/blog/kafka-object-storage-cost-optimization-disaggregated-architecture/): Learn how disaggregated architectures using object storage cut Kafka costs by 71%. Real-world strategies for migrating from EBS to S3 at scale. - [Solving the Hidden Consistency Crisis in Apache Kafka CDC Pipelines: Why Your Multi-Table Transactions Are Breaking Your Data](https://oso.sh/blog/apache-kafka-cdc-transaction-consistency-debezium-guide/): Learn how transaction-aware Change Data Capture solves consistency problems in Kafka pipelines when database transactions span multiple tables. - [How to Unify Multi-Platform Kafka Operations: A Complete Guide to Hybrid Deployment Success](https://oso.sh/blog/how-to-manage-hybrid-kafka-deployments-guide/): Learn how to unify multi-platform Kafka operations with proven strategies for hybrid deployments, monitoring consolidation, and seamless application management. - [How to Use Kafka Streams Processor API for Complex Enterprise Applications](https://oso.sh/blog/how-to-use-kafka-streams-processor-api-for-complex-enterprise-applications/): Master Kafka Streams Processor API for enterprise applications. Learn when to move beyond DSL for complex state management and advanced streaming use cases. - [Building a Unified Schema Registry: How to Eliminate Cross-System Compatibility Failures in Enterprise Data Architectures](https://oso.sh/blog/unified-schema-registry-cross-system-compatibility-enterprise/): Learn how unified schema registry architecture eliminates cross-system compatibility failures in enterprise data ecosystems with early validation and seamless schema evolution. - [KIP-714 Client Metrics: How Apache Kafka Finally Solved Its Biggest Monitoring Challenge](https://oso.sh/blog/kip-714-kafka-client-metrics-centralized-monitoring-solution/): Learn how KIP-714 transforms Kafka monitoring with centralized client telemetry. Discover how to eliminate monitoring complexity and catch misconfigurations automatically. - [Complete Guide to Building Self-Service Kafka Operations: Eliminating Manual Bottlenecks at Enterprise Scale](https://oso.sh/blog/complete-guide-self-service-kafka-operations-enterprise/): Learn how to eliminate 5-day Kafka delays with self-service automation. Complete guide to GitOps workflows, governance, and enterprise scaling. - [7 Must-Have CLI Tools Every Apache Kafka Engineer Should Know in 2025](https://oso.sh/blog/7-must-have-cli-tools-every-apache-kafka-engineer-should-know-in-2025/): Discover 7 essential Apache Kafka CLI tools that transform operations in 2025. From kcat to topicctl, master the command-line tools every Kafka engineer needs. - [Apache Kafka's KRaft Protocol: How to Eliminate Zookeeper and Boost Performance by 8x](https://oso.sh/blog/apache-kafkas-kraft-protocol-how-to-eliminate-zookeeper-and-boost-performance-by-8x/): Discover how Apache Kafka's KRaft protocol eliminates Zookeeper dependencies and delivers 8x performance improvements. Complete guide to KRaft architecture, migration, and real-world benefits. - [How to Build High-Performance Kafka Streams Applications That Scale Beyond Partition Limits](https://oso.sh/blog/how-to-build-high-performance-kafka-streams-applications-that-scale-beyond-partition-limits/): Build high-performance Kafka Streams apps that scale beyond partition limits using async processing. Learn techniques that delivered 6x performance gains in production. - [Productionise Kafka Streams State Stores: How Unbundled Architecture Transforms Kafka Streams Operations](https://oso.sh/blog/productionise-kafka-streams-state-stores-how-unbundled-architecture-transforms-kafka-streams-operations/): In the Kafka Streams ecosystem, we’ve all experienced the fundamental paradox that haunts stateful applications: the framework promises elasticity, fault... - [How to auto scale Apache Kafka with Tiered Storage in Production](https://oso.sh/blog/how-to-auto-scale-apache-kafka-with-tiered-storage-in-production/) - [How Kafka Consumer 4.0's New Rebalance Protocol Eliminates the Two-Phase Bottleneck](https://oso.sh/blog/kafka-consumer-4-0-new-rebalance-protocol/): Kafka Consumer 4.0 transforms rebalancing with incremental server-side processing. No more slow member bottlenecks or two-phase synchronisation delays. - [Building Bulletproof Disaster Recovery for Apache Kafka: A Field-Tested Architecture](https://oso.sh/blog/building-bulletproof-disaster-recovery-for-apache-kafka-a-field-tested-architecture/): Master Apache Kafka disaster recovery with proven strategies from OSO engineers handling 300M+ daily transactions. Learn MirrorMaker setup, active-active patterns, and failover procedures with code examples. - [How to run Apache Kafka across multiple Kubernetes clusters](https://oso.sh/blog/how-to-run-apache-kafka-across-multiple-kubernetes-clusters/): The Definitive Guide to Running Kafka Across Multiple Kubernetes Clusters using Strimzi for Automation, Dynamic Scaling and Failover - [Kafka queues in Apache Kafka 4.0 via Share Groups](https://oso.sh/blog/kafka-queues-in-apache-kafka-4-0-via-share-groups/): The consequences of this design become apparent when we examine scalability. If your topic has five partitions, you can have... - [Apache Kafka outbox pattern](https://oso.sh/blog/apache-kafka-outbox-pattern/): Implementing the outbox pattern, you need a reliable mechanism to move messages from your outbox table to Kafka. - [Guide to ZooKeeper to KRaft migration](https://oso.sh/blog/guide-to-zookeeper-to-kraft-migration/): Successful KRaft migration begins with thorough preparation. Read these 5 steps to migrate successfully. - [How to scale Apache Kafka in your organisation](https://oso.sh/blog/how-to-scale-apache-kafka-in-your-organisation/): How do you build capabilities across dozens of teams without creating bottlenecks or sacrificing quality - [Multi-message types in Kafka topics: Lessons from a real-world pipeline](https://oso.sh/blog/multi-message-types-in-kafka-topics/): By challenging the standard pattern and designing a solution tailored to our domain, we created a more flexible, maintainable, and developer-friendly system. - [How to design Kafka events and event streams](https://oso.sh/blog/how-to-design-kafka-events-and-event-streams/): Events are records of significant moments in a system and are essential to decentralised computing and modular service architectures. Effective... - [How to secure your Apache Kafka cluster](https://oso.sh/blog/how-to-secure-your-apache-kafka-cluster/): Apache Kafka offers several methods for authenticating Kafka clients to brokers. This guide aims to showcase these methods, provide guidance... - [Kafka Consumer Configuration Cheatsheet](https://oso.sh/blog/kafka-consumer-configuring-cheatsheet/): Configuring Apache Kafka consumers is confusing! This table helps us the remember what each parameter means. - [Apache Kafka - Glossary](https://oso.sh/blog/apache-kafka-glossary/): Kafka terminology can be confusing! Here is everything you need to know on one page! - [The Future of Streaming Data](https://oso.sh/blog/the-future-of-streaming-data/): As we kick off January, it’s time to share OSO’s annual trends and predictions for data streaming technology. Here are our top three predictions for 2025. - [Apache Kafka 2024 Review](https://oso.sh/blog/apache-kafka-2024-review/): This year, OSO invested in growing the London community. We hosted nine meetups, brought together 470 new members passionate about technology and innovation, produced 19 articles and blog posts, shared insights through three podcasts, and developed 35 code repositories. - [Confluent Certified Developer for Apache Kafka :: Study Guide](https://oso.sh/blog/confluent-certified-developer-for-apache-kafka-study-guide/): This guide will help developers understand the essential principles of Apache Kafka, and will prepare them for the CCDAK exam. - [Serverless Tiered Storage in Apache Kafka](https://oso.sh/blog/serverless-tiered-storage-in-apache-kafka/): Apache Kafka now leverages cloud object storage to provide cost-effective and scalable storage for Kafka data. - [Machine Learning with Apache Kafka](https://oso.sh/blog/machine-learning-with-apache-kafka/): How to build streaming architecture using Kafka to feed model training - [What are the key concepts of Apache Kafka](https://oso.sh/blog/what-are-the-key-concepts-of-apache-kafka/): A Kafka cluster is a group of broker nodes working together to provide, scalability, availability, and fault tolerance. - [How to increase the adopt of Kafka in your team](https://oso.sh/blog/how-to-increase-the-adopt-of-kafka-in-your-team/) - [Current 2024: No clear vision for the future of Confluent?](https://oso.sh/blog/confluent-current-2024/): We’re still seeing the classic Kafka training and technical foundation talks at Current, but make no mistake—the overall vision of... - [Kafka producer best practices](https://oso.sh/blog/kafka-producer-best-practices/): Learn which configuration values to chance on the Producer API to achieve high performance on your Apache Kafka producers - [5 Kafka Error Handling Tips](https://oso.sh/blog/5-kafka-error-handling-tips/): Best practice tips and examples from Kafka Experts on how to handle Kafka errors in your applications - [How to migrate Kafka to KRaft mode](https://oso.sh/blog/how-to-migrate-kafka-to-kraft-mode/): Step-by-Step migration process to migrate Kafka from Zookeeper to KRaft - [The Kafka Report 007: Batch to Real-Time with Kafka, and Resources](https://oso.sh/blog/the-kafka-report-007-batch-to-real-time-with-kafka-and-resources/): We’ve started gearing up for Big Data LDN, added more info on migrating from batch to real-time to our Expert Resource Hub, and launched Kafka Support. - [Apache Kafka Security using OAuth](https://oso.sh/blog/apache-kafka-security/): Apache Kafka security has emerged as an essential component ensuring data integrity, reliability, and safe data streaming and... - [How to Migrate Kafka Cluster with Limited Downtime](https://oso.sh/blog/how-to-migrate-kafka-cluster/): How to Migrate Kafka Cluster with Limited Downtime? Migrating Apache clusters can be complex, especially when moving large amounts of data... - [The future of Data Streaming using shared data products](https://oso.sh/blog/the-future-of-data-streaming/): What is the future of data streaming and how it can revolutionise the way companies share and collaborate with their partners? - [🚀 Big Data LDN X Kafka Meetup London Special Event 🚀](https://oso.sh/blog/the-kafka-meetup-big-data-ldn-september-2023/): Join us for a special event at Big Data LDN as we delve into the fascinating world of Fast Data and Apache Kafka. - [How to manage schemas in Apache Kafka](https://oso.sh/blog/how-to-manage-schemas-in-apache-kafka/): Manage schemas in Apache Kafka? The concept of data contracts is relatively new, how can they be used for long-term schema management. - [5 tips on designing event-driven architectures](https://oso.sh/blog/designing-event-driven-architectures/): Designing event-driven architectures is becoming increasingly popular as organisations strive to build scalable and resilient systems. - [Tips for moving from Batch to Real Time data streaming](https://oso.sh/blog/tips-for-moving-from-batch-to-real-time-data-streaming/): Our solution supports you in the journey of migrating a batch to real time streaming data pipeline using Apache Kafka. - [5 Powerful Steps to Real-Time Data Magic with DBT Transformation](https://oso.sh/blog/dbt-for-real-time-data/): How to use DBT for real time data transformation and analytics? DBT focuses on the T in ETL... - [Batch vs Real time stream processing: A beginners guide ](https://oso.sh/blog/batch-vs-real-time-stream-processing/): What is the difference between batch vs real time stream processing? Both approaches have their own advantages and challenges - [Kafka Streams Rebalances and Assignments 101](https://oso.sh/blog/kafka-streams-101/): OSO has deployed its fair share of Kafka Streams applications, rebalances and assignments are part and parcel of this architecture - [How to avoid configuration drift across multiple Kafka environments using GitOps](https://oso.sh/blog/configuration-drift-management/): The best practices and ideas on how we maintain consistent configurations and avoid configuration drift across multiple Kafka environments. - [How to deploy Kafka consumers on Kubernetes](https://oso.sh/blog/kafka-consumers-on-kubernetes/): You may ask the question on how to deploy Kafka consumers on Kubernetes. Distributed systems, like Kafka, can be challenging to manage... - [Latest Kafka Industry Trends, Kafka Playbook, Kafka Resources [ July 2023 ] ](https://oso.sh/blog/latest-kafka-industry-trends-kafka-playbook-kafka-resources-july-2023-2/): Welcome to the Kafka Report. We are excited to bring you the latest insights and updates on the world of... - [How to build a custom Kafka Streams Statestores](https://oso.sh/blog/kafka-streams-statestores/): Building Kafka Streams Statestores? There are a number of challenges and opportunities of building your own state source. - [Bringing Kafka Without Zookeeper Into Production](https://oso.sh/blog/kafka-without-zookeeper/): The time is finally here where we can run Kafka without ZooKeeper. We will discuss the limitations of using ZooKeeper for replication. - [Kafka as a service: All You Need To Know](https://oso.sh/blog/kafka-as-a-service-all-you-need-to-know/): What is Kafka as a Service? Built in or purchased Kafka Services? - [What is Apache Kafka? The Ultimate Kafka Guide ](https://oso.sh/blog/what-is-apache-kafka/): What is Apache Kafka? To keep up with other companies in real-time processing, you need a powerful and distributed event-driven architecture like Apache Kafka. - [Kafka Streams Memory Best Practices](https://oso.sh/blog/kafka-streams-memory-best-practices/): Joining data streams is complex, there are many techniques and best practices for joining these data Kafka streams memory, we... - [The new consumer rebalance protocol KIP-848](https://oso.sh/blog/the-new-consumer-rebalance-protocol-kip-848/): The consumer rebalance protocol (KIP-848) is an important aspect of Apache Kafka that ensures the fair distribution of partitions. - [Reducing the TCO of Kafka on public cloud](https://oso.sh/blog/kafka-on-public-cloud-reducing-the-tco-of-kafka/): How to reduce the TCO of Kafka on public cloud? Optimising costs and increasing efficiencies are top priorities for organisations. - [Introduction to migrating to batch processes to Kafka](https://oso.sh/blog/introduction-to-migrating-to-batch-processes-to-kafka/): This article is an introduction to migrating to batch processes to Kafka. Migrating a batch system can be a complex and challenging endeavour. - [Kafka Streams vs Flink](https://oso.sh/blog/kafka-streams-vs-flink/): When it comes to Kafka Streams vs Flink both frameworks offer powerful capabilities for processing real-time data. - [End-to-End Tracing in Event Driven Architectures](https://oso.sh/blog/end-to-end-tracing-in-event-driven-architectures/): It is crucial to have end-to-end tracing in event driven architectures to understand the flow of messages and ensure proper processing. - [Real-time Push APIs Using Kafka ](https://oso.sh/blog/real-time-push-apis-using-kafka/): We all love real world examples of how to leverage the power of Kafka, let's explore the concept of building real-time push APIs using Kafka! - [Kafka Cruise Control 101](https://oso.sh/blog/kafka-cruise-control/): Kafka Cruise Control from LinkedIn is a powerful open source tool that helps manage and optimise Apache Kafka clusters. We... - [Apache Kafka Common Mistakes and Bad Practices](https://oso.sh/blog/apache-kafka-common-mistakes/): At OSO we have configured, deployed and administered 100's of Kafka clusters, we'll outline some of the Apache Kafka common mistakes. - [Kafka for End to End Analytics](https://oso.sh/blog/kafka-for-end-to-end-analytics/): We are going to learn how to build a Kafka for End to End Analytics. We want to share our experience of using Apache Kafka to enhance analytics. - [Kafka performance best practices for monitoring and alerting](https://oso.sh/blog/kafka-performance-best-practices/): We will discuss some best Kafka performance practices for monitoring and improving the performance of Kafka. - [Apache Kafka: How to reprocess messages in Kafka](https://oso.sh/blog/reprocess-messages-in-kafka/): Let’s explore the patterns on how to reprocess messages in Kafka. When working with Kafka, it is important to handle... - [Kafka Performance 101](https://oso.sh/blog/kafka-performance-101/): Explore the factors that impact the Kafka performance from an infrastructure perspective. Let's dive in to Kafka performance! - [Apache Kafka Guide: A Beginner's Guide](https://oso.sh/blog/apache-kafka-guide/): Welcome to the Apache Kafka Guide, your ultimate resource for mastering the power of Apache Kafka. Explore Apache Kafka! - [Understanding Apache Kafka: When NOT to Use Apache Kafka?](https://oso.sh/blog/understanding-apache-kafka/): Understanding Apache Kafka is hard. When to use or when not to use Apache Kafka? What is not Kafka and what is Kafka? - [Kafka Clusters: Achieving Load Balancing](https://oso.sh/blog/load-balancing-in-kafka-clusters/): Load balancing is a critical aspect of optimising performance and cost efficiency in Kafka clusters. Learn how! - [Kafka Updates in June 2023](https://oso.sh/blog/kafka-updates-june-2023/): The Latest kafka updates of June in this Kafka Report! Read to get the latest information about AWS Summit 2023, Kafka Meetup London. - [Protecting Kafka Cluster: Kafka before client - Putting the guardrails in place for Apache Kafka clients](https://oso.sh/blog/protecting-kafka-cluster/): What is Apache Kafka Cluster? What is the importance of protecting Kafka Cluster? - [Kafka: Data Mesh and Kafka Streams | OSO](https://oso.sh/blog/kafka-streams-kafka-meet-up-london-april-2023/): Discover a practical guide to building a Data Mesh on Apache Kafka and implementing a Kafka Streams microservices architecture. - [Kafka MeetUp London - June](https://oso.sh/blog/the-kafka-meetup-june-2023/): The upcoming Kafka Meetup from OSO and Sainsbury's June 2023 - [Kafka Summit London 2023: Life Hacks, Best Moments!](https://oso.sh/blog/kafka-summit-london-2023/): Dive into what happened in Kafka Summit London 2023! The latest from the world of Kafka and event streaming in May 2023 - [Looking back at Kafka Summit London 2023](https://oso.sh/blog/looking-back-at-kafka-summit-london-2023/): A full recap of the Kafka Summit London event, breakout sessions and opening keynote from Jay Kreps by our CTO,... - [Kafka Summit London [OSO's 7-Step Prep Guide] ](https://oso.sh/blog/kafka-summit-london-2023-osos-7-step-prep-guide/): 7 step guide to Kafka Summit London. What to do, certifications and trainings, network and connecting and so on! - [Debezium Updates and Ansible Updates](https://oso.sh/blog/debezium-updates-ansible-updates/): In this latest Kafka report, we talk about Debezium Updates and Ansible Updates and Success Stories. - [4 Part Guide to Apache Kafka](https://oso.sh/blog/guide-to-apache-kafka/): Here is your Guide to Apache Kafka. Learn more about Kafka updates, Kafka events and Kafka Architecture. - [Master Apache Kafka 101](https://oso.sh/blog/apache-kafka-101/): Learn Apache Kafka - A comprehensive introduction to the fundamentals and components of this distributed streaming platform. - [How to run Kafka on Kubernetes: 5 easy steps using AWS](https://oso.sh/blog/running-kafka-on-kubernetes/): How To Run Kafka on AWS. A 5-step guide to setting up a simple Apache Kafka cluster suitable for a test environment. - [How to Find an Optimal Kafka SetUp - Apache Kafka Report 001](https://oso.sh/blog/optimal-kafka-setup/): Find the optimal Kafka setup: Explore this kafka-docker-playground repository to experiment with hundreds of possible Kafka Docker configurations. - [Presentations from Kafka Meet Up London January 2023](https://oso.sh/blog/presentations-from-kafka-meet-up-london-january-2023/): A Practical Guide: Building a Data Mesh on Apache Kafka by Neil Avery and Rethinking latency SLAs with Apache Kafka... - [Confluent Expert: Elevate Your Kafka Projects to the Next Level Now](https://oso.sh/blog/confluent-expert-elevating-kafka/): Our Certified Kafka Confluent experts have the experience and knowledge to produce impactful enterprise business outcomes. - [Looking back at Big Data LDN 2022](https://oso.sh/blog/looking-back-at-big-data-ldn-2022/): Our CTO, Sion Smith, shares his thoughts and reflections from Big Data LDN 2022, with a healthy slice of Data... - [Event Driven Architecture: A Simple Guide](https://oso.sh/blog/event-driven-architecture-a-simple-guide/): This is a simple guide to Event Driven Architecture which are a popular way to build highly scalable, loosely coupled maintainable software. - [Introducing SpecMesh: Specification-driven data mesh methodology for enterprise](https://oso.sh/blog/introducing-specmesh/): Introducing SpecMesh: An innovative open-source project revolutionising data mesh methodology for enterprises. - [Apache Kafka on Azure: Set Up Confluent Kafka Without Hesitation](https://oso.sh/blog/getting-started-with-apache-kafka-on-azure/): In this guide we show you how to set up Apache Kafka on Azure. We'll share best practices and provide you with Terraform Scripts - [Watch Our Kafka Summit Talk: Offering Kafka as a Service in Your Organisation](https://oso.sh/blog/kafka-as-a-service-summit-talk/): Watch our talk at Kafka Summit about offering Kafka as a Service in your organisation - [Flux 2 Migration - Watch Our Insightful Talk at GitOps Days 2022](https://oso.sh/blog/gitops-days-flux-2/): One of our DevOps Engineers, Josh Callis, was invited as a guest speaker at GitOps Days! He shares his learnings... - [Stay contracting or build your own business?](https://oso.sh/blog/stay-contracting-or-build-your-own-business/): Not so fast. When Rich and I shifted from contracting in the tech and DevOps space to launching our own... - [OSO Is Rebranding!](https://oso.sh/blog/oso-is-rebranding/): Here’s the scoop: OSO are rebranding to focus on emerging technologies. So, what does that mean? We’re removing DevOps from... - [AWS re:Invent - Stacking The Odds In Your Favour](https://oso.sh/blog/aws-reinvent-stacking-odds-your-favour/): So the hottest conference is hitting Las Vegas once again. AWS re:Invent is about to descend on the Entertainment Capital... - [OSO Has Successfully Achieved AWS Select Partner Status](https://oso.sh/blog/oso-has-achieved-aws-select-partner-status/): The OSO team are delighted to announce, we are now an Amazon Web Services (AWS) Select Consulting Partner. This accreditation... --- # # Detailed Content ## Pages > Deploy enterprise-grade Apache Kafka on your infrastructure with full management. Eliminate operational complexity while maintaining complete control. 24/7 expert support included. - Published: 2025-08-20 - Modified: 2026-02-26 - URL: https://oso.sh/fully-managed-kafka/ --- - Published: 2023-06-22 - Modified: 2023-06-22 - URL: https://oso.sh/download-kafka-as-a-service-white-paper/ --- > Explore how OSO helps companies migrate their streaming data processing from Apache Kafka to Amazon MSK, to reduce TCO and manage clusters more efficiently. - Published: 2023-06-02 - Modified: 2026-02-26 - URL: https://oso.sh/technologies/apache-kafka-migration-to-amazon-msk/ --- > Enterprise Kafka support from the experts of Apache Kafka - Scale, optimise and secure your data platform with OSO - Published: 2023-05-23 - Modified: 2026-02-26 - URL: https://oso.sh/kafka-support/ --- > Learn about Apache Kafka - a distributed streaming platform that enables companies to build scalable, fault-tolerant, and real-time data pipelines - Published: 2023-03-23 - Modified: 2026-02-26 - URL: https://oso.sh/technologies/kafka/ --- > Maximise the Value of Streaming Technologies with Our World-Leading Kafka Expert Consultants and Specialised Kafka Services - Published: 2023-01-19 - Modified: 2026-02-26 - URL: https://oso.sh/kafka-services/ --- > We deliver Confluent advisory and implementation services to accelerate your data platform - Published: 2022-03-09 - Modified: 2026-02-26 - URL: https://oso.sh/technologies/confluent/ --- - Published: 2022-03-09 - Modified: 2026-02-26 - URL: https://oso.sh/technologies/ --- - Published: 2021-09-16 - Modified: 2021-09-16 - URL: https://oso.sh/solutions/pedigree/ --- - Published: 2021-09-15 - Modified: 2024-05-18 - URL: https://oso.sh/dev-tools/ --- > We understand different industries have different requirements when it comes to real-time data. Different industries have different requirements when it comes to real-time data. We build sector specific solutions that meet your specific challenges. - Published: 2021-09-15 - Modified: 2026-02-26 - URL: https://oso.sh/solutions/ --- - Published: 2021-09-15 - Modified: 2021-09-15 - URL: https://oso.sh/solutions/tableau-deployment-library/ --- > OSO help teams for building event driven applications on Kafka to enable real-time access. - Published: 2021-09-15 - Modified: 2026-02-26 - URL: https://oso.sh/who-we-are/ --- > Oso has Kafka Experts help teams build event driven applications and provides Kafka services. Enable real-time access to your data. - Published: 2021-08-26 - Modified: 2026-02-26 - URL: https://oso.sh/ --- --- ## Case Studies - Published: 2026-02-18 - Modified: 2026-02-26 - URL: https://oso.sh/case-studies/how-oso-helped-colliers-design-a-real-time-event-driven-crm-to-erp-integration-architecture-across-emea/ OSO provided Kafka architecture design, expert consultancy, and a comprehensive enterprise backup to support Colliers' most complex EMEA integration challenge. --- > Our client, a leading Formula One racing team, scaled their deployment of Apache Kafka to analyse race data and enable split-second strategic decisions. - Published: 2025-02-10 - Modified: 2026-02-26 - URL: https://oso.sh/case-studies/oso-helps-formula-1-racing-team-introduce-kafka-architecture-for-2026/ In a sport where every millisecond counts, OSO helped our F1 client scale Kafka architecture to stream competitive data from their cars on the racetrack. --- > Explore how Kafka architecture in manufacturing transformed Scania Group's factory systems at its new battery operations plant in Södertälje, Sweden. - Published: 2024-06-30 - Modified: 2024-07-10 - URL: https://oso.sh/case-studies/kafka-architecture-in-manufacturing/ Explore how Kafka architecture in manufacturing transformed Scania Group's factory systems at its new battery operations plant in Södertälje, Sweden. --- > Production-grade Kafka platform build that helped Lloyds launch new banking products and enhance its customer experience - Published: 2023-11-14 - Modified: 2026-02-26 - URL: https://oso.sh/case-studies/how-we-helped-lloyds-banking-group-deploy-kafka-on-prem/ When Lloyds Banking Group needed to set up an on-premise Kafka cluster, Kafka Support helped the bank fill its gaps in expertise and build a platform aligned with its high standards for security, availability, and service. --- - Published: 2023-08-30 - Modified: 2026-02-26 - URL: https://oso.sh/case-studies/cambridge-university-kafka-support/ A real-time data pipeline powered by Apache Kafka and submission service helped Cambridge University Press synchronise its data between content management systems and offer a smoother user experience to students. --- > An Apache Kafka proof of concept helped Dufry's Head of Data, Chief Revenue Officer, and Chief Technical Officer perform better. - Published: 2023-05-25 - Modified: 2024-07-10 - URL: https://oso.sh/case-studies/dufry-apache-kafka-proof-of-concept-poc/ A Kafka proof of concept helped Dufry’s Head of Data, Chief Revenue Officer, and Chief Technical Officer perform their due diligence on the adoption of a Kafka architecture. --- > After carefully analysing its options, the DfE selected a Kafka architecture to improve its digital infrastructure and generate new value from its data - Published: 2023-03-16 - Modified: 2026-02-26 - URL: https://oso.sh/case-studies/department-for-education-kafka-architecture/ After carefully analysing its options, the DfE selected a Kafka architecture to improve its digital infrastructure and generate new value from its data --- > One of OSO's Apache Kafka Best Practices. See How OSO helped Marlow's undergoing digital transformation - Published: 2023-02-07 - Modified: 2024-07-10 - URL: https://oso.sh/case-studies/apache-kafka-best-practices-marlow/ How OSO assessed Kafka best practices with a Kafka Health Check --- --- ## Blog > Learn how to eliminate the five hidden cost traps in Kafka-to-Iceberg pipelines. Reduce cloud spend by removing connectors, duplicate data, and over-provisioned compute. - Published: 2026-02-26 - Modified: 2026-02-26 - URL: https://oso.sh/blog/how-to-reduce-kafka-to-iceberg-pipeline-costs/ Learn how to eliminate the five hidden cost traps in Kafka-to-Iceberg pipelines. Reduce cloud spend by removing connectors, duplicate data, and over-provisioned compute. --- > Learn how to build a real-time context engine for AI agents using Apache Kafka. Architecture, governance, MCP integration, and practical implementation guide. - Published: 2026-02-09 - Modified: 2026-02-09 - URL: https://oso.sh/blog/how-to-build-real-time-context-engine-ai-agents-apache-kafka/ --- > The Kafka ecosystem is shifting dramatically in 2026. OSO engineers break down what's changing, what it means for your architecture, and how to prepare. - Published: 2026-01-09 - Modified: 2026-01-09 - URL: https://oso.sh/blog/what-every-kafka-engineer-needs-to-know-2026/ --- > Learn how Kafka's binary protocol enables partition remapping proxies to cut managed Kafka costs by 90% whilst remaining completely transparent to applications. - Published: 2025-12-19 - Modified: 2025-12-19 - URL: https://oso.sh/blog/kafka-protocol-internals-partition-remapping-proxy/ --- > Learn how to back up your Kafka cluster properly with point-in-time recovery and consumer offset restoration. Discover why replication alone isn't enough. - Published: 2025-12-09 - Modified: 2025-12-12 - URL: https://oso.sh/blog/how-to-back-up-kafka-cluster-point-in-time-recovery/ --- > Current Summit New Orleans 2025: A critical look at the biggest Kafka conference. What worked, what didn't, and where data streaming is heading. - Published: 2025-11-13 - Modified: 2025-11-13 - URL: https://oso.sh/blog/current-summit-new-orleans-2025-review/ --- > Apache Flink is overkill for AI agents. Learn why Akka's purpose-built framework delivers simpler, more maintainable agentic systems in production. - Published: 2025-10-18 - Modified: 2025-10-20 - URL: https://oso.sh/blog/flink-vs-akka-agentic-ai-comparison/ --- > Discover how to build scalable multi-region orchestration using Apache Kafka's pull-based architecture. Real-world patterns from production systems. - Published: 2025-10-16 - Modified: 2025-10-24 - URL: https://oso.sh/blog/kafka-multi-region-orchestration-architecture-pull-based-workflows/ --- > Learn how disaggregated architectures using object storage cut Kafka costs by 71%. Real-world strategies for migrating from EBS to S3 at scale. - Published: 2025-10-03 - Modified: 2025-10-03 - URL: https://oso.sh/blog/kafka-object-storage-cost-optimization-disaggregated-architecture/ --- > Learn how transaction-aware Change Data Capture solves consistency problems in Kafka pipelines when database transactions span multiple tables. - Published: 2025-09-24 - Modified: 2025-09-24 - URL: https://oso.sh/blog/apache-kafka-cdc-transaction-consistency-debezium-guide/ --- > Learn how to unify multi-platform Kafka operations with proven strategies for hybrid deployments, monitoring consolidation, and seamless application management. - Published: 2025-09-10 - Modified: 2025-10-15 - URL: https://oso.sh/blog/how-to-manage-hybrid-kafka-deployments-guide/ --- > Master Kafka Streams Processor API for enterprise applications. Learn when to move beyond DSL for complex state management and advanced streaming use cases. - Published: 2025-09-03 - Modified: 2025-10-15 - URL: https://oso.sh/blog/how-to-use-kafka-streams-processor-api-for-complex-enterprise-applications/ --- > Learn how unified schema registry architecture eliminates cross-system compatibility failures in enterprise data ecosystems with early validation and seamless schema evolution. - Published: 2025-09-01 - Modified: 2025-09-01 - URL: https://oso.sh/blog/unified-schema-registry-cross-system-compatibility-enterprise/ --- > Learn how KIP-714 transforms Kafka monitoring with centralized client telemetry. Discover how to eliminate monitoring complexity and catch misconfigurations automatically. - Published: 2025-08-28 - Modified: 2025-08-28 - URL: https://oso.sh/blog/kip-714-kafka-client-metrics-centralized-monitoring-solution/ --- > Learn how to eliminate 5-day Kafka delays with self-service automation. Complete guide to GitOps workflows, governance, and enterprise scaling. - Published: 2025-08-21 - Modified: 2025-08-21 - URL: https://oso.sh/blog/complete-guide-self-service-kafka-operations-enterprise/ --- > Discover 7 essential Apache Kafka CLI tools that transform operations in 2025. From kcat to topicctl, master the command-line tools every Kafka engineer needs. - Published: 2025-08-07 - Modified: 2025-08-07 - URL: https://oso.sh/blog/7-must-have-cli-tools-every-apache-kafka-engineer-should-know-in-2025/ --- > Discover how Apache Kafka's KRaft protocol eliminates Zookeeper dependencies and delivers 8x performance improvements. Complete guide to KRaft architecture, migration, and real-world benefits. - Published: 2025-07-29 - Modified: 2025-07-29 - URL: https://oso.sh/blog/apache-kafkas-kraft-protocol-how-to-eliminate-zookeeper-and-boost-performance-by-8x/ --- > Build high-performance Kafka Streams apps that scale beyond partition limits using async processing. Learn techniques that delivered 6x performance gains in production. - Published: 2025-07-19 - Modified: 2025-07-19 - URL: https://oso.sh/blog/how-to-build-high-performance-kafka-streams-applications-that-scale-beyond-partition-limits/ --- - Published: 2025-07-06 - Modified: 2025-07-06 - URL: https://oso.sh/blog/productionise-kafka-streams-state-stores-how-unbundled-architecture-transforms-kafka-streams-operations/ --- > Kafka Consumer 4.0 transforms rebalancing with incremental server-side processing. No more slow member bottlenecks or two-phase synchronisation delays. - Published: 2025-06-13 - Modified: 2025-06-21 - URL: https://oso.sh/blog/kafka-consumer-4-0-new-rebalance-protocol/ --- > Master Apache Kafka disaster recovery with proven strategies from OSO engineers handling 300M+ daily transactions. Learn MirrorMaker setup, active-active patterns, and failover procedures with code examples. - Published: 2025-05-14 - Modified: 2025-06-13 - URL: https://oso.sh/blog/building-bulletproof-disaster-recovery-for-apache-kafka-a-field-tested-architecture/ --- - Published: 2025-04-30 - Modified: 2025-04-30 - URL: https://oso.sh/blog/kafka-queues-in-apache-kafka-4-0-via-share-groups/ --- > Implementing the outbox pattern, you need a reliable mechanism to move messages from your outbox table to Kafka. - Published: 2025-04-29 - Modified: 2025-04-29 - URL: https://oso.sh/blog/apache-kafka-outbox-pattern/ --- > Successful KRaft migration begins with thorough preparation. Read these 5 steps to migrate successfully. - Published: 2025-04-27 - Modified: 2025-04-28 - URL: https://oso.sh/blog/guide-to-zookeeper-to-kraft-migration/ --- > By challenging the standard pattern and designing a solution tailored to our domain, we created a more flexible, maintainable, and developer-friendly system. - Published: 2025-04-25 - Modified: 2025-04-25 - URL: https://oso.sh/blog/multi-message-types-in-kafka-topics/ --- - Published: 2025-01-21 - Modified: 2025-04-25 - URL: https://oso.sh/blog/how-to-design-kafka-events-and-event-streams/ --- - Published: 2025-01-17 - Modified: 2025-04-25 - URL: https://oso.sh/blog/how-to-secure-your-apache-kafka-cluster/ Apache Kafka offers several methods for authenticating Kafka clients to brokers. This guide aims to showcase these methods, provide guidance on enabling and configuring authentication options, and assist in determining the most suitable approach for your specific use case. --- > Configuring Apache Kafka consumers is confusing! This table helps us the remember what each parameter means. - Published: 2025-01-16 - Modified: 2025-01-16 - URL: https://oso.sh/blog/kafka-consumer-configuring-cheatsheet/ --- > Kafka terminology can be confusing! Here is everything you need to know on one page! - Published: 2025-01-16 - Modified: 2025-01-16 - URL: https://oso.sh/blog/apache-kafka-glossary/ --- > As we kick off January, it’s time to share OSO’s annual trends and predictions for data streaming technology. Here are our top three predictions for 2025. - Published: 2025-01-02 - Modified: 2025-01-05 - URL: https://oso.sh/blog/the-future-of-streaming-data/ --- > This year, OSO invested in growing the London community. We hosted nine meetups, brought together 470 new members passionate about technology and innovation, produced 19 articles and blog posts, shared insights through three podcasts, and developed 35 code repositories. - Published: 2024-12-18 - Modified: 2024-12-28 - URL: https://oso.sh/blog/apache-kafka-2024-review/ --- > This guide will help developers understand the essential principles of Apache Kafka, and will prepare them for the CCDAK exam. - Published: 2024-11-12 - Modified: 2024-11-15 - URL: https://oso.sh/blog/confluent-certified-developer-for-apache-kafka-study-guide/ --- > Apache Kafka now leverages cloud object storage to provide cost-effective and scalable storage for Kafka data. - Published: 2024-11-04 - Modified: 2024-11-04 - URL: https://oso.sh/blog/serverless-tiered-storage-in-apache-kafka/ --- > How to build streaming architecture using Kafka to feed model training - Published: 2024-10-31 - Modified: 2024-10-31 - URL: https://oso.sh/blog/machine-learning-with-apache-kafka/ --- > A Kafka cluster is a group of broker nodes working together to provide, scalability, availability, and fault tolerance. - Published: 2024-10-24 - Modified: 2024-10-24 - URL: https://oso.sh/blog/what-are-the-key-concepts-of-apache-kafka/ --- > Learn which configuration values to chance on the Producer API to achieve high performance on your Apache Kafka producers - Published: 2024-06-27 - Modified: 2024-06-28 - URL: https://oso.sh/blog/kafka-producer-best-practices/ --- > Best practice tips and examples from Kafka Experts on how to handle Kafka errors in your applications - Published: 2024-06-21 - Modified: 2024-06-21 - URL: https://oso.sh/blog/5-kafka-error-handling-tips/ --- > Step-by-Step migration process to migrate Kafka from Zookeeper to KRaft - Published: 2024-06-05 - Modified: 2024-06-06 - URL: https://oso.sh/blog/how-to-migrate-kafka-to-kraft-mode/ --- > We’ve started gearing up for Big Data LDN, added more info on migrating from batch to real-time to our Expert Resource Hub, and launched Kafka Support. - Published: 2023-08-31 - Modified: 2025-05-07 - URL: https://oso.sh/blog/the-kafka-report-007-batch-to-real-time-with-kafka-and-resources/ --- > Apache Kafka security has emerged as an essential component ensuring data integrity, reliability, and safe data streaming and... - Published: 2023-08-21 - Modified: 2024-05-06 - URL: https://oso.sh/blog/apache-kafka-security/ --- > How to Migrate Kafka Cluster with Limited Downtime? Migrating Apache clusters can be complex, especially when moving large amounts of data... - Published: 2023-08-18 - Modified: 2024-05-06 - URL: https://oso.sh/blog/how-to-migrate-kafka-cluster/ --- > What is the future of data streaming and how it can revolutionise the way companies share and collaborate with their partners? - Published: 2023-08-17 - Modified: 2024-05-06 - URL: https://oso.sh/blog/the-future-of-data-streaming/ --- > Join us for a special event at Big Data LDN as we delve into the fascinating world of Fast Data and Apache Kafka. - Published: 2023-08-15 - Modified: 2024-05-06 - URL: https://oso.sh/blog/the-kafka-meetup-big-data-ldn-september-2023/ --- > Manage schemas in Apache Kafka? The concept of data contracts is relatively new, how can they be used for long-term schema management. - Published: 2023-08-08 - Modified: 2024-05-06 - URL: https://oso.sh/blog/how-to-manage-schemas-in-apache-kafka/ --- > Designing event-driven architectures is becoming increasingly popular as organisations strive to build scalable and resilient systems. - Published: 2023-08-08 - Modified: 2024-05-06 - URL: https://oso.sh/blog/designing-event-driven-architectures/ --- > Our solution supports you in the journey of migrating a batch to real time streaming data pipeline using Apache Kafka. - Published: 2023-08-08 - Modified: 2024-05-06 - URL: https://oso.sh/blog/tips-for-moving-from-batch-to-real-time-data-streaming/ --- > How to use DBT for real time data transformation and analytics? DBT focuses on the T in ETL... - Published: 2023-08-08 - Modified: 2024-05-06 - URL: https://oso.sh/blog/dbt-for-real-time-data/ --- > What is the difference between batch vs real time stream processing? Both approaches have their own advantages and challenges - Published: 2023-08-08 - Modified: 2024-05-06 - URL: https://oso.sh/blog/batch-vs-real-time-stream-processing/ --- > OSO has deployed its fair share of Kafka Streams applications, rebalances and assignments are part and parcel of this architecture - Published: 2023-08-07 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-streams-101/ --- > The best practices and ideas on how we maintain consistent configurations and avoid configuration drift across multiple Kafka environments. - Published: 2023-08-02 - Modified: 2024-05-06 - URL: https://oso.sh/blog/configuration-drift-management/ --- > You may ask the question on how to deploy Kafka consumers on Kubernetes. Distributed systems, like Kafka, can be challenging to manage... - Published: 2023-08-02 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-consumers-on-kubernetes/ --- - Published: 2023-07-31 - Modified: 2024-05-06 - URL: https://oso.sh/blog/latest-kafka-industry-trends-kafka-playbook-kafka-resources-july-2023-2/ --- > Building Kafka Streams Statestores? There are a number of challenges and opportunities of building your own state source. - Published: 2023-07-31 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-streams-statestores/ --- > The time is finally here where we can run Kafka without ZooKeeper. We will discuss the limitations of using ZooKeeper for replication. - Published: 2023-07-31 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-without-zookeeper/ --- > What is Kafka as a Service? Built in or purchased Kafka Services? - Published: 2023-07-24 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-as-a-service-all-you-need-to-know/ --- > What is Apache Kafka? To keep up with other companies in real-time processing, you need a powerful and distributed event-driven architecture like Apache Kafka. - Published: 2023-07-24 - Modified: 2025-05-07 - URL: https://oso.sh/blog/what-is-apache-kafka/ --- - Published: 2023-07-18 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-streams-memory-best-practices/ --- > The consumer rebalance protocol (KIP-848) is an important aspect of Apache Kafka that ensures the fair distribution of partitions. - Published: 2023-07-17 - Modified: 2024-05-06 - URL: https://oso.sh/blog/the-new-consumer-rebalance-protocol-kip-848/ --- > How to reduce the TCO of Kafka on public cloud? Optimising costs and increasing efficiencies are top priorities for organisations. - Published: 2023-07-17 - Modified: 2024-05-18 - URL: https://oso.sh/blog/kafka-on-public-cloud-reducing-the-tco-of-kafka/ --- > This article is an introduction to migrating to batch processes to Kafka. Migrating a batch system can be a complex and challenging endeavour. - Published: 2023-07-17 - Modified: 2024-05-06 - URL: https://oso.sh/blog/introduction-to-migrating-to-batch-processes-to-kafka/ --- > When it comes to Kafka Streams vs Flink both frameworks offer powerful capabilities for processing real-time data. - Published: 2023-07-14 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-streams-vs-flink/ --- > It is crucial to have end-to-end tracing in event driven architectures to understand the flow of messages and ensure proper processing. - Published: 2023-07-14 - Modified: 2024-05-06 - URL: https://oso.sh/blog/end-to-end-tracing-in-event-driven-architectures/ --- > We all love real world examples of how to leverage the power of Kafka, let's explore the concept of building real-time push APIs using Kafka! - Published: 2023-07-14 - Modified: 2024-05-06 - URL: https://oso.sh/blog/real-time-push-apis-using-kafka/ --- - Published: 2023-07-13 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-cruise-control/ --- > At OSO we have configured, deployed and administered 100's of Kafka clusters, we'll outline some of the Apache Kafka common mistakes. - Published: 2023-07-13 - Modified: 2024-05-06 - URL: https://oso.sh/blog/apache-kafka-common-mistakes/ --- > We are going to learn how to build a Kafka for End to End Analytics. We want to share our experience of using Apache Kafka to enhance analytics. - Published: 2023-07-12 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-for-end-to-end-analytics/ --- > We will discuss some best Kafka performance practices for monitoring and improving the performance of Kafka. - Published: 2023-07-12 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-performance-best-practices/ --- - Published: 2023-07-10 - Modified: 2024-12-04 - URL: https://oso.sh/blog/reprocess-messages-in-kafka/ --- > Explore the factors that impact the Kafka performance from an infrastructure perspective. Let's dive in to Kafka performance! - Published: 2023-07-05 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-performance-101/ --- > Welcome to the Apache Kafka Guide, your ultimate resource for mastering the power of Apache Kafka. Explore Apache Kafka! - Published: 2023-07-05 - Modified: 2024-05-06 - URL: https://oso.sh/blog/apache-kafka-guide/ --- > Understanding Apache Kafka is hard. When to use or when not to use Apache Kafka? What is not Kafka and what is Kafka? - Published: 2023-07-04 - Modified: 2024-05-06 - URL: https://oso.sh/blog/understanding-apache-kafka/ --- > Load balancing is a critical aspect of optimising performance and cost efficiency in Kafka clusters. Learn how! - Published: 2023-06-30 - Modified: 2024-08-31 - URL: https://oso.sh/blog/load-balancing-in-kafka-clusters/ --- > The Latest kafka updates of June in this Kafka Report! Read to get the latest information about AWS Summit 2023, Kafka Meetup London. - Published: 2023-06-30 - Modified: 2025-05-07 - URL: https://oso.sh/blog/kafka-updates-june-2023/ --- > What is Apache Kafka Cluster? What is the importance of protecting Kafka Cluster? - Published: 2023-06-23 - Modified: 2024-05-06 - URL: https://oso.sh/blog/protecting-kafka-cluster/ --- > Discover a practical guide to building a Data Mesh on Apache Kafka and implementing a Kafka Streams microservices architecture. - Published: 2023-06-02 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-streams-kafka-meet-up-london-april-2023/ A Practical Guide: Building a Data Mesh on Apache Kafka by Neil Avery and Rethinking latency SLAs with Apache Kafka by Dare Famuyiwa --- - Published: 2023-06-01 - Modified: 2024-05-06 - URL: https://oso.sh/blog/the-kafka-meetup-june-2023/ The upcoming Kafka Meetup from OSO and Sainsbury's June 2023 --- > Dive into what happened in Kafka Summit London 2023! The latest from the world of Kafka and event streaming in May 2023 - Published: 2023-05-31 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-summit-london-2023/ The latest from the world of Kafka and event streaming in April 2023 --- > 7 step guide to Kafka Summit London. What to do, certifications and trainings, network and connecting and so on! - Published: 2023-05-14 - Modified: 2024-05-06 - URL: https://oso.sh/blog/kafka-summit-london-2023-osos-7-step-prep-guide/ How to make the most of Kafka Summit London's 2 days of technical sessions, industry speakers and experts, and sessions on Apache Kafka and real-time data technologies. --- > In this latest Kafka report, we talk about Debezium Updates and Ansible Updates and Success Stories. - Published: 2023-05-03 - Modified: 2024-05-18 - URL: https://oso.sh/blog/debezium-updates-ansible-updates/ The latest from the world of Kafka and event streaming in April 2023 --- > Here is your Guide to Apache Kafka. Learn more about Kafka updates, Kafka events and Kafka Architecture. - Published: 2023-03-28 - Modified: 2024-05-18 - URL: https://oso.sh/blog/guide-to-apache-kafka/ The latest from the world of Kafka and event streaming in March 2023 --- - Published: 2023-02-28 - Modified: 2024-05-06 - URL: https://oso.sh/blog/apache-kafka-101/ Learn Apache Kafka - A comprehensive introduction to the fundamentals and components of this distributed streaming platform. --- > How To Run Kafka on AWS. A 5-step guide to setting up a simple Apache Kafka cluster suitable for a test environment. - Published: 2023-02-10 - Modified: 2025-04-25 - URL: https://oso.sh/blog/running-kafka-on-kubernetes/ A 5-step guide to setting up a simple Apache Kafka cluster suitable for a test environment. We'll use Confluent for Kubernetes (CFK) and AWS EC2. --- > Find the optimal Kafka setup: Explore this kafka-docker-playground repository to experiment with hundreds of possible Kafka Docker configurations. - Published: 2023-02-09 - Modified: 2024-05-18 - URL: https://oso.sh/blog/optimal-kafka-setup/ The latest from the world of Kafka and event streaming in February 2023 --- - Published: 2023-02-02 - Modified: 2024-05-06 - URL: https://oso.sh/blog/presentations-from-kafka-meet-up-london-january-2023/ A Practical Guide: Building a Data Mesh on Apache Kafka by Neil Avery and Rethinking latency SLAs with Apache Kafka by Dare Famuyiwa --- - Published: 2022-11-29 - Modified: 2024-05-06 - URL: https://oso.sh/blog/looking-back-at-big-data-ldn-2022/ Our CTO, Sion Smith, shares his thoughts and reflections from Big Data LDN 2022, with a healthy slice of Data Mesh. --- > This is a simple guide to Event Driven Architecture which are a popular way to build highly scalable, loosely coupled maintainable software. - Published: 2022-09-26 - Modified: 2024-05-06 - URL: https://oso.sh/blog/event-driven-architecture-a-simple-guide/ Event-driven architectures are a popular way to build highly scalable, loosely coupled maintainable software. In this guide, we'll cover what event driven architecture is, introduce some common terminologies and discuss some pros and cons of using this approach. --- > Introducing SpecMesh: An innovative open-source project revolutionising data mesh methodology for enterprises. - Published: 2022-09-22 - Modified: 2024-05-06 - URL: https://oso.sh/blog/introducing-specmesh/ A specification-driven data mesh methodology for the enterprise --- > In this guide we show you how to set up Apache Kafka on Azure. We'll share best practices and provide you with Terraform Scripts - Published: 2022-09-02 - Modified: 2024-02-17 - URL: https://oso.sh/blog/getting-started-with-apache-kafka-on-azure/ In this guide we show you how to set up Apache Kafka on Azure. We'll share best practices and provide you with Terraform Scripts --- - Published: 2022-06-08 - Modified: 2024-05-18 - URL: https://oso.sh/blog/gitops-days-flux-2/ One of our DevOps Engineers, Josh Callis, was invited as a guest speaker at GitOps Days! He shares his learnings about Flux 2 migration --- > It's hard to summarise DevOps in one sentence. But in this short blog we attempt to answer the question: What is DevOps? - Published: 2016-11-16 - Modified: 2023-06-09 - URL: https://oso.sh/blog/what-is-devops/ It's hard to summarise DevOps in one sentence. But in this short blog we attempt to answer the question: What is DevOps? --- --- > For enquiries, visit https://oso.sh/contact/ or email enquiries@oso.sh ---