Learn how OSO’s Kafka support helped Cambridge University Press migrate to a modern digital learning system

The benefits of real-time data in higher education

• Easily track student progress on activities, assessments, quizzes, and learning modules. 
• Synchronise data across platforms and content management systems. 
• Provide staff and students with consistent, accurate information and dashboards. 
• Make strategic data driven decisions with confidence. 
• Build trust in your organisation’s ability to offer a modern learning experience. 
• Scale your platform to reach a variety of students around the globe.

A publishing company with a global mission

The mission of Cambridge University Press, part of the fourth-oldest university in the world, is to ‘spread knowledge, spark curiosity, and aid understanding’. Over the years, Cambridge University has delved into research spanning shellfish and global food security, uncertainty and machine learning systems, robots, the metaverse, and genetics variants. The Press complements this research by printing academic research, as well as English language learning resources, quizzes, and assessments for students hoping to expand their knowledge by attaining mastery of a second language. Altogether, the institution serves more than 20,000 students from 140+ countries, as well as governments, private citizens, and nonprofit organisations.

Pursuing a modern learning environment

Cambridge University Press also prides itself on the quality of the academic and educational resources it provides via Contentful. As it expanded its library of resources and served a growing number of global learners, however, University Press leaders realised that their current system needed to meet student expectations and provide staff with accurate data with which to make strategic decisions. Contentful and the Press’s internal systems frequently reflected different data values, which meant no one knew which version was correct. Staff members knew the data could be inaccurate, preventing them from making strategic decisions.

To equip the next generation of leaders with the best possible educational resources, Cambridge University Press needed to be able to easily update its syllabi, e-learning modules, quizzes, tests, and assessments, and ensure those changes rippled out across all its platforms to give students a streamlined learning experience. The Press also needed to be able to track students’ progress as they progressed through digital courses, checking off exercises, quizzes, and learning assessments—otherwise, students would have a suboptimal course experience.

Adopting a real-time data pipeline

To implement a solid and reliable data pipeline and submission service, Cambridge University Press selected OSO’s Kafka Support, a dedicated Kafka enterprise service that offered Kafka expertise at a fraction of the cost of hiring a full-time engineer on staff. The pipeline synced data between Contentful, the Press’s content management system, and Cambridge University Press’s internal systems. As Sion Smith, OSO’s CTO, explained: ‘[Cambridge] introduced Kafka because they needed to make sure they could share content between multiple systems and access the most up-to-date state of their content’. 

OSO’s Kafka Support gave the Cambridge team on-demand access to a dedicated Apache Kafka engineer, Slack chats with our experts, and supported the validation and monitoring of the pipeline’s initial setup. OSO’s Kafka experts also conducted rigorous testing at each stage of the system’s development to make sure everything ran smoothly, performing user tests, integration tests, and end-to-end testing.

Adopting a real-time data pipeline

To implement a solid and reliable data pipeline and submission service, Cambridge University Press selected OSO’s Kafka Support, a dedicated Kafka enterprise service that offered Kafka expertise at a fraction of the cost of hiring a full-time engineer on staff. The pipeline synced data between Contentful, the Press’s content management system, and Cambridge University Press’s internal systems. As Sion Smith, OSO’s CTO, explained: ‘[Cambridge] introduced Kafka because they needed to make sure they could share content between multiple systems and access the most up-to-date state of their content’. 

OSO’s Kafka Support gave the Cambridge team on-demand access to a dedicated Apache Kafka engineer, Slack chats with our experts, and supported the validation and monitoring of the pipeline’s initial setup. OSO’s Kafka experts also conducted rigorous testing at each stage of the system’s development to make sure everything ran smoothly, performing user tests, integration tests, and end-to-end testing.

Paving the way for international projects

Following the Kafka best practice and deploying the architecture blueprints OSO provided are the next step for the Press’s large-scale educational projects. Since its inception, the Press has helped Mexico’s government launch its National Program for English in Basic Education, collaborated with Kazakh leaders to build a better national textbook and educational publishing system in Kazakhstan, architected Malaysia’s Education Blueprint through 2025, and reviewed and revised Ethiopia’s K-12 curriculum. With its new data pipeline and submission service, the Press will be equipped to launch more of these international, data-intensive projects moving forward.

Setting a new global standard

Cambridge University Press’s new initiative isn’t just about modernising its data infrastructure; it’s about reimagining the future of global education. With accurate and real-time data at their fingertips powered by Apache Kafka, leaders at the Press aren’t just ensuring it offers the latest digital content to students across the globe—they’re reshaping the way universities and educational institutions perceive and handle their digital resources. This data-driven approach will allow them to be more proactive in content creation, predict educational trends, and provide unparalleled learning.  In short, the new pipeline isn’t just a technological upgrade. It’s a way to fulfil the Press’s mission to spark global curiosity, understanding, and innovation.

About Kafka Support 

Kafka Support is a monthly dedicated enterprise support service for Kafka infrastructure. Instead of hiring an on-staff engineer, companies can access advanced Kafka expertise for a fraction of the cost. Teams receive a guaranteed response to Kafka-related requests in two business days or less, with the option to message Kafka experts in real time via Slack and request support on-demand. To get started, contact OSO’s team to schedule a complimentary consultation. 

Read More

• How designing the right Kafka architecture helped The Department for Education become a data-synchronous organisation 
• Digital tools for real-time data collection in education — Brookings
• Tips for moving from batch to real-time data streaming
• What is Apache Kafka? The Ultimate Guide 
• Kafka Support 
• Big Data LDN X Kafka Meetup London Special Event