Enterprise

Explore how Lloyds Banking Group designed a secure and scalable environment with Kafka 24/7 support

When Lloyds Banking Group needed to set up an on-premise Kafka cluster, Kafka Support helped the bank fill its gaps in expertise and build a platform aligned with its high standards for security, availability, and service.

Explore how Lloyds Banking Group designed a secure and scalable environment with Kafka 24/7 support

A digital bank with an emphasis on the customer experience

Lloyds Banking Group, along with HSBC and Barclays, has made a name for itself as one of the UK’s largest and most well-established financial institutions. Counting around 40 million account holders as part of its network, it consistently invests in its clients: as of 2023, it had given £15 billion to social housing, 95k to small businesses and startups, £22.4 million to charity, and £14.3 billion to first-time home buyers. Fittingly, its mantra is ‘Helping Britain Prosper’. ‘We’re an organisation that’s striving to do the very best for our customers’, Lloyds Banking Group’s Mass Affluent division CEO Jo Harris explained. In recent years, as part of that mission, the bank decided to invest in open-source Kafka technology to provide its customers with better, faster service and synchronous data.


“The insights and data we gather from our products enables us to continuously improve and is absolutely vital to our performance.”

LLOYDS BANKING GROUP’S CHIEF DATA AND ANALYTICS OFFICER, RANIL BOTEJU — 


Invested in design and build a Kafka environment 

Lloyds, however, lacked deep expertise in Kafka technology, limiting its ability to set up and configure a new Kafka environment. The bank was already using Kafka, but its support costs had escalated rapidly with its old provider, and its team wasn’t confident they could build a production grade Kafka cluster without unexpected downtime or technical errors. This continuity would be critical: Lloyds served 20 million digital users, so it couldn’t afford to have any downtime in service. 

 

Security was also paramount. In Lloyds’ position as the UK’s leading digital banking company, it needed to ensure its systems remained operational throughout the build process. If the bank’s core event systems went down, its team had to be able to send instant notifications to customers. And to maintain overall compliance with UK laws, Lloyds needed to fully understand how Kafka aligned with its stringent data segregation security standards. 

Kafka 24/7 support

LLOYDS BANKING GROUP

These stakes, understandably, made Lloyds’ team cautious. ‘The use of open-source software comes with certain risks [that] need to be managed’, explained Lloyds Banking Group’s Head of Open Source Program Office Pooi Ling Cheong. If Lloyds incorrectly designed or configured the platform, the bank could lose information and intellectual property, face issues with software versioning, and inadvertently create a clunkier, less effective customer experience. 

 

As a result, rather than develop the platform in-house, Lloyds Banking Group decided to seek out a third-party Kafka consultancy that could counsel and support it through the process of designing, testing, and configuring a production ready Kafka environment. Its Kafka project had more at stake than data—the outcome could affect its customers’ loyalty and trust.  


“We work with suppliers who take time to understand our business and look for ways to bring insight and enhancements.”

LLOYDS BANKING GROUP — 


Configuring Kafka with professional support 

To launch the new platform, Lloyds chose OSO’s Kafka Support team, whose professional experience could fill its technical gaps and help it set up a fully operational Kafka environment. Once on board, OSO handled the platform’s design, architecture, and configuration whilst providing advice and operational support throughout the whole deployment phase. 

 

This joint ownership helped the bank comply with important security standards. OSO’s consulting team guaranteed service continuity, conducting extensive performance testing and health checks before the bank’s Kafka environment went live, and adhered to banks security standards and platform availability and resilience. ‘Throughout, it was important that we followed best practices for Apache Kafka architecture and fault-tolerant design’, Sion Smith said. ‘We even implemented production-grade run books for use in disaster recovery’. 

Lloyds Banking Group’s Kafka architecture

Five months later, Lloyds’ newly configured Kafka environments were running smoothly. The dev/test and production Kafka environments had created a centralised platform, one where, Sion Smith explained, “all functions of the retail bank could produce and consume real-time data.” The build complied with relevant security standards, adhered to the industry’s best practices for data sovereignty and development, and incorporated extensive safety, optimisation, and health checks. What’s more, Lloyds had wrapped the project with zero downtime or interruptions in service.  

 

High availability was achieved through architectural best practices, harnessing the power to automation and a multi-DC strategy in an active/passive configuration, and OSO implemented the platform in a way where it could scale in the future to achieve higher storage and throughput.

 

The project was deemed a success. “Sion and the OSO team are rock star engineers”, Lloyds’ Client Data and Analytics Platform Lab Product Owner Abdur Rahman said. “We love working with them.”

Rising above the banking competition 

Now, in a world where customers expect easy, instant connections between their card transactions and banking services, Lloyds Banking Group wants to use Kafka to firmly position itself as a trusted leader in open-source technology and data. 

In September 2023, it launched Lloyds Bank 360 Solution for 50,000 pilot customers, an online option that helps clients manage their money, invest their savings, and learn about ‘confusing financial terms and products’. Lloyds 360 is part of a broader strategy to differentiate itself in the market and delineate itself as the UK’s most attractive and competitive digital bank.


Want to configure an on-premise Kafka environment? 

Explore OSO’s Kafka Services and Kafka Support

4 OSO kafka services are Kafka consultancy, streaming applications, Kafka Infrastructure and Professional Kafka Support

About Kafka Support 

OSO’s Kafka Support team helps digital banks and financial institutions design and configure Kafka environments. OSO experts provide support during the architecture, setup, and configuration stages of a project, as well as ongoing operational support covered by Service Level Agreements. All service includes extensive health checks, security and bug reports, and communication with bank representatives about upgrades and updates. To determine if your team would be a good candidate, contact OSO’s team to schedule a complimentary call. 

 

Read More

Get started with emerging technologies today

Have a conversation with one of our experts to discover how we can work with you to adopt emerging technologies to keep your business growing.

Book a call