Event Driven Architecture: A Simple Guide

Event-driven architectures are a popular way to build highly scalable, loosely coupled maintainable software. In this guide, we’ll cover what event driven architecture is, introduce some common terminologies and discuss some pros and cons of using this approach.

Event Driven Architecture: What is an Event?

An event is something that happens. In the world of software design, an event is defined as “a significant change in state”. For example; a user clicks a button, a sensor detects a change in temperature or a user clicks send on an email.

Events can be generated by humans, by computers or by external systems.

A Simple Real-time Example

Consider this very over-simplified Deliveroo process (other food delivery services are available!)

1. You order a meal on the app
2. A chef is notified of the order and starts cooking
3. When the meal is nearly ready, the chef tells the Deliveroo system and a nearby driver is notified
4. When the driver collects the meal, you are notified the meal is en route
5. And finally, you are sent a notification just before the driver arrives outside

Throughout the process, people (and systems) are triggering events (producers) and others are being told about events that have happened (consumers). This is all coordinated by the ‘broker’, which sits within the Deliveroo software. These events are happening in real-time and in an async manner. In other words, the driver is not just waiting for that particular chef, they are busy delivering other meals.

The alternative to an event-driven architecture would be a request-driven approach. In our example, this would be more like the customer calling up every few minutes to ask where their order is.

The Building Blocks of Event Driven Architecture

Producers and Consumers

At each end of an event-driven application, you have two things: producers and consumers. Producers generate events. Consumers receive and process them, often in real time. These are typically deployed as microservices.

In our example, a chef is acting as a producer when they notify the Deliveroo system that the food is nearly cooked. The delivery driver is acting as a consumer when they get notified to collect the food.

Brokers

To pass events from producers to consumers, you need a broker. A broker is responsible for logging each event that is sent to it – in order. These events are then passed to any consumers that are subscribed to that type of event. This is often called an event log. Each broker has a maximum amount of events they can process. If we need to increase this limit we will need to partition the log, however this needs to be planned carefully as we lose the guarantee of ordering. Watch this space for mode detailed software architecture topics in the future.

Events are grouped into topics, used by brokers to categorise events. Consumers, or subscribers, will subscribe to topics that they are interested in. Although be aware that this specific terminology is more relevant to Kafka.

For Deliveroo this could be a ‘Food for pick up’ topic within the Deliveroo broker.

Immutable

Each event recorded by the broker is immutable. This means that an event cannot be changed after it has been created. This is important for two reasons:

1. It guarantees that events are processed in the same order every time
2. It ensures that data is not corrupted or amended.

Events

Events are the messages that flow through an event-driven system. These contain data that describe something that has happened, typically represented as a JSON object. This makes them easy to read and understand for both humans and computers.

{
"order id": "583837",
"order status": "food being prepared",
"collection restaurant": "pinche pinche"
"collection address": "45 harrogate road, Leeds",
"customer name: "beard meats food",
"delivery address: "123 roundhay road, Leeds,"
"driver id": "6435"
"collection time: "18:45 2022/10/15"
}

In the real world, there will be a lot of complex matching and logistics algorithms between the producer and consumer. These would be built as intermediate consumers and producers along the process flow.

Potential Pitfalls to Watch Out for with Event-streaming systems

Repetition of events

It is important that events are not unintentionally duplicated, otherwise 2 drivers might turn up to collect one meal, or perhaps you get 2 meals delivered! This can of course be avoided with careful design, but it is best to avoid such pitfall before they happen.

Publishing Events with Large Payloads

With the potential to process so many events in real-time, it is best practice to keep the size of the data contained in each event to a minimum. If large file sizes are required it is common to store the actual data in block storage and reference the location of the file in the metadata of the event.

Perhaps in our example the details of the recipe are stored in a book, and the message references the location of the recipe.

The Benefits of Event-driven Architecture

The key benefits of event-driven applications are:

1. Architectural Decoupling
2. Asynchronous processing of events
3. Inbuilt scalability

1) Decoupled Architecture

In a decoupled or loosely-coupled system, different parts of the system are not directly dependent on each other. This is in contrast to a tightly coupled system where different parts of the system are directly dependent on each other. In a tightly-coupled system, changing one part of the system can directly affect another, thus increasing the risk of regression issues

In a decoupled system, if one part of the system fails, it is isolated from the rest of the system and can be fixed without affecting the rest of the system.

In our example, if the chef is having issues with the cooker which delays that particular meal, the driver can continue to deliver other meals to other customers without it affecting all customers in the area.

2) Asynchronous event processing

Event-driven architecture enables asynchronous event processing. This is where different parts of the system run independently of each other. This is in contrast to a synchronous system where different parts of the system have a sequence of tasks to complete, each must wait for each other in order to complete the whole process.

In our example, the customer can send a message for the driver to leave the meal at the gate at the point he / she places the order. This will held at the broker level until a driver has been assigned to the delivery – at this point they are made aware of the detailed delivery instructions.

3) Inbuilt Scalability

A scalable system can handle an increased load without introducing down-time or any breaking changes to either the producers or consumers.

In our example, the Deliveroo platform will have a maximum number of orders they can accept in an hour. To increase this we could either employ more staff to facilitate these orders quicker (horizontal scaling) or expand the number of restaurants on the platform (vertical scaling), neither of which affects current customers or their orders.

Popular event-driven platforms

Some common event-driven platforms include:

• Apache Kafka – An open source, distributed event streaming platform
• Confluent Kafka – The enterprise version of Kafka, trusted by leading global companies across the world
• MSK – Fully managed Kafka by AWS
• Google Cloud Pub/Sub – A messaging service from Google Cloud Platform
• Apache Pulsar – Could-native messaging and streaming

Event Driven Architecture in Summary

Adopting an event-first approach in your software development approach represents a new opportunity, a new frontier for technology and businesses to go back to first principles of system architecture and design.

In our ever changing world, software must be flexible enough to quickly evolve, domain models that are supported by events can be decomposed and restructured to include new attributes or areas in the business quickly without need for major refactoring. It is this elastic nature by design which makes event driven architectures, and the information within the event data so power for digital native businesses.

If your business needs some help with building or optimising your own event-driven use case we’d love to help. Just drop us a message and arrange a call today.

Fore more content:

How to take your Kafka projects to the next level with a Confluent preferred partner

Event driven Architecture: A Simple Guide

Watch Our Kafka Summit Talk: Offering Kafka as a Service in Your Organisation

Successfully Reduce AWS Costs: 4 Powerful Ways

Master Apache Kafka 101