Event-Driven Architectures

In a traditional API-style pattern, consumers send a request to a service and wait for a reply. This meant that if there were long-running jobs in-between a request, the consumer had to wait for all the processes to finish before it could continue with its following action. Since the consumers are usually responsible for orchestrating the requests to end services, this also means that additional work is needed whenever new features or integration is added.

For example, when a new order is checked out from an online store, the orchestrator is responsible for the following steps:

1. Process the payment
2. Update rewards department
3. Inform the warehouse service of a new order
4. Inform the shipping service of a new shipping request

In this flow, the order is only successfully processed once all the steps have been executed. Before the completion of the flow, any new requests will not be processed.

In Serverless, Event-Driven Architecture (EDA) is a commonly used pattern for microservices to communicate with one another. Within EDA, instead of “waiting” for a consumer to send a request before processing, the services “probe” the environment for changes and automatically react based on these changes. This communication style is asynchronous; whenever an event of interest happens within the environment, it is published into an event bus. The event bus then routes the event to subscribers interested in these changes.

In the example above, the “payment processed” event is published once a payment is processed. Subscribers, in this case, the rewards, warehouse, and shipping services, react to the “payment processed” event and executes their task in parallel. Services do not need to wait for one another before they can start.

In this article, we will explore using Amazon EventBridge, a Serverless event bus, for routing events within an environment. We will be using Terraform, a popular Infrastructure-as-Code (IaC) tool. Below is an example of the architecture that we will be building.

First, let’s create an EventBridge bus using Terraform. This event bus is used for ingesting and routing the events that we are creating as part of this demo.

resource "aws_cloudwatch_event_bus" "this" {
  name = "eda_bus"
}

Next, let’s create an AWS Lambda function that acts as the payment processing microservice. This Lambda function will first process the payment and publish the event using the put_events method in the Boto3 events client.

Create the following file in the ./payment/src directory and name is lambda_function.py .

import json
import os

import boto3

eventbridge_client = boto3.client('events', region_name=os.getenv("region"))
eventbridge_name = "eda_bus"


def lambda_handler(event, context):
    # Process payment
    
    response = eventbridge_client.put_events(Entries=[
        {
            'Source': "demo.payments",
            'DetailType': 'Payment Processed',
            'Detail': json.dumps({
                "total_cost": 242.58,
                "customer_id": "3213",
                "address": "123 Main Street, Anytown, USA 12345",
                "items": ["apple", "banana", "carrots"]
            }),
            'EventBusName': eventbridge_name
        }
    ])

    return {
        'statusCode': 200,
        'body': json.dumps({
            "message": "success"
        })
    }

To publish the Lambda function,

resource "aws_lambda_function" "payment" {
  filename         = "./payment/src"
  function_name    = "payment"
  role             = aws_iam_role.lambda_iam_role.arn
  handler          = "lambda_function.lambda_handler"
  runtime          = "python3.9"
}

resource "aws_iam_role" "lambda_iam_role" {
  name               = "lambda-role"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Action" : "sts:AssumeRole",
        "Principal" : {
          "Service" : ["lambda.amazonaws.com"]
        },
        "Effect" : "Allow"
      }
    ]
  })
}

resource "aws_iam_policy" "lambda_policy" {
  name        = "Lambda policy"

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "events:PutEvents"
      ],
      "Effect": "Allow",
      "Resource": aws_cloudwatch_event_bus.this.arn
    }
  ]
}
EOF
}

resource "aws_iam_role_policy_attachment" "attach_policy" {
  role       = aws_iam_role.lambda_iam_role.name
  policy_arn = aws_iam_policy.lambda_policy.arn
}

resource "aws_iam_role_policy_attachment" "basic" {
  role       = aws_iam_role.lambda_iam_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

Let’s also create another Lambda function and call it rewards. Create a new lambda_function.py file in the ./rewards/src directory.

import json
import os

def lambda_handler(event, context):
    print(event)

    # Rewards processing

    return {
        'statusCode': 200,
        'body': json.dumps({
            "message": "success"
        })
    }

To publish the lambda function,

resource "aws_lambda_function" "rewards" {
  filename         = "./rewards/src"
  function_name    = "rewards"
  role             = aws_iam_role.lambda_iam_role.arn
  handler          = "lambda_function.lambda_handler"
  runtime          = "python3.9"
}

Next, let’s create an EventBridge rule to filter out events and set the rewards Lambda as a target. With this rule, the rewards function will only be processed when a payment of more than 200 dollars has been made.

resource "aws_cloudwatch_event_rule" "payment_processed" {
  name           = "payment-processed"
  event_bus_name = aws_cloudwatch_event_bus.this.name

  event_pattern = jsonencode({
    source      = ["demo.payments"]
    detail-type = [
      "Payment Processed"
    ]
    detail = {
      "total_cost": [ { "numeric": [ ">", 200 ] } ]
    }
  })
}


resource "aws_cloudwatch_event_target" "lambda_target" {
  rule           = aws_cloudwatch_event_rule.payment_processed.name
  arn            = aws_lambda_function.rewards.arn
  event_bus_name = aws_cloudwatch_event_bus.this.name
}


resource "aws_lambda_permission" "allow_invoke_from_eventbridge" {
  statement_id  = "AllowExecutionFromEventBridge"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.rewards.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_bus.this.arn
}

When we do a test function on the Payment Lambda, a new Event will be sent to EventBridge. If we look at the CloudWatch logs of the Rewards Lambda, we will see a log record of the event that was triggered when EventBridge forwards the event to it.

Conclusion

In conclusion, event-driven architecture offers a powerful and flexible approach to building scalable and resilient cloud-based systems. By leveraging asynchronous, decoupled communication through events, EDA enables systems to respond quickly and efficiently to real-time changes and events. New integrations and functionalities can be easily added to your environment simply by subscribing to the events through the Event Bus with no changes to your existing environment, making it highly scalable.