No doubt like many developers reading this article, my previous experience developing for the web resides largely in building multi-tiered applications with many parts being conscious of each other and all running on the same server, or perhaps distributed over a cluster and communicating via some kind of service.

Microservices, however, are designed to be independent and self-sufficient; they are designed to do one thing and to do that one thing well. It should be simple to consume a microservice. The more intuitive it is to use a microservice, the better. A microservice should be self-contained and not depend upon any other service or microservice unless it is a clearly defined integration coupling like a DynamoDB table or an S3 Glacier archive. Microservices should be as ignorant as this guy. Credit: Buzzfeed

While the true implementation of microservices (and services in general) often differs from system to system, or developer to developer, the aforementioned characteristics are pretty reasonable. Abstracted Microservice Architecture

The architecture that I am discussing here is simply one of abstraction and generally nothing new in the world of software development. Managing Lambda functions and their integration points though can be tricky and troublesome, but thanks to the Serverless Framework’s use of CloudFormation stacks and a straight-forward YAML configuration file, building your application with an abstracted architecture is quite simple and can make your code far more manageable, logical and flexible. I should note that it is not necessary or essential to use the Serverless Framework to build this architecture — you could do this manually if you wanted.

Lambda functions are designed for integration and are generally invoked either by another AWS service or via another AWS service. For example, I might make a request to my API Gateway which will in turn invoke a Lambda function, or my SNS Topic might contain a message to which another Lambda function might be subscribed.

In a serverless.yml file it is possible to configure the events that your Lambda functions may respond to:

functions:
  getUser:
    name: getUser
    handler: users.get
    event:
      — http:
        path: users
        method: get

In this example, a function in the users.js file with the name get will respond to an API Gateway HTTP GET resource invocation. The Serverless framework will automatically map our query string parameters, headers and other request values to the event parameter of our Lambda handler. Normally, this is where many applications will process the request and return whatever data is necessary.

When you deploy a Serverless service (using the frameworks definition) it is created using CloudFormation. When the API is created, it is created with a unique endpoint. If you deploy another service with http events, that will also have an API created with a unique endpoint. One service may have and endpoint that starts with https://fghijklmnop.execute-api… and the other https://abcdwxyz.execute-api… If you wanted a single API with a common URL that you can use to test your application or even use in production, then you will probably be quite frustrated with this result.

A similar problem can occur if you are trying to deploy an application with multiple services with various functions that wish to subscribe to any number of SNS topics. Service A and Service B both have functions that want to subscribe to topic OnUserCreated. You deploy Service A and everything is fine — your functions and SNS topic are created. You then try to deploy Service B but it fails and an error message lets you know that a topic called OnUserCreated already exists. You can always put in the ARN for the SNS topic, but that can be problematic if you want to have different topics for testing/staging/live environments. Project code structure.

A simple solution is to have services that are dedicated to integrating with specific AWS services. Their sole purpose is to define the API endpoints or SNS topics (or any number of AWS service integrations) required and then provide functions that will respond to these events, parse the incoming data and invoke any necessary microservices that would otherwise be directly bound to these events. This way, these microservices can remain agnostic to any integration point and improve their flexibility and reuse. By abstracting the integration responsibility, all services can have a consistent event schema and become much more useful.

Instead of having most business logic now associated with the function that integrates with API Gateway, an integration function could look like the following:

module.exports.retrieveFile = (event, context, cb) => {
  // Process integration-specific input.
  const data = event.query;
  const stage = process.env.STAGE;
  const region = process.env.REGION;

  const functionName = `${region}-${stage}-file-manager-retrieveFile`;

  // Map input data to a format the microservice is expecting.
  const arn = `${region}-${stage}-file-manager-retrieveFile`;
  const params = {
    FunctionName: arn,
    InvocationType: 'RequestResponse',
    Payload: JSON.stringify(data),
  };

  // Call the microservice and return the resulting data as normal.
  const lambda = new AWS.Lambda({ apiVersion: '2015-03-31' });
  lambda.invoke(params, (error, result) => {
    if (error) {
      cb(error);
    } else {
      // You could always do something fancier here, like invoke another service, for example.
      cb(null, result);
    }
  });
};

The number of microservices the integration service invokes is up to the developer/architect involved, but it would be quite straight-forward to invoke multiple microservices and aggregate the responses, particularly if Promises were used to orchestrate the lambda invocations. At least now all API functionality can be organised and managed from a single point, rather than having to maintain multiple configuration files or services that arguably should not be aware of an API endpoint in the first place.

Another benefit of this loosely-coupled design means integrations can be added without needing to redeploy underlying microservices. Consider a Reporting microservice that generates reports on S3 data usage that is normally invoked via API Gateway. A new requirement comes in that the report needs to be generated whenever a new file is added to any S3 bucket. Using this architecture, it is possible to simply add a new integration specifically to respond to any S3 creation event, which can then invoke our pre-existing (but integration independent) Reporting microservice. The integration function can then pass the response that would normally have been returned as a HTTP response to another service, or store the data in an S3 bucket, but the important part is that the Reporting service does not have to change to accommodate this new requirement.

I have created a sample project to demonstrate this architectural approach. I use GitLab and have configured the project to use GitLab’s CI tools to deploy the application. It is quite basic and is very much a work in progress, but it should hopefully demonstrate the principles that I have discussed in this article. I am also very keen for feedback. If you have any opinions or suggestions, please feel free to comment below or raise an issue on the repo itself.