Take A Node.js With Express API Serverless Using AWS Lambda

Not too long ago I had written about creating an API with Node.js and Express that accepted image uploads and manipulated the images to be Android compliant before returning them in a ZIP archive. This article was titled, Create an Android Launcher Icon Generator RESTful API with Node.js, and Jimp, and it was a great example of creating APIs that that did most of their work in memory. I even demonstrated how to containerize the application with Docker.

Applications that manipulate media will need to be able to scale, otherwise there is a risk of the application crashing from not enough resources, or too many resources can get expensive. For this reason, it makes perfect sense to take the previous example serverless with Amazon’s Lambda and API Gateway offerings.

We’re going to see how to use API Gateway to accept HTTP requests with binary image data and process that data with Lambda to return various sized Android launcher images packaged in a ZIP archive.

Installing the Serverless Framework and Creating a Project

While not absolutely necessary, we’re going to be making use of Serverless Framework to make our development lives a little easier. If you’ve not heard of Serverless Framework, it is a tool that generates projects for AWS Lambda and IBM OpenWhisk as well as deploys them.

To install the Serverless Framework, execute the following:

npm install -g serverless

With Serverless available on our computer, we need to create a new project. Choose a location on your computer and execute the following command:

serverless create --template aws-nodejs --path image-service

The above command will create an AWS project using Node.js rather than one of the other supported AWS languages. This project will be called image-service and it should have the following in it:

handler.js
serverless.yml

The serverless.yml file will hold our configurations and the handler.js file will hold our Lambda functions. Before we touch these files, we need to get the rest of our project dependencies.

From the command line, with the project as the active path, execute the following:

npm install bluebird jimp node-zip serverless-apigw-binary serverless-apigwy-binary --save

If you went through the previous tutorial on creating an API that creates Android launcher icons, the bluebird, jimp, and node-zip packages should be familiar to you. The serverless-apigw-binary package will allow us to configure AWS API Gateway to support binary files and the serverless-apigwy-binary package will allow us to configure binary responses for our project.

If you’re familiar with Serverless Framework, you might be familiar with the serverless-offline package. This package allows you to test everything locally. I recommend that you do not use it for this example because it doesn’t work correctly with binary data. This will result in you banging your head for hours like I did.

Configuring Serverless Framework to use Amazon Web Services

We won’t be testing anything locally so we’ll need to configure Serverless Framework to use your AWS account. Sign into the AWS portal and choose to manage your security credentials.

Choose Create New Access Key and take note of your public and private keys. Using those keys, execute the following from the command line:

export AWS_ACCESS_KEY_ID=<your-key-here>
export AWS_SECRET_ACCESS_KEY=<your-secret-key-here>

Take note that the above commands will only persist for your current session. If you wish to have something a little more long term, consider altering your ~/.profile file or similar on Windows.

When it comes to management of Lambda and API Gateway, the CLI will take care of the rest from this point on.

Creating an AWS Lambda Function for Image Processing

The basis of this example will use a lot of code that we’ve already seen. However, in the previous example that used Jimp, we were using Express Framework. We need to make a few changes.

Within your project, create an image.js file. It should contain the following JavaScript:

const Zip = new require('node-zip')();
const Bluebird = require("bluebird");
const Jimp = require("jimp");

Jimp.prototype.getBufferAsync = Bluebird.promisify(Jimp.prototype.getBuffer);

class Image {

    constructor(url) {
        this.url = url;
    }

    generate(callback) {
        Jimp.read(this.url, (error, image) => {
            if(error) {
                return callback(error, null);
            }
            var images = [];
            images.push(image.resize(196, 196).getBufferAsync(Jimp.AUTO).then(result => {
                return new Bluebird((resolve, reject) => {
                    resolve({
                        size: "xxxhdpi",
                        data: result
                    });
                });
            }));
            images.push(image.resize(144, 144).getBufferAsync(Jimp.AUTO).then(result => {
                return new Bluebird((resolve, reject) => {
                    resolve({
                        size: "xxhdpi",
                        data: result
                    });
                });
            }));
            images.push(image.resize(96, 96).getBufferAsync(Jimp.AUTO).then(result => {
                return new Bluebird((resolve, reject) => {
                    resolve({
                        size: "xhdpi",
                        data: result
                    });
                });
            }));
            images.push(image.resize(72, 72).getBufferAsync(Jimp.AUTO).then(result => {
                return new Bluebird((resolve, reject) => {
                    resolve({
                        size: "hdpi",
                        data: result
                    });
                });
            }));
            images.push(image.resize(48, 48).getBufferAsync(Jimp.AUTO).then(result => {
                return new Bluebird((resolve, reject) => {
                    resolve({
                        size: "mdpi",
                        data: result
                    });
                });
            }));
            Bluebird.all(images).then(data => {
                for(var i = 0; i < data.length; i++) {
                    Zip.file(data[i].size + "/icon.png", data[i].data);
                }
                var d = Zip.generate({ base64: true, compression: "DEFLATE" });
                var response = {
                    statusCode: 200,
                    headers: {
                        "Content-Type": "application/zip",
                        "Content-Disposition": "attachment; filename=android.zip"
                    },
                    body: d,
                    isBase64Encoded: true
                };
                callback(null, response);
            });
        });
    }

}

module.exports = Image;

The above code represents an Image class. The constructor will take either an image buffer or an image URL. We’ll be focusing on the image buffer. The generate method will take a callback which Lambda will use and it will resize our image and return it within a ZIP archive. There are a few notable difference here. We’re not using Express and we’re not using Multer.

Let’s take note of the following:

Bluebird.all(images).then(data => {
    for(var i = 0; i < data.length; i++) {
        Zip.file(data[i].size + "/icon.png", data[i].data);
    }
    var d = Zip.generate({ base64: true, compression: "DEFLATE" });
    var response = {
        statusCode: 200,
        headers: {
            "Content-Type": "application/zip",
            "Content-Disposition": "attachment; filename=android.zip"
        },
        body: d,
        isBase64Encoded: true
    };
    callback(null, response);
});

After each asynchronous image manipulation runs, the Bluebird.all method runs. We’re adding each image buffer to the ZIP archive and generating the archive. Previously we were generating the ZIP archive as binary. This time around we’re generating the ZIP archive as a base64 string which Lambda will process correctly.

Within the response, which is AWS Lambda compliant, we are defining our headers, the response body, and the format of the response. The header information will be very important in the next steps.

With the Image class created, let’s create our Lambda function. Open the project’s handler.js file and include the following:

'use strict';

const Image = require("./image");

module.exports.generate = (event, context, callback) => {
    var i = new Image(Buffer.from(event.body, "base64"));
    i.generate(callback);
};

We created the Image class to keep our handler.js file clean. Within the handler.js file we have a single function. When called, the image data taken from the base64 request body will be converted to a buffer and passed to our image generator. When finished, the generator will call the callback which will send a response to the client.

Having a Lambda function isn’t totally useful to us, so we’re going to configure the API Gateway.

Configuring AWS API Gateway for Accepting HTTP Requests and Communicating with Lambda

The API Gateway, like the Lambda functions, can be managed from the serverless.yml file. Open it and include the following YAML:

service: image-service

provider:
  name: aws
  runtime: nodejs6.10

functions:
  generate:
    handler: handler.generate
    events:
        - http:
            path: process
            method: post
            contentHandling: CONVERT_TO_BINARY

plugins:
  - serverless-apigw-binary
  - serverless-apigwy-binary

custom:
  apigwBinary:
    types:
      - 'application/octet-stream'
      - 'application/zip'
      - 'image/png'
      - 'image/jpeg'

In the functions section, we are defining our only function, the generate function. It will respond to HTTP events at the /process path, similar to our Express example. It is important that we define the contentHandling because the requests and responses need to be in binary format.

To have Serverless Framework configure our API Gateway, the two packages that we had installed, need to be added to the plugins section. To configure the plugins, the apigwBinary section must have a list of MIME types that should be treated as binary, rather than encoded as text. AWS will encode anything not in this list as text, which will completely break the file.

At this point we can deploy our project.

Deploying the Serverless Framework Project to API Gateway and Lambda

With the Serverless Framework CLI available to us, execute the following command:

serverless deploy

It may take a while for the project to deploy. When it does, the CLI will provide a link on where to find the API Gateway URL. Future changes to the generate function can be made by executing:

serverless deploy function --function generate

By deploying only the function, the time will be reduced significantly.

Testing the Application with cURL

Like with the previous example that used Node.js and Express, we’re going to be using cURL to test. We could use a tool like Postman, but because we anticipate downloading a ZIP archive, Postman isn’t the best solution.

From the Command Prompt (Windows) or Terminal (Mac and Linux), execute the following:

curl -X POST -H "Accept: image/png" -H "Content-Type: image/png" --data-binary @./icon.png https://yyy.execute-api.us-east-1.amazonaws.com/dev/process >> android.zip

There are a few things to note about the above command. First, we’re going to be sending a portable network graphic (PNG) file, hence the header information. If you’re trying to manipulate a different file, change the headers where appropriate. Next we have @./icon.png which represents a file at our relative path. The @ symbol allows us to send an actual file.

Make sure you change the Amazon URL to that of your own. The results should be output to a file called android.zip as per our cURL command.

Conclusion

You just saw how to take a powerful and potentially resource hungry Node.js API and bring it to AWS Lambda and AWS API Gateway using the Serverless Framework. This tutorial was an extension to a previous tutorial titled, Create an Android Launcher Icon Generator RESTful API with Node.js, and Jimp. This time around we went serverless instead of creating our own API server running Node.js and Express, or even going down the Docker containerization route.

A few important things to note when it comes to processing binary data with API Gateway and Lambda. Don’t expect to be able to test locally with Serverless Framework. The requests and responses will be mangled and leave you wondering why. It is also very important that the appropriate binary file types are listed, otherwise API Gateway will encode them as text, also mangling the requests and responses.

Nic Raboy

Nic Raboy is an advocate of modern web and mobile development technologies. He has experience in C#, JavaScript, Golang and a variety of frameworks such as Angular, NativeScript, and Unity. Nic writes about his development experiences related to making web and mobile development easier to understand.