Deploying Native Node.js Dependencies On AWS Lambda

I was recently working on a Functions as a Service (FaaS) project using AWS Lambda and Node.js. However, I was running into an issue where my package dependencies found in my node_modules directory were for the wrong platform once deployed to Lambda. This is not the first time I experienced a problem like this. I knew the issue straight away because I encountered the same thing when trying to use a node_modules directory generated on Mac from a Windows computer.

When uploading a package developed with Node.js to AWS Lambda, the package.json file is not considered. Instead you are uploading a package that contains the node_modules directory and all dependencies. So how do you develop for AWS Lambda from Mac and Windows, but have it work once deployed?

We’re going to see how to use Docker to get our Node.js FaaS project dependencies designed for Amazon’s flavor of Linux.

Going forward, I need to give credit where credit is deserved. I learned a lot of what I’m sharing from an article by Eoin Mullan titled, Using Native Dependencies with AWS Lambda.

Creating an AWS Lambda Function with the Serverless Framework

If you’ve ever worked with AWS Lambda and AWS API Gateway, you’ll know it is a hassle. Instead, people came together to build the Serverless Framework. While the actual project isn’t very important, we’re going to be using Serverless Framework to get us going.

If you’ve not already, install it by executing the following:

npm install -g serverless

With the CLI installed, we need to create a fresh project. We can use one of the already available templates to get us going quickly. Execute the following from the command line:

serverless create --template aws-nodejs --path ./my-project

Now let’s assume we want to use an SDK that is designed to be platform specific. The Couchbase Node.js SDK is an example of this where the Mac version is not compatible on Windows or Linux and likewise for any other combination.

In a typical scenario we’d execute the following within our project:

npm install couchbase

We can’t do this from our Mac and Windows computer because it won’t work on AWS Lambda. Instead we need an Amazon Linux compatible set of dependencies.

This is where Docker comes into the mix.

Deploying a Docker Container for Amazon Linux

As previously mentioned, Eoin Mullan had written about using an Amazon Linux Docker image to handle any native dependencies. With Docker installed, you can execute the following to use Amazon Linux in your project:

docker pull amazonlinux
docker run -v $(pwd):/lambda-project -it amazonlinux

The first command will download the Amazon Linux image from Docker Hub. The second command will deploy the image as a container and give you the interactive terminal. For convenience, the project directory would be mapped as a volume within the container.

This is great, but the Amazon Linux image was designed to be lightweight. This means it won’t have Node.js installed and ready to go for us.

Installing Node.js in Amazon Linux and Downloading the Dependencies

The next step is to install Node.js in our currently deployed Amazon Linux container. After it is installed, we can grab our project dependencies and prepare for deployment.

You should be within the interactive terminal for your container. From it, execute the following command:

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.6/install.sh | bash
. ~/.nvm/nvm.sh

The above commands will install and activate the Node Version Manager (NVM). With it we can choose to install any version of Node.js within our container.

We want to try to match the version of Node.js for our project, so let’s install the following:

nvm install 6.11.5

If these steps look familiar, it is because I took them straight from the Amazon Web Services documentation.

Remember that volume we mapped when deploying the container? Let’s navigate to it from within the interactive terminal:

cd /lambda-project

Within this project, we can download our dependencies. To keep with the example, let’s install the Couchbase SDK for Node.js:

npm install couchbase

If you had already run the above command from the host machine, make sure you remove the node_modules directory before trying to run it from Amazon Linux. You want a clean slate.

When finished, you can destroy the container because a new node_modules directory should exist in your project, this time with the proper Amazon Linux native dependencies.

Designing an Amazon Linux Docker Image with Node.js

Deploying an Amazon Linux Docker container, configuring the Node Version Manager (NVM), and installing Node.js is cool, but it isn’t the most efficient strategy. Instead it might make sense to create an Amazon Linux image preconfigured and ready to go with Node.js.

Create a directory somewhere on your computer. Within this directory create a Dockerfile file. This file should contain the following:

FROM amazonlinux

RUN curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.6/install.sh | bash
RUN /bin/bash -c "source /root/.nvm/nvm.sh; nvm install 6.11.5"

CMD /bin/bash -c "source /root/.nvm/nvm.sh; nvm use 6.11.5"

So what is happening in the above image configuration?

First we’re using the vanilla Amazon Linux image as our base. The RUN commands allow us to run commands at image build time while the CMD command allows us to run commands at container runtime. The commands we issue are slightly different than what we saw in the manual process because of how the NVM script is.

More information on customizing a Docker image can be found in a previous tutorial I wrote titled, Build a Custom Docker Image for Your Containerized Web Application.

With the Dockerfile file ready to go, we can build the image with the following command:

docker build -t custom-amazon-linux .

The above command will build the image at the local path and name it custom-amazon-linux.

Bundling the Project for Deployment in the Amazon Cloud

At this point in time, you should have a Serverless Framework project for Node.js that has a node_modules directory with native dependencies for Amazon Linux, what AWS Lambda operates on.

To be able to deploy your function or functions, you need to configure your Serverless Framework CLI with the appropriate AWS tokens. We won’t get into that here, but once you’re configured for Amazon Web Services, execute the following:

serverless deploy

If everything went as planned, the native dependencies obtained through the Amazon Linux Docker container should give you a working set of functions.

Conclusion

When developing Node.js applications, regardless if they are on AWS Lambda, it is important to remember that some dependencies are native to a particular platform at the time of the download. Because AWS Lambda doesn’t use a Node.js package.json file, the dependencies must be obtained through an Amazon Linux instance, easily deployable with Docker.

More information on using Serverless Framework and AWS Lambda with Node.js can be found in a previous tutorial I wrote titled, Take a Node.js with Express API Serverless Using AWS Lambda.

Nic Raboy

Nic Raboy is an advocate of modern web and mobile development technologies. He has experience in C#, JavaScript, Golang and a variety of frameworks such as Angular, NativeScript, and Unity. Nic writes about his development experiences related to making web and mobile development easier to understand.