Load Balancing A Docker Swarm Cluster With An NGINX Reverse Proxy

If you’ve been working towards containerizing your web applications like I have, you might be at a point where you’re ready to start clustering your containers. Previously I had written about creating a container cluster with Docker Swarm and using NGINX as a reverse proxy for a few containers. The catch here is that neither of these previous tutorials were meant to work together. In the previous example we were using a reverse proxy for containers on a single server. While Docker Swarm offers it’s own load balancing, you’ll find it makes sense to have NGINX as well because not every container can run on the host as port 80.

We’re going to see how to create two service containers that are replicated across several nodes. These services will be a simple Apache and NGINX web applications. Then we’re going to throw an NGINX reverse proxy into the mix that keeps track of the upstream nodes for its own load balancing.

If you haven’t already seen how to setup a Swarm cluster, I suggest you check out my previous tutorial on the topic, as the Docker Swarm fundamentals won’t be present here.

Let’s say you have a three node Swarm with two services that are replicated five times each. What we need to do is create a reverse proxy that can accommodate each of the three nodes in the Swarm. In the land of NGINX reverse proxies, the server nodes are referred to as upstream servers.

To make this reverse proxy possible, we’re going to need to create a custom image, similar to how we did it in the previous article that I wrote. When we previously created a reverse proxy, we created a custom nginx.conf file that we copied into the image. This time around our custom configuration file will look something like this:

worker_processes 1;

events { worker_connections 1024; }

http {

    sendfile on;

    upstream docker-nginx {
        server 192.168.99.100:8080;
        server 192.168.99.101:8080;
        server 192.168.99.102:8080;
    }

    upstream docker-apache {
        server 192.168.99.100:8081;
        server 192.168.99.101:8081;
        server 192.168.99.103:8081;
    }

    server {
        listen 8000;
        #server_name example1.com;

        location / {
            proxy_pass         http://docker-nginx;
            proxy_redirect     off;
            proxy_set_header   Host $host;
            proxy_set_header   X-Real-IP $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Host $server_name;
            add_header         X-Upstream $upstream_addr;
        }
    }

    server {
        listen 8001;
        #server_name example2.com;

        location / {
            proxy_pass         http://docker-apache;
            proxy_redirect     off;
            proxy_set_header   Host $host;
            proxy_set_header   X-Real-IP $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Host $server_name;
            add_header         X-Upstream $upstream_addr;
        }
    }

}

In the above example, we have two applications, one represented by docker-nginx and the other represented by docker-apache, both of which are standard web applications. The names we chose are meaningless as long as we’re consistent. In each of the upstream we add each server that these applications appear on. Remember, they are part of a Swarm that span three nodes. In our scenario the containers will end up on all the servers, but if you have special labels set up, yours may not.

Notice the ports used in each of the upstream sections. We’re assuming that our NGINX replicas are accessible to the Docker host on port 8080 and our Apache replicas 8081. These are not the ports of the containers, but the ports bound to the host.

In the server sections we have two reverse proxy scenarios. Since this is local to my machine, the reverse proxy is going to listen on two different ports which don’t have to match the upstreams. Should this be deployed and have a domain name attached, you’ll want to use port 80.

server {
    listen 8000;
    #server_name example1.com;

    location / {
        proxy_pass         http://docker-nginx;
        proxy_redirect     off;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Host $server_name;
        add_header         X-Upstream $upstream_addr;
    }
}

In the above example, if we hit port 8000 on any of Swarm nodes, the NGINX web application will be served. Likewise if we hit port 8001, the Apache application will be served.

Now, we are actually using the NGINX reverse proxy as a load balancer. This means that as requests come in, they will be dispersed in a special way to each of the upstream nodes. By default NGINX will do round robin, but that can be changed if you’d like.

Notice the add_header line in each of the server sections. The $upstream_addr is the upstream node that is actually being served, not the reverse proxy itself. If you open Chrome Inspector and view the response header when you load the page, it will show a different upstream on every request. That is how you know it is working.

This tutorial probably wouldn’t be complete unless we saw how to deploy everything. Assuming you already have a Swarm configured, let’s deploy these three containers with replicas. First let’s build our reverse proxy. If you don’t already have a Dockerfile file sitting next to your nginx.conf file, create one with the following content:

FROM nginx:alpine

COPY nginx.conf /etc/nginx/nginx.conf

Now we can build our reverse proxy. Just make sure you’ve updated the upstream servers to match your Swarm nodes. From the Docker Shell, execute the following:

docker build -t reverseproxy /path/to/directory/with/dockerfile

The NGINX and Apache web application images don’t need to be custom because for this example, the stock images are fine. For more information on building custom Docker images, check out a previous tutorial I wrote on the subject.

From the manager node, execute the following to deploy an NGINX web server with replicas:

docker service create --replicas 5 -p 8080:80 --name nginx nginx:alpine

The above command will deploy NGINX with five replicas and they will be accessible from the host on each node on port 8080, which is what our reverse proxy expects.

From the manager node, execute the following to deploy an Apache web server with replicas:

docker service create --replicas 5 -p 8081:80 --name apache httpd:alpine

Again, pay attention to the port as it matches what we defined in our reverse proxy configuration. As of right now if we were to navigate to any of the Swarm nodes via a web browser, matching the port 8080 or 8081, we can show the appropriate web application. However, we want the NGINX reverse proxy to handle this with load balancing.

From the manager node, execute the following to deploy the reverse proxy with replicas:

docker service create --replicas 5 -p 8000-8001:8000-8001 --name reverseproxy reverseproxy

The above command is similar, but not the same. In our example, the reverse proxy is listening on port 8000 and 8001. In production, it will probably be port 80 with domain names attached.

You should be able to hit any Swarm node on port 8000 or 8001 and be load balanced to the correct web application on the appropriate upstream server.

Conclusion

You just saw how to use an NGINX reverse proxy as a load balancer to web applications in a Docker Swarm. This means that your NGINX reverse proxy can act as a passthrough for requests and route traffic between the nodes in Swarm. By default this is done in round robin, but it can be adjusted as necessary. If a node is unavailable, the load balancer will not route traffic to it.

If you’d like detailed instructions on configuring a Docker Swarm, check out the tutorial I wrote called Creating a Cluster of Microservice Containers with Docker Swarm. For more information on working with a reverse proxy with Docker, check out the tutorial I wrote called Using NGINX as a Reverse Proxy to Your Containerized Docker Applications.

While this tutorial was all done locally, if you’d like to take your Docker Swarm to the cloud, I highly recommend using Digital Ocean or Linode virtual private servers (VPS) as they are cheap, yet powerful.

Nic Raboy

Nic Raboy is an advocate of modern web and mobile development technologies. He has experience in C#, JavaScript, Golang and a variety of frameworks such as Angular, NativeScript, and Unity. Nic writes about his development experiences related to making web and mobile development easier to understand.