More and more companies are moving to containerized deployments and it is very common to do rolling deploys of a new image across multiple machines piece by piece to avoid downtime. Generally, I see devs using bash scripts for their build and deploy steps but I have been increasingly moving towards replacing bash with go as much as possible. This post is to show what a go script looks like to deploy a new docker image across different servers.
To complete this tutorial, you will need:
- Two or more servers with docker installed
- A Load balancer
- Basic knowledge of Golang
The entire script is available as a snippet on Gitlab here.
The steps are as follows:
To use the DigitalOcean SDK and container registry you will first need to create an access token. You can create one in the API section found in the left navigation bar in the
Token/Keys tab - docs here
You will need a container registry to hold all your application images. DigitalOcean provides a container registry option backed by Spaces. Product page here
The first thing we need is to get a list of droplets to deploy to. Assuming the droplets are tagged with
nginx-service we can get a list of droplets using DropletService
ListByTag(context.Context, string, *ListOptions) (Droplet, *Response, error) method. Ref: Line 143 of the script
Now we will fetch the load balancer that hosts the droplets from step 4. We need the loadbalancer so that we can remove droplets from it so they don’t get any traffic while the containers are being swapped - this avoids any user disruptions. Unfortunately, there is no list by tag method on the load balancer service - so we have to List all loadbalancers and use the first one that has the tag. Ref Line 174 of the script
To avoid deploying while client requests are in flight you will drain the droplet from the load balancer. This step is simple as we can simple untag the droplet that leads to it getting picked up by the loadbalancer by calling the
UntagResources(context.Context, string, *UntagResourcesRequest) (*Response, error) method. Ref Line 61 of the script
Now we need to create a docker client that connects to the docker daemon running on the droplet that was drained. Docker runs on port 2375 so make sure that the machine that this script runs on can access it on the remote droplet. Ref Line 211
Before you can deploy the new image - you will need to stop and remove the current version of the container on the server. First we will get the ID of the container with the service tag
nginx-service Line 225 and then we can call the
ContainerStop(ctx, containerID, timeout) method to kill the container Line 91
Now we need to pull the new version of the image from DigitalOcean’s container registry. We use the
ImagePull method in the Docker SDK with DigitalOcean’s access token as both the username and password for auth. Ref Line 242. This makes sure that the new image is available on the remote server to be run.
We can now start the container using the image pulled. We need to create a container config that maps the host port to the container port and tag the container. Ref Line 261. Creating the container config is separate from actually running the container. The config then gets passed to the
ContainerStart method ref Line 119
With the image updated the droplet can be safely added back to the load balancer and it can serve client requests using the new version of the image. This is as simple as retagging the droplet so that the loadbalancer picks it up again. Ref Line 125
This post shows how to do a rolling deploy on droplets but is more so to show that Golang is an excellent language to replace adhoc bash scripts. It is also extremely portable because of easy cross-compilation and statically linked binaries. I encourage you - next time you are planning on using bash to script something - ask yourself can I use Go instead?