Self hosted Docker registry

ci, docker, github-actions

Introduction

If I am running some software, written by me or someone else, on a server, such as a DigitalOcean droplet, the aim is to always deploy it via docker. On my server, I want a single large docker compose file that has everything I want running.

This gives the benefit that nothing special needs to be installed/configured on the server other than installing docker, which most providers provide pre-baked server images for - so you can just assume docker is already present on the system.

Even my own apps, such as my api run as docker containers but before a container can be run an image needs to be built and pushed to a registry somewhere that the server which will pull it can reach it.

In this post I will walkthrough how I have my self hosted docker registry setup, where images are pushed to automatically as I push code to main on github.

Components

  1. A server with docker installed on it
  2. Docker registry will be run as a container on a server
  3. Nginx on the same server as a proxy to handle tls and forward requests to the registry container
  4. Project pom file to build and push image using the Jib Maven plugin
  5. Github actions ci pipeline to build and push docker image on commit to main

And a DNS, such as Cloudflare, that will provide routing to the server

A server with docker installed on it

This can be any server that you have running around. I use a DigitalOcean droplet that comes preinstalled with Docker on it. You can use my referral link to get $200 in credits to try it out https://m.do.co/c/b5f565690240

The droplet with docker installed on it can be created with one curl request [replace the token with your auth token]

1
2
3
4
curl -X POST -H 'Content-Type: application/json' \
     -H 'Authorization: Bearer '$TOKEN'' -d \
    '{"name":"docker-registry-server","region":"sfo2","size":"s-2vcpu-4gb","image":"docker-20-04"}' \
    "https://api.digitalocean.com/v2/droplets"

Docker registry will be run as a container on a server

The docker registry runs as a docker container itself. Setup a docker compose file which will have our container specifications

The registry can be run using the following yaml.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
registry:
  image: registry:2.8.1
  container_name: registry
  restart: unless-stopped
  environment:
    REGISTRY_AUTH: htpasswd
    REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
    REGISTRY_AUTH_HTPASSWD_PATH: /auth/registry.password
    REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
    REGISTRY_HTTP_HOST: "https://registry.xyz.io" # your domain name here
  volumes:
    - ./registry/registry.password:/auth/registry.password
    - ./registry/data:/data
  ports:
    - 5000

The registry has the registry folder mounted so that the data persists after container restarts. The password file registry.password contains the auth information and the /data folder will contain the image bytes. The path to these folders are then passed to the container as environment variables REGISTRY_AUTH_HTPASSWD_PATH and REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY respectively

The authentication file is created using htpasswd using the command htpasswd -Bc <output-file-name> <wanted-username-of-registry> this will then prompt for a password after which the auth information will be dropped on disk as a file.

Nginx on the same server as a proxy to handle tls and forward requests to the registry container

Nginx is also running as a docker container and is part of the docker compose file. The yaml for it is as follows

1
2
3
4
5
6
7
8
9
10
11
12
nginx:
  image: nginx:latest
  restart: unless-stopped
  container_name: nginx
  depends_on:
    - registry
  volumes:
    - ./nginx/nginx.conf:/etc/nginx/nginx.conf
    - ./nginx/cert.pem:/etc/ssl/certs/cert.pem
    - ./nginx/key.pem:/etc/ssl/certs/key.pem
  ports:
    - "443:443"

The cert and key pem files are for TLS - I use the ones that Cloudflare provides for my domain

The nginx.conf file has a server block that listens on the domain and forwards request to the registry docker container on port 5000

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
upstream registry {
    server registry:5000;
}

server {
    server_name registry.xyz.io;
    listen 443 ssl http2;
    ssl_certificate /etc/ssl/certs/cert.pem;
    ssl_certificate_key /etc/ssl/certs/key.pem;

    # Improve HTTPS performance with session resumption
    ssl_session_cache shared:SSL:20m;
    ssl_session_timeout 30m;
    ssl_session_tickets off;
    # Enable server-side protection against BEAST attacks
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384";
    # Aditional Security Headers
    # ref: https://developer.mozilla.org/en-US/docs/Security/HTTP_Strict_Transport_Security
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
    # ref: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
    add_header X-Frame-Options DENY always;
    # ref: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Content-Type-Options
    add_header X-Content-Type-Options nosniff always;
    # ref: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection
    add_header X-Xss-Protection "1; mode=block" always;
    resolver 1.1.1.1 1.0.0.1 [2606:4700:4700::1111] [2606:4700:4700::1001] valid=300s;
    resolver_timeout 5s;
    location / {
        proxy_pass http://registry;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header Docker-Distribution-Api-Version registry/2.0;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

run docker-compose up -d to have a registry and nginx proxy running in the background

Project pom file to build and push image using the Jib Maven plugin

I use Jib, which is a maven plugin to build and push my docker images. The main advantages are that it doesn’t require the docker daemon nor does it need hand written dockerfiles

Add a plugin tag to configure jib

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<plugin>
   <groupId>com.google.cloud.tools</groupId>
   <artifactId>jib-maven-plugin</artifactId>
   <version>1.8.0</version>
   <configuration>
       <from>
           <image>ibm-semeru-runtimes:open-16-jre</image>
       </from>
       <to>
           <image>registry.xyz.io/awesome-app</image>
           <tags>
               <tag>${tagSha}</tag>
           </tags>
       </to>
       <container>
           <ports>
               <port>8080</port>
               <port>8081</port>
               <port>8888</port>
           </ports>
           <mainClass>dev.aawadia.MainKt</mainClass>
           <jvmFlags>
               <jvmFlag>-Xms:1g</jvmFlag>
               <jvmFlag>-Xmx:1g</jvmFlag>
           </jvmFlags>
       </container>
   </configuration>
</plugin>

To compile the application, build and push the docker image run mvn -T32 clean compile package jib:build -DtagSha=$(git rev-parse HEAD)

This will push the image tagged with the sha of the commit so you know exactly what code was used to build the image.

Github actions ci pipeline to build and push docker image on commit to main

To automate the process above, a github actions pipeline can be setup which will push an image on every commit to main. In the app folder create .github/workflows/main-ci.yaml with the following content

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
name: main-ci

on:
  push:
    branches:
      - "main"

jobs:
  docker:
    runs-on: ubuntu-latest
    steps:
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Login to docker
        run: echo ${{ secrets.REGISTRY_TOKEN }} | docker login registry.xyz.io/v2 -u ${{ secrets.REGISTRY_USERNAME }} --password-stdin

      - uses: actions/checkout@v2

      - name: Build with Maven
        run: mvn -T32 clean compile package jib:build -DtagSha=$(git rev-parse HEAD)

Now on every commit to a main a new image will automatically be built and pushed to the registry.

The tags pushed can be seen by making a request on https://registry.xyz.io/v2/awesome-app/tags/list

Conclusion

Point your DNS to the IP of the server and you should be good to go

Once the image is pushed then all that is left is to update the sha of the app’s compose file and restart to get the new version running