Continuous Deployment Pt. 1 - Making Docker Containers Accessible with Traefik

Imagine you have a piece of software you want to make accessible on the internet and that the software is packed as a Docker image. Imagine further, you want to (re-) deploy the software often, under a specific sub-domain, with a Lets Encrypt certificate, and maybe you require a simple form of access control. Once you have discovered how to do all of this painlessly, you want to repeat the process not for just one piece of software, but a bunch of it. Rinse and repeat.

flavor wheel

In the following, we will explain a rather simple approach for making Docker containers available which allows for a high degree of automation. The protagonist in this article will be Traefik. Basic knowledge of Docker and Docker-Compose will be helpful.

Enter Traefik

Traefik is an open source reverse proxy, load balancer and edge router and it is fairly easy to use. There are a couple of features which make it particularly interesting in our case:

  • It discovers Docker container as they start and stop.
  • It works descriptively by reading Docker-Compose files (of the target Docker container).
  • It has built in support for Lets Encrypt.

There are, however, other helpful features, too: Right out of the box, it brings a dashboard where you can see, which container have been discovered by Traefik. Also, it has monitoring / metric endpoints built in. So if you want to monitor your system using for example prometheus you can go right ahead.

For further details on how to enable the dashboard, please see the treafik documentation.

Setting up Traefik

For sake of simplicity, we will use a Docker-Compose file to set up Traefik.

To work properly, Traefik needs to be accessible on ports 80 for HTTP and 443 for HTTP/S (1). We explicitly create a network traefiknet, which is a bridge to the host (7), to achieve this. Make sure, ports 80 and 443 have been opened in your firewall.

In the setup we're laying out, port 80 is required so that Lets Encrypt can do its challenge / response requests when issuing a TLS certificate. Further requests on this port will be forwarded to port 443.

Containers, which are made accessible through Traefik, need to be reached by trafik. Otherwise, incoming requests cannot be forwarded to those containers. We deal with this by creating an external traefik_proxy network (8), which all "public" containers have to join. Obviously, Traefik has to join it, too (3).

Please note, only those containers need to join, which shall be exposed. So if you have a Docker-Compose file which describes a self-contained system, you would add the network only to those containers, which need to be reached from the web.

As we want Traefik to discover when Docker containers start and stop automatically, we need to give it access to /var/run/docker.sock (4).

Traefik needs a bit of configuration itself, so we mount the configuration file into the container (5). The acme.json file (6) is used by Traefik to write Lets Encrypt key information to it. We externalized both files, so we can recreate the container and backup the data independently.

version: '3'

services:
  traefik:
    image: traefik:v1.7
    container_name: traefik
    hostname: traefik  
    restart: always
    ports:
      - 80:80                                       # (1)
      - 443:443
    networks:
      - traefiknet                                  # (2)
      - traefik_proxy                               # (3)
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock   # (4)
      - ./conf/traefik.toml:/traefik.toml           # (5)
      - ./conf/acme.json:/acme.json                 # (6)

networks:
  traefiknet:
    driver: bridge                                  # (7)
  traefik_proxy:
    external:
      name: traefik_proxy                           # (8)

Creating the traefik_proxy network

As specified above, Traefik uses an external network to forward requests to containers it exposes. In the most simple case, creating the external traefik_proxy network is as easy as:

docker network create traefik_proxy

If you want to specify the details of your Traefik proxy network, please have a look at the Docker documentation (Docker Networks).

The only thing missing now, is the configuration file for Traefik: traefik.toml.

Traefik Config File

The Traefik configuration follows a basic approach which is used for typical Lets Encrypt setups. In our case, we're using the Lets Encrypt challenge approach. See https://docs.traefik.io/configuration/acme/ for more details on this topic.

In Traefik, entrypoints denote network ports where requests may be received. In this example, there are two entrypoints: port 80 for HTTP requests and port 443 for HTTP/S requests (2). Also, if Traefik receives requests on HTTP, we want them to be redirected to HTTP/S. If no entrypoint is specified by a deployed Docker container, the ones listed under (1) are used by default.

The domain name (4) and e-mail address (6) are required for the Lets Encrypt certificate.

The acme.json file (7) is specified, and also mounted by the Docker-Compose file above, to write Lets Encrypt data and keys to it. Create an empty file for the beginning. Traefik will take care of the rest.

Traefik needs a way to discover when containers start and stop, it does this by reading the docker.sock file (3). This is the reason, why we mounted it in the Docker-Compose file above.

We want to exert some control over which containers are exposed by Traefik, thus we set exposedByDefault to false. This way, we have to tell Traefik explicitly, which containers to make available.

debug = false
logLevel = "INFO"
defaultEntryPoints = ["https","http"]               # (1)

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]                     # (2)
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
  [entryPoints.https.tls]

[retry]

[docker]
endpoint = "unix:///var/run/docker.sock"            # (3)
domain = "<DOMAIN>"                                 # (4)
watch = true
exposedByDefault = false                            # (5)

[acme]
email = "<E-MAIL>"                                  # (6)
storage = "acme.json"                               # (7)
entryPoint = "https"
onHostRule = true

[acme.httpChallenge]
entryPoint = "http"

Okay, once you got your docker-compose, traefik.toml and the empty acme.json files ready, you are good to go.

Starting Traefik

Now, it is time to start Traefik. If you are in the same directory as your docker-compose file just run:

docker-compose up -d

Traefik should now be waiting for other containers to start up. You may watch it wait, by observing its log:

docker logs -f traefik

Once Traefik is up and running, let us deploy a test container and see that everything is working as planned.

Deploying a Container

In the following, we will deploy the nginx hello world container, but you should easily see, how this set up applies to more complex environments, too.

First, we need to make sure Traefik is able to reach the container. So we add the external traefik_proxy (7) network, we created in the steps above, and add the network to the conatiner (1). Once this basic plumbing is established we shall concentrate on the more interesting part: the labels section.

In (2), we tell Traefik, to make the container available under the <SUB-DOMAIN> of our <DOMAIN> (4), thus, it should be reachable under https://<SUB-DOMAIN>.<DOMAIN> (for example http://www.my-website.com). Internally, treafik will forward incoming requests via the network (6) to port (5) of the container.

On a side node: there are other ways for routing traffic to your Docker container. For example you can configure Traefik as a reverse-proxy and have it route based on the (url) path. Please refer to the Traefik documentation for further details.

The traefik.backend (3) designator is an identifier for backends to route web requests to. This becomes interesting, if you deploy the same container multiple times using the same traefik.backend label. Now Traefik will be able to load balance your requests. - But that is a topic for another time.

version: '3'

services:
  nginx:
    image: nginxdemos/hello
    container_name: nginx-hello
    hostname: nginx
    restart: always
    ports:
      - "8011:80"
    networks:                                 
      - traefik_proxy                               # (1)
    labels:
      traefik.enable: "true"                        # (2)
      traefik.backend: "nginx-hello"                # (3)
      traefik.frontend.rule: "Host:<SUB-DOMAIN>.<DOMAIN>" # (4)
      traefik.port: "8011"                          # (5)
      traefik.docker.network: "traefik_proxy"       # (6)

traefik_proxy:
  external:
    name: traefik_proxy                             # (7)

That's it! Start the compose file with docker-compose up -d. In the Docker logs docker logs -f traefik you should see, how Traefik starts collecting the TLS certificate from Lets Encrypt.

Congratulations, now your container is accessible from the web, with a valid TLS certificate. You may access the hello world container from your favorite browser, now. You can extend this example to deploy much more complex Docker-Compose files and expose only the front-facing containers. Quiet easily, you could use the Traefik annotations in your CI-CD pipeline, so an automatically deployed container becomes available on the web (as done here: Continuous Deployment Pt. 2 - Deploying Docker Containers with Ansible from GitLab).

DNS Settings

Have in mind, that for using sub-domains for your container, you need to set up your DNS entries accordingly. Either you operate an own DNS-Server, or you set up the A records of your DNS server, like this:

Host Name Record Type Target
* A IPv4 of your Server
* AAAA IPv6 of your Server

Adding Access Control

If a container shall not be openly accessible, you can add basic authentication (BasicAuth), treafik will take care of the rest. All you need to do, is add another label to your compose file:

  traefik.frontend.auth.basic.users: "USER:HASH,USER:HASH"

Just add a user name and a hashed password.

To create the password hash, you can use openssl, like this: openssl passwd -apr1 <PASSWORD>. Because the $ character is used for accessing environment variables in Docker-Compose files, you need to escape it: openssl passwd -apr1 <PASSWORD> | sed 's/\$/\$\$/g'

Further Reading