Traefik¶

Traefik is a modern reverse proxy, directly plugged to the docker engine. It is in our opinion so much more convenient than nginx as all containers can be exposed simply via traefik just by setting some labels in their docker-compose service sections. This means that thanks to docker compose all the stack is version controlled.

Huge source of inspiration¶

If you only had to check one traefik resource in your life (other than official documentation, well written but terribly lacking concrete examples in our opinion), it would of course be Tiangolo one. Our work is heavily inspired by his 🥰. As usual we cannot thanks Tiangolo enough for sharing all his knowledge!

Setting up the VM¶

All the stacks assume that you have docker and related installed on your machine.

If working with a recent Ubuntu Operating System, you can go check on our setup script

Setting up the traefik stack¶

There is no sense in setting up locally Traefik, we thus just present the production setup.

All relevant information can be found in the docker-compose.traefik.yml file (here).

You will need to create one folder traefik_volume that will welcome certificates generated by let's encrypt.

Note

if you want to learn more on https and DNS, we highly recommend these funny resources:

Warning

Traefik combined with let's encrypt will create an acme.json file in traefik_volume. Do NOT change access rights of the file, nor edit it manually, otherwise you'll break all https accesses to your websites.

Warning

In the past two years, it happened to us once or twice that a human action broke the traefik container. When deleting and recreating the traefik container with docker-compose.traefik.yml, be aware that you recreate a new traefik-network. This network is then different from the one still used by both all your alive containers working hand in hand with traefik, and the ones presumably referenced (in a way too advanced for our current understanding of the matter 😭. If you know more, do not hesitate to contact us 😊.) in the acme.json. As a consequence, https will stop working. To restore it:

stop-remove all your containers
delete acme.json
recreate the Traefik stack
recreate the db stack
recreate all other needed stacks

This should restore correct https behaviour! Technical parenthesis that gave us some headache at some point closed 😂

You will need to create a .env file and setting

traefik_username: username that will be used in a basic-auth middleware that can serve as a basic authentication mecanism. it is used in this very stack to protect the traefik dashboard that comes out of the box (accessible at traefik_dashboard_url)
traefik_hashed_password: hashed PASSWORD associated to traefik_username. To be generated with openssl passwd -apr1 PASSWORD.
traefik_dashboard_url: entry to the DNS address you created in order to reach the traefik dashboard
traefik_volume: where to store the acme.json generated by let's encrypt
acme_email: the email of the person (hopefully) responsible for this stack. Has to be filled, the actual value is not that relevant (in our experience).

To start the stack in production, you can make use of the Makefile provided in the repo (make help to list available commands). Then launch

make traefik-launch

traefik dashboard will be accessible at traefik_dashboard_url (protected by traefik_username and PASSWORD)

Traefik middlewares¶

Out of the box, our docker-compose stacks implements five traefik middlewares, all related to security.

a middleware providing basic authentication
a middleware to ensure http to https redirection
an implementation of the fail2ban middleware
an implementation of the traefik-modsecurity-plugin: a web application firewall(WAF) middleware
a middleware to ensure secure https headers to urls protected by it.

Let us review them in order

Basic Authentication¶

This middlewares is quite straightforward. It protects applications that cannot do so on their own with a basic username and password mechanism.

To use it, add in the labels section of the traefik service:

# admin-auth middleware with HTTP Basic auth Using env variables USERNAME and HASHED_PASSWORD
# https://doc.traefik.io/traefik/v2.0/middlewares/basicauth/
- traefik.http.middlewares.admin-auth.basicauth.users=${traefik_username}:${traefik_hashed_password}

See the official documentation for more information.

http to https redirection¶

This middlewares is also quite straightforward. It ensures that all http requests are redirected towards https.

To use it, add in the labels section of the traefik service:

# https-redirect middleware to redirect HTTP to HTTPS
# https://doc.traefik.io/traefik/v2.0/middlewares/redirectscheme/
- traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https-redirect.redirectscheme.permanent=true

See the official documentation for more information.

fail2ban¶

This middleware does not exist in vanilla traefik, and a plugin has to be used in order for using fail2ban.

To install it, add in the command section of the traefik service:

# Add fail2ban plugin
- --experimental.plugins.fail2ban.modulename=github.com/tomMoulard/fail2ban
- --experimental.plugins.fail2ban.version=v0.7.1

Then in the labels section:

# Fail2ban middleware, see https://github.com/tomMoulard/fail2ban for more information
- traefik.http.middlewares.fail2ban.plugin.fail2ban.rules.bantime=1h
- traefik.http.middlewares.fail2ban.plugin.fail2ban.rules.enabled=true
- traefik.http.middlewares.fail2ban.plugin.fail2ban.rules.findtime=1m
- traefik.http.middlewares.fail2ban.plugin.fail2ban.rules.maxretry=500

The meaning of each option should be self-explanatory, and additional information can be found in the associated gihtub repository.

traefik-modsecurity-plugin¶

This plugin is morally a wrapper around owasp modsecurity crs (CRS means Core Rule Set, for french speaking person this could have been a good joke 😋, but it is not) .

To install it, add in the command section of the traefik service:

# Add waf plugin
- --experimental.plugins.traefik-modsecurity-plugin.modulename=github.com/acouvreur/traefik-modsecurity-plugin
- --experimental.plugins.traefik-modsecurity-plugin.version=v1.3.0

Then in the labels section:

# Waf middleware. See https://github.com/acouvreur/traefik-modsecurity-plugin for more info
- traefik.http.middlewares.waf.plugin.traefik-modsecurity-plugin.modSecurityUrl=http://waf:8080
- traefik.http.middlewares.waf.plugin.traefik-modsecurity-plugin.maxBodySize=10485760

the modSecurityUrl is the only compulsory option.

Warning

Contrary to what is explained documentation the port to put is 8080 (and not 80). Presumably the official documentation was not updated since this breaking change. This was the source of a lot of headache on our side, so if we can save your time, it is for the better 😊

the waf service defined in the label corresponds to the following service

  waf:
    # see https://github.com/acouvreur/traefik-modsecurity-plugin for more information
    image: owasp/modsecurity-crs:nginx
    container_name: waf
    environment:
      - BACKEND=http://dummy
    volumes:
      - ./RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf:/etc/modsecurity.d/owasp-crs/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf
    env_file:
      - ./.env
    networks:
      # Use the public network created to be shared between Traefik and
      # any other service that needs to be publicly available with HTTPS
      - traefik-network

Info

We use the nginx version and not the apache one for this very this very technical reason.
Core rule set is as its name clearly states a set of rules that trigger in specific conditions. Additional information can be read here and here
Unfortunately, this rule system can trigger some false positives (quite a lot in fact). It is a tedious task to find them for a specific application, and implement the custom exceptions associated.
This is done in our case with the AFTER exceptions (that applies at waf boot, just after all the rules have been loaded.
Exceptions should be put in a file called RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf sitting alongside with docker-compose.traefik.yml.

Other options are available and can be put in the asociated .env. They are documented here better than we could. The dummy service corresponds to

  dummy:
    # Subtlety to trick waf and avoid one of this for every service we want to protect with waf
    # info: https://korteke.medium.com/traefik-proxy-with-web-application-firewall-waf-cb4cd65f34f7
    image:  containous/whoami
    container_name: dummy
    env_file:
      - ./.env
    networks:
      # Use the public network created to be shared between Traefik and
      # any other service that needs to be publicly available with HTTPS
      - traefik-network

This is needed in order to protect several urls behind the same waf. The (subtle 😋) reason is explained here in a very pedagogical manner.

Secure headers¶

The security-headers middleware is heavily based on good header practices presented in the official traefik documentation.

Another resource that we found useful can be found here.

To create the security-headers middleware, add in the labels section of the traefik service:

# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#accesscontrolallowmethods
- traefik.http.middlewares.security-headers.headers.accesscontrolallowmethods=GET, OPTIONS, PUT
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#accesscontrolmaxage
- traefik.http.middlewares.security-headers.headers.accesscontrolmaxage=100
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#addvaryheader
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Vary
- traefik.http.middlewares.security-headers.headers.addvaryheader=true
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#hostsproxyheaders
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Host
- traefik.http.middlewares.security-headers.headers.hostsproxyheaders=X-Forwarded-Host
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#sslredirect
- traefik.http.middlewares.security-headers.headers.sslredirect=true
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#sslproxyheaders
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Proto
- traefik.http.middlewares.security-headers.headers.sslproxyheaders.X-Forwarded-Proto=https
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#stsseconds
- traefik.http.middlewares.security-headers.headers.stsseconds=63072000
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#stsincludesubdomains
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security
- traefik.http.middlewares.security-headers.headers.stsincludesubdomains=true
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#stspreload
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security#preloading_strict_transport_security
- traefik.http.middlewares.security-headers.headers.stspreload=true
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#forcestsheader
- traefik.http.middlewares.security-headers.headers.forcestsheader=true
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#framedeny
# https://developer.mozilla.org/fr/docs/Web/HTTP/Headers/X-Frame-Options
# If used with pgadmin, too restrictive https://www.pgadmin.org/docs/pgadmin4/development/config_py.html
# - traefik.http.middlewares.security-headers.headers.framedeny=true
# Less restrictive but still restrictive  https://developer.mozilla.org/fr/docs/Web/HTTP/Headers/X-Frame-Options
- traefik.http.middlewares.security-headers.headers.customFrameOptionsValue=sameorigin
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#contenttypenosniff
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Content-Type-Options
- traefik.http.middlewares.security-headers.headers.contenttypenosniff=true
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#referrerpolicy
# https://developer.mozilla.org/fr/docs/Web/HTTP/Headers/Referrer-Policy
- traefik.http.middlewares.security-headers.headers.referrerpolicy=same-origin
# https://doc.traefik.io/traefik/v2.0/middlewares/headers/#customresponseheaders
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy
- traefik.http.middlewares.security-headers.headers.customresponseheaders.permissions-policy=camera=(); microphone=(); geolocation=(); payment=(); usb=(); vr=();
# https://robots-txt.com/x-robots-tag/
- traefik.http.middlewares.security-headers.headers.customresponseheaders.X-Robots-Tag=none,noarchive,nosnippet,notranslate,noimageindex
# https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
- traefik.http.middlewares.security-headers.headers.customresponseheaders.content-security-policy=style-src 'self' 'unsafe-inline';
# see https://github.com/traefik/traefik/issues/9859 for this particular corner case
# Needed to please https://www.hardenize.com/
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection
- traefik.http.middlewares.security-headers.headers.customresponseheaders.X-XSS-Protection=0

Hopefully each header has its corresponding documentation and useful information listed.

Middlware chain.¶

To ease reuse of the middlewares, we provide two chains

# Add secured middleware chain
- traefik.http.middlewares.secured.chain.middlewares=fail2ban,waf,security-headers
# Add auth-secured middleware chain
- traefik.http.middlewares.auth-secured.chain.middlewares=fail2ban,waf,security-headers,admin-auth

To use them, just put in the SERVICE label section of your docker compose stack

# Enable auth-secured chain, using the middleware chain created above
- traefik.http.routers.SERVICE-https.middlewares=auth-secured