Aller au contenu

Reverse proxy and https

How does a reverse proxy work

A reverse proxy works as the gateway to several web applications; all traffic goes to the reverse proxy, and it decides which app to get the content from. It can cache responses and mutate them before sending them back to the client.

Thus it creates a layer between web applications and the outside world. Speaking in terms of containers, a reverse proxy is the only container with published ports : it receives all incoming requests and fetches the responses from other containers. That means all your application containers become internal components, which can make it easier to scale, update, and secure them.

The reverse proxy catches the http request, then fetches the result from the corresponding web app and sends it back.

The host which made the request doesn't know where the web app it gets the result from is located, its only interaction is with the reverse proxy.

flowchart TD
    A{{Outside world}}

    A -->|HTTP request| B1
    B1 -->|Send result| A

    subgraph Internal Network
        B1[Reverse Proxy]
        B2[Web app1]
        B3[Web app2]

        B1 -.- B2
        B1 -.- B3
    end

Example

Suppose you have developed your api using FastAPI (this one for example), then you can test it locally by running the following (or similar) command.

1
uvicorn app.main:app --reload --workers 1 --host 0.0.0.0 --port 8000

and then you should get the following response on the terminal.

1
2
3
4
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Meaning that your api will run locally on the following address http://0.0.0.0:8000. You can even query it with tools like httpie or curl.

Querying with httpie
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
❯ http 0.0.0.0:8000/hello/
HTTP/1.1 200 OK
content-length: 17
content-type: application/json
date: Sat, 03 Sep 2022 20:16:07 GMT
server: uvicorn

{
    "Hello": "World"
}

Now that you've tested your api, you want to deploy it. Like everybody, you turn it into a docker image so that you can deploy it anywhere (well, at least on any Linux host).

Let's say that your api is registered under vorphus/helloworld-api:1.0-slim on dockerhub.

1
docker pull vorphus/helloworld-api:1.0-slim

1
docker run --rm vorphus/helloworld-api:1.0-slim

You should get the same response than before on the terminal.

1
2
3
4
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

But remember that the port described here is internal to your docker network, thus you have to forward it.

1
docker run --rm -p 8000:8000 vorphus/helloworld-api:1.0-slim
And now everything works.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
❯ http 0.0.0.0:8000/hello/
HTTP/1.1 200 OK
content-length: 17
content-type: application/json
date: Sat, 03 Sep 2022 20:16:07 GMT
server: uvicorn

{
    "Hello": "World"
}

TODO : write an example of localhost reverse proxy, like fastapi.localhost.

You're all set. You can now deploy it to a VPS/web host and buy a Domain Name to easily access it and enable https.

HTTPS

Note that https is not a property of reverse proxy, it's an internet communication protocol that protects the integrity and confidentiality of data between the user's computer and the site, but modern reverse proxies allow for an easy implementation of https.

To do https with a reverse proxy, you'll need :

  • A Domain Name, check Name.com to buy one, the cheapest one can be around 2 euros for a year.
  • A VPS, you can but one for example on DigitalOcean, for around 5 euros/month. That's the solution I have chosen.

Quote

Domain names and web hosting are two different services. However, they work together to make websites possible.

Basically a domain name system is like a massive address book that is constantly updated. Behind each domain name, there is an address of the web hosting service storing the website’s files.

Without domain names, it will not be possible for people to find your website and without web hosting you cannot build a website.

What’s the Difference Between Domain Name and Web Hosting (Explained)

Connect your domain name to your VPS

Now that you have a Domain Name, you have to connect it to your VPS. So first you'lle have to record your Domain Name to your VPS, in my case this is explained here.

You'll need to add this domain to your project and create a DNS record. Setting The apex domain @, and a wildcard * to get subdomains will be enough.

Now you have done that you also might need to change the Name Servers on your Domain Name provider side.

The update of worldwide DNS servers might take some time, but once it is done, you'll be able to ping your VPS using your domain name.

You can use for example one of the following command, they do not do the same thing, but a response from one of them is a sign that the connection between tour domain name and your vps has succedded.

1
2
3
whois mathieuklimczak.com
ping mathieuklimczak.com
dig @ns1.digitalocean.com mathieuklimczak.com NS

Traefik

Traefik is an open-source Edge Router that makes publishing your services a fun and easy experience. It receives requests on behalf of your system and finds out which components are responsible for handling them.

Traefik is natively compliant with every major cluster technology, such as Kubernetes, Docker, Docker Swarm, AWS, Mesos, Marathon, etc; and can handle many at the same time.

Dynamic configuration is one of the major benefits of a container-aware proxy like Traefik. You don’t need to start your upstream apps before you run Traefik because it watches for new containers while it’s running. You don’t have to restart Traefik or reload configuration to make a change to your application setup—that’s all part of your application deployment. Traefik has its own API and web UI that shows the rules, so you can run Traefik without any other containers and then deploy an application and see how the config gets built.

To do that, Traefik check the labels you put in the other containers of your stack to handle their connection to the virtual network created by the reverse proxy.

Traefik configuration

To be able to have one configuration for each environment (dev, prod), we'll use the overriding and composition properties of docker-compose. That way, we have a base configuration which we can extend and override with the development or production configuration.

Traefik base configuration

The base configuration is used for :

  • Defining the container,
  • Defining the porst we will use : 80 for http, 443 for https, and 8080 for the Traefik Dashboard.

We also tell Traefik that we will use Docker, so that the addresses it will have to redirect will comme from docker containers.

The network we will use will be the network called traefik-public, and it will be an external one, so that other containers like our backend, frontend, will be able to connect to it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
---
services:
    traefik:
        # yamllint disable-line rule:line-length
        # Use the latest v2.x.x Traefik image available, check https://hub.docker.com/_/traefik?tab=tags
        image: traefik:v2.8.4
        container_name: traefik-reverse-proxy
        ports:
            # yamllint disable-line rule:line-length
            - 80:80  # Listen on port 80, default port for HTTP, necessary to redirect to HTTPS
            # port_outside:port_inside_docker_network
            - 443:443  # Listen on port 443, default for HTTPS
            - 8080:8080  # expose 8080 for traefik dashboard
        labels:
            # yamllint disable-line rule:line-length
            traefik.enable: true  # Enable Traefik for the traefik dashboard, to make it available in the public network
            # yamllint disable-line rule:line-length
            traefik.http.services.traefik-dashboard.loadbalancer.server.port: 8080  # Define the port inside of the Docker service to use
            # yamllint disable-line rule:line-length
            traefik.http.routers.traefik-dashboard-http.entrypoints: http  # Make Traefik use this domain in HTTP
            # yamllint disable-line rule:line-length
            traefik.docker.network: traefik-public  # Use the traefik-public network (declared below)
        volumes:
            # yamllint disable-line rule:line-length
            - /var/run/docker.sock:/var/run/docker.sock:ro  # Add Docker sock as a mounted volume, so that Traefik can read the labels of the other services
            # yamllint disable-line rule:line-length
            - traefik-public-certificates:/certificates  # Mount the volume (named volume) to store the certificates
        command:
            # yamllint disable-line rule:line-length
            - --providers.docker  # Enable Docker in Traefik, so that it reads labels from Docker services
            # yamllint disable-line rule:line-length
            - --providers.docker.exposedbydefault=false  # Do not expose all Docker services, only the ones explicitely exposed
            # yamllint disable-line rule:line-length
            - --entrypoints.http.address=:80  # Create an entrypoint "http" listening on port 80
            - --accesslog  # Enable the access log, with HTTP requests
            - --log  # Enable the Traefik log, for configurations and errors
            - --api  # Enable the dashboard and API
        networks:
            # Use the public network created to be shared between Traefik and
            # any other service that needs to be publicly available with https
            - traefik-public

volumes:
    # yamllint disable-line rule:line-length
    # Create a volume to store the certificates, there is a constraint to make sure
    # yamllint disable-line rule:line-length
    # Traefik is always deployed to the same Docker node with the same volume containing
    # the HTTPS vertificates
    traefik-public-certificates:


networks:
    # yamllint disable-line rule:line-length
    # Use the previsouly created network "traefik-public" as an external network,
    # yamllint disable-line rule:line-length
    # shared with other services that needs to be publicly available wia this Traefik
    traefik-public:
        external: true

Traefik dev configuration

The configuration in development mode adds the address of the traefik dashboard : monitor.localhost.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
---
services:

    traefik:
        restart: unless-stopped
        labels:
            # yamllint disable-line rule:line-length
            traefik.http.routers.traefik-dashboard-http.rule: Host(`monitor.localhost`)
        command:
            # yamllint disable-line rule:line-length
            - --providers.docker  # Enable Docker in Traefik, so that it reads labels from Docker services
            # yamllint disable-line rule:line-length
            - --providers.docker.exposedbydefault=false  # Do not expose all Docker services, only the ones explicitely exposed
            # yamllint disable-line rule:line-length
            - --entrypoints.http.address=:80  # Create an entrypoint "http" listening on port 80
            - --accesslog  # Enable the access log, with HTTP requests
            - --log  # Enable the Traefik log, for configurations and errors
            - --api.insecure  # Enable the dashboard and API
            - --api

Traefik prod configuration

The configuration in development mode enables https.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
---
services:

    traefik:
        restart: always
        labels:
            # yamllint disable-line rule:line-length
            traefik.http.routers.traefik-dashboard-http.rule: Host(`monitor.mathieuklimczak.com`)
            # Use the traefik-public network (declared below)
            # HTTPS
            # Use https
            traefik.http.routers.traefik-dashboard-https.entrypoints: https
            # yamllint disable-line rule:line-length
            traefik.http.routers.traefik-dashboard-https.rule: Host(`monitor.mathieuklimczak.com`)
            traefik.http.routers.traefik-dashboard-https.tls: true
            # yamllint disable-line rule:line-length
            traefik.http.routers.traefik-dashboard-https.tls.certresolver: le  # Use the "le" (Let's Encrypt) resolver created below
            # yamllint disable-line rule:line-length
            traefik.http.routers.traefik-dashboard-https.service: api@internal  # Use the special Traefik service api@internal with the web UI/dashboard
            # yamllint disable-line rule:line-length
            traefik.http.middlewares.https-redirect.redirectscheme.scheme: https  # https-redirect middleware to redirect HTTP to HTTPS
            # yamllint disable-line rule:line-length
            traefik.http.middlewares.https-redirect.redirectscheme.permanent: true
            # Middlerware to redirect HTTP to HTTPS
            # yamllint disable-line rule:line-length
            # Anything that happend to our app on HTTP will be redirect by the middlerware
            # above on HTTPS
            # yamllint disable-line rule:line-length
            traefik.http.routers.traefik-dashboard-http.middlewares: https-redirect
            # admin-auth middleware with HTTP basic auth
            # Using the env variables USERNAME and HASHED_PASSWORD
            # yamllint disable-line rule:line-length
            traefik.http.middlewares.admin-auth.basicauth.users: ${USERNAME}:${HASHED_PASSWORD}  # Enable HTTP Basic auth, using the middleware created above
            traefik.http.routers.traefik-dashboard-https.middlewares: admin-auth
        command:
            # yamllint disable-line rule:line-length
            - --providers.docker  # Enable Docker in Traefik, so that it reads labels from Docker services
            # yamllint disable-line rule:line-length
            - --providers.docker.exposedbydefault=false  # Do not expose all Docker services, only the ones explicitely exposed
            # yamllint disable-line rule:line-length
            - --entrypoints.http.address=:80  # Create an entrypoint "http" listening on port 80
            # yamllint disable-line rule:line-length
            - --entrypoints.https.address=:443  # Create an entrypoint "https" listening on port 443
            # yamllint disable-line rule:line-length
            - --certificatesresolvers.le.acme.email=klimczak.mathieu@pm.me  # Create the certificate resolver "le" for Let's Encrypt, uses the environment variable EMAIL
            # yamllint disable-line rule:line-length
            - --certificatesresolvers.le.acme.storage=/certificates/acme.json  # Store the Let's Encrypt certificates in the mounted volume
            # yamllint disable-line rule:line-length
            - --certificatesresolvers.le.acme.tlschallenge=true  # Use the TLS Challenge for Let's Encrypt
            - --accesslog  # Enable the access log, with HTTP requests
            - --log  # Enable the Traefik log, for configurations and errors
            - --api  # Enable the dashboard and API

Caddy2