Ingress-Nginx: Custom error pages that won't burn (chrome users') eyes out

Posted: 2021-08-07 17:51:49 |
Last update: 2021-09-22 19:52:03

What and why?

In my last post I wrote about how I moved (almost) everything I'm hosting into a Kubernetes cluster and how I am utilizing the Nginx Ingress Controller to route traffic to the correct pods.

This ingress controller has an interesting feature called the default backend.
This service handles all requests that cannot be mapped to an ingress (which would then be connected to a service to host something.)

In such cases, it will show a default error message.
This message discloses that you're running Nginx (which some consider a security issue, even though mitigating it seems a lot like security by obscurity but who am I to tell) and also tells your users roughly why their request could not be fulfilled.
These default error pages also contain no styling and no meta tag to tell the browser what color schemes they support.
This means when browsing something with a dark mode, clicking on a dead link will probably blind you if you're sitting in a dark room.
Luckily there is an easy way to serve custom errors.

So what to do?

Since the documentation is pretty thorough (as long as you find the correct place to look for the example manifest) I won't go into detail on how to set this up here.
This is a private blog and therefore much more prone to becoming outdated over the years, so it's best to just stick with the official docs here.
~~Just note that as of writing the example manifest contains an error so after copying it for your deployment, you still have to update the image tag.~~ This issue is fixed now.

Once you've got the error backend up and running, your error pages will be replaced by.. incomplete and extremely basic HTML.
You're no longer directly disclosing that you're running Nginx (if you cared about that, however since these example strings are public, finding this out would not take very long) but those pages don't solve the color scheme problem.
In order to fix this, you will have to override the HTML files served by the backend.

To start off, you will have to create a config map that looks something like this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: error-pages
  namespace: ingress-nginx

data:
  404.html: |-
    <!DOCTYPE html>
    <html lang="en">
      <head>
        <title>Page not found</title>
        <meta name="color-scheme" content="dark light">
      </head>
      <body>
        <h1>The page you are looking for could not be found.</h1>
        <p>Check the URL if there is a typo in there somewhere.<br/>
        Sorry for any inconvenience caused by this.</p>
      </body>
    </html>

This will create a replacement 404 error page that is valid HTML and has a little meta tag inside the head.
This tag tells browsers (those who want to listen to it, which is every major one except Firefox, sadly) that they can feel free to override the default colors with ones fitting the system's preference.
Since we're not actually setting any colors ourselves here, this can be done without risking any styles breaking.

If want to support Firefox as well, you will have to write some color rules using the CSS media feature.
This is not quite as quickly done as adding a single meta tag, but it also shouldn't take too long to do.

Now that we have our custom 404 page (note that for any other errors handled by the error backend, you will need to create another entry into the data field) we can make sure that it's actually served.
To do so, all you have to do is edit the backend's manifest to mount the config map.
To accomplish this, you will need a volume mount for the container

volumeMounts:
  - mountPath: /www
    name: custom-errors

and a volume for the deployment template spec.

      volumes:
        - name: custom-errors
          configMap:
            name: error-pages
            items:
              - key: 404.html
                path: 404.html
              [..]

For this example I have opted to manually specify which files should be mounted where.
You can also just mount the entire config map into this directory.
It's mostly personal preference but I feel like this gives me a bit control over what ends up where. (So any accidentally added entries won't be mounted)

Note that this will also overwrite the default json errors so if a REST client tries to call a route that is currently broken, they won't be getting a json response back.
To fix that, you can simply add the correct json files (404.json, etc) to your configmap and also mount those.

Conclusion

While this post is pretty basic, I hope it might be able to help someone (or just give them the idea to do something like this, even if they knew how easy it was before.)
I spend a large part of today writing an absolutely useless patch to the error backend (see the PR I linked earlier for more info) and I thought I might feel better about spending so much time on this if I wrote a blog post about it.

kubernetes tech infrastructure hosting