What to do when Postgres suddenly complains about file permissions
Last update: 2025-01-26 15:37:17
tl;dr
Make sure you have storage space available for your database container.
The error message might be very misleading.
The situation
I am running harbor on my Kubernetes cluster to host container images for my blog, GoToSocial fork, and a few other internal things.
It randomly died two weeks ago when I was sick. I looked at the error message and it complained about file permissions for the database. This made no sense to me as I had not done anything to the database that could've broken the file permissions.
It scared me that something might be broken in my ceps cluster and I was too sick to actually go into debugging this.
Luckily, nobody except me relies on the container registry and since my cluster has already pulled all images, I could put off fixing this until I felt better.
Then last week happened, and for many reasons it was an absolutely horrible week, so I did not really have the energy to fix it, leading to a two week long outage, which is not great but the impact was essentially zero, so I was not too stressed about it.
Now, a while later, I finally had some time and energy to look into fixing it.
Firstly, I updated harbor to see if this might be a bug fixed in a newer version, but that sadly did not work.
Next, not actually believing it would work either, I just doubled the PVC size of the database storage, and restarted the pod.
🎉My registry worked again!
Why am I posting this?
When I was googling the error, I found absolutely nothing.
I am posting this tiny blog post in the hope that if someone else runs into the same issue, they will find it and be able to fix their issue more quickly.