A power cut of three seconds rebooted the Pi that answers DNS, and its SD card came back read-only. Every device in the house depended on it.
Resolved
Critical
Nothing in the house could resolve a name for 98 minutes on Saturday evening. A power cut of about three seconds rebooted pi-dns, the only machine in the loft that is not on the UPS, and its SD card came back read-only. Unbound could not start, and the router handed out pi-dns as the only DNS server.
Timeline
- 2026-10-03 19:12 CEST Power cut of about three seconds. The rack stays up on the UPS,
pi-dns reboots. - 19:14 Investigating: Websites stop loading on every phone and laptop.
ping 9.9.9.9 works, so the line is fine. - 19:31 Identified: Unbound on
pi-dns fails to start. The root file system is mounted read-only after errors on the SD card. - 19:44 Unbound enabled on the router, which the DHCP server now hands out as a second DNS server.
- 20:20 Monitoring: Phones and laptops resolve again once they renew their lease. The TV holds on to its old one.
- 20:50 Resolved: The TV renews its lease. Every device has two DNS servers now.
What went wrong
We put the rack on a UPS at the end of August and forgot the Pi, which hangs on the wall beside it with its own power supply. One DNS server was fine for as long as it never rebooted at a bad moment, and an SD card makes every reboot a bad moment.
What changed
pi-dns is on the UPS.- It boots from a USB SSD now. The SD card is in a drawer with a label on it.
- The DHCP server hands out
pi-dns first and the router second. - Leases last four hours instead of a day, so the next change reaches the TV the same evening.