Skip to content
alunduil
Go back

How I'd Recover a Box I Can't Reach

Somewhere in the American Midwest, a NanoPi-NEO3 runs one service and nothing else. It’s a Tailscale exit node—the box that lets my traffic surface on a US address when I need it to—and that’s its whole life. I built it to be forgotten. Unattended upgrades keep it patched, a Grafana Alloy agent ships its metrics somewhere I can glance at them, and for months at a stretch I don’t think about it at all.

I live in London. For almost everything the box does, the ocean between us is a detail that never comes up.

What tells me something is wrong is silence. UptimeRobot watches for a heartbeat on a schedule, and when the check-ins stop, it pages me. The monitor has to run off the box: a dead box can’t report its own failure, and neither can the telemetry running on it.

Every hour, the box also backs itself up. A systemd timer fires an rclone job that pushes three things into a corner of my Google Drive: the list of every installed package, the whole of /etc, and the whole of /home. Configuration, package selection, and my files—everything a working system is made of, copied off the board and out of the house before anything can happen to the SD card it lives on.

So I’m covered. The data is safe, an hour old at worst, sitting somewhere I trust. Recovery, if it ever came to it, felt like a problem I’d already solved.

Then the heartbeat stops. Most of the time it’s nothing dramatic: a cable has worked loose, or the switch the box hangs off has rebooted, and the box is fine again the moment the network comes back. For all the planning that follows, the thing that takes this box down is usually the cable, not the card. And even that I can’t fix from London—a loose cable waits on someone walking over to the board.

The failures that actually need a plan are rarer: an SD card that has quietly corrupted itself, or an upgrade that went in wrong and won’t boot again. Either way the fix is the same, and it starts with a new card.

That’s the first problem, because I can’t touch the card. The people I have on-site are willing but not technical. What I can ask of them is small and physical: take out the old card, put in one that boots, and power the board back on. Everything after that has to come from me, over SSH, from another continent. So the plan has to work at that distance. It has to ask little of the hands at one end and forgive me at the other.

Even that first step isn’t fully solved. Someone has to flash the card, and the hands I have can’t. The best answer I have right now is to walk them through it on a video call, which is its own kind of fragile.

Once the box boots, most of the recovery is quick. A saved list of package selections goes back with dpkg --set-selections and an apt pass to pull them in. /home comes down from Google Drive with the same rclone that put it there. Grafana Alloy isn’t in Debian’s archives, so it gets reinstalled from Grafana’s own repository. Nine-tenths of the box comes back in a handful of commands, and none of it is where the risk lives.

The risk lives in the last tenth. There is a next move that suggests itself, and it is the wrong one. My backup has the whole of /etc, every configuration file the running system had, so the temptation is to rsync it straight back over the fresh image and be done. That is the failure mode.

The trap is /etc/fstab, the file that names which disk to mount as the root. A freshly flashed card has a new filesystem with a new UUID, and the base image’s fstab already points at it. My backed-up fstab points at the old card’s UUID, which no longer exists. Copy it over and the box can’t find its root on the next boot. It won’t come up, I won’t get an SSH prompt, and the only fix is a pair of hands in the Midwest.

Networking could strand me the same way. A wrong interface name or a stale static address, and there is no route home. I took that failure off the table on purpose: the box uses DHCP. Whatever the fresh image calls its interface, it asks the network for an address and gets one, and I can reach it. There is no network configuration to merge, because I chose not to have any.

It gets touchier still. The card I reflash to probably won’t be the same base image the box runs now. I built the current one from the vendor’s default and upgraded it by hand, and a rebuild would likely start from a different image. Its defaults in /etc will differ from mine in ways I can’t fully predict. That is one more reason a blind copy is reckless.

So the real work is smaller than it looks. The data was never the hard part; rclone has all of it. The hard part is /etc, and on a box an ocean away a wrong file there is not an inconvenience. It is a second trip I have to arrange from London. The discipline comes down to almost a single file: leave the fresh image’s fstab alone, let DHCP keep me on the network, copy the hostname back, and rsync only the application configs that are safe to restore.

There is a hole in all of this that a clean /etc merge doesn’t close. The thing that lets me reach the box at all is Tailscale, and Tailscale’s identity for this node lives in /var/lib/tailscale, which my backup never touches. It syncs /etc and /home and nothing else. So even a perfect restore brings the exit node back with no idea who it is on my network. I would have to re-authenticate it before it is an exit node again, and before I can reach it the usual way.

The fallback is uglier than I’d like. I can talk the people on-site through forwarding the SSH port on the router, so I connect straight to the box over the internet with no Tailscale in the way. But that only works if my key is already trusted by the machine I’m connecting to, and a fresh image is exactly the machine that hasn’t been told to trust it yet. The way back in depends on the access I’m trying to restore. I don’t have a clean answer for that yet; it’s the weakest link, and it’s on the list.

Every step of this is written down, and not one of it has been run. It’s a plan on paper, and I know why it stays that way. The only real test is a real restore: wipe the card, flash it, and walk the whole sequence on the live hardware. But the live hardware is a working exit node I depend on, sitting where I can’t get to it. To prove the recovery works I’d have to break the thing I’m protecting, on a box I can’t reach if the proof goes wrong. The distance that makes this plan worth writing is the distance that keeps me from testing it.

There is a better version waiting to be built. If I baked my configuration into the image itself, so a rebuild started from a card that already knew how to be this box, the merge would shrink toward nothing and the job at the far end would get simpler for the people doing it. That’s the direction. I haven’t built it yet.

For now I have hourly backups I trust and a recovery I’ve never rehearsed. Hands-off was always the point of this box; an untested plan for the day it isn’t is the tax I pay for the quiet.


Share this post on:

Next Post
Under the Instruments