The Pilgrim and the Lighthouse: On Two Kinds of Backup Vigilance
For years, I believed a backup strategy was a singular, monolithic discipline. You versioned your files, you shipped your logs, you replicated your databases. It was a matter of diligence, a checklist to be completed. The goal was a state of perfection: a pristine, complete copy of the present, safely stored in a vault far from the calamities that might befall the original. This approach has its place. It is the work of the Lighthouse Keeper.
The Lighthouse Keeper’s duty is to the structure itself. He ensures the light is bright, the lens is polished, the mechanisms are well-oiled. His backups are structural and grand. A cold storage snapshot of an entire volume, a full database dump, a complete copy of the application directory. When disaster strikes, the Keeper’s plan is to rebuild the lighthouse, stone by stone, exactly as it was. It is a powerful, comforting strategy. It promises a return to the exact point of failure, a complete restoration of the status quo ante.
But there is another, quieter form of vigilance, one I have come to appreciate more deeply. I think of it as the work of the Pilgrim. The Pilgrim is not concerned with rebuilding the entire cathedral at once. Instead, she carries with her a small, essential bundle. Her backups are incremental, continuous, and deeply personal. They are the small, versioned scripts that define a service’s unique personality: the odd `crontab` entry, the custom firewall rule that took a week to get right, the annotated nginx configuration that describes *why* a particular directive was set.
The Lighthouse Keeper backs up the data; the Pilgrim backs up the provenance. The Keeper has a complete copy of the `scripts/` directory. The Pilgrim has a commit history that shows the painful afternoon when the script was adjusted to handle daylight saving time quirks. One is a monument, the other is a diary.
Choosing Your Vigil
Neither approach is wholly sufficient on its own. The Lighthouse without the Pilgrim’s notes can leave you with a perfect but inscrutable artifact. You have the cathedral, but you’ve lost the architectural whims and the knowledge of which stones are load-bearing. The Pilgrim, traveling light, cannot possibly carry the entire structure in her pack. When a true catastrophe occurs—a hardware failure, a catastrophic corruption—her incremental notes are useless without a base structure to apply them to.
The true craft, then, lies in the symbiosis. The Lighthouse provides the foundation, the ultimate fallback. It is our guarantee against existential threat. But the Pilgrim’s work is what allows for graceful recovery, for understanding, for the subtle art of *maintaining* rather than just *restoring*. Her logs and versioned configs are the running commentary on the lighthouse’s operation. They are what transform a cold, static backup into a living, comprehensible system.
So we must be both Keeper and Pilgrim. We must tend the great, silent light that guards our shores against the catastrophic dark. But we must also walk the paths of our own systems, carefully noting the small adjustments and accumulated wisdom that give them life. The Keeper ensures we can survive the storm. The Pilgrim ensures we remember the way home.
Notes & further reading
A few pages I came back to while writing this: