The Quiet Betrayal of the Backup Chain
We have been taught, as an article of faith, to revere the backup chain. It is the foundational ritual of our craft: a golden sequence of full, incremental, and differential backups, each link dependent on the last, creating a pristine timeline from which any past state can be resurrected. It is elegant. It is logical. We diagram it on whiteboards and defend its integrity with the fervor of archivists. But I have come to believe that this revered chain is, in many of our smaller systems, not a safety net but a subtle trap, a fragile monument to theoretical purity that quietly increases our risk.
The argument for the chain is mathematically sound: it saves space. A full backup is costly, so we interleave smaller, faster incrementals. The problem is not the math of storage; it's the psychology of recovery. A backup chain forces you to perform archaeology under duress. When the proverbial fire is raging, the last thing you need is a multi-step, order-dependent incantation to summon your data back from the void. The failure mode is no longer just a corrupted file; it's a broken link in the chain. If Tuesday’s incremental backup is lost or damaged, then Wednesday’s, Thursday’s, and every backup that follows becomes a useless artifact, a treasure map leading to a blank spot on the page.
We build these chains expecting a clean, controlled laboratory restoration. Reality, however, delivers chaos, caffeine-fueled panic, and the sinking dread of the clock. In that moment of crisis, complexity is the enemy. The more steps, the more potential for human error—the wrong restore order, a missed dependency, a forgotten transaction log. The chain, designed for efficiency, becomes a game of Jenga where pulling the wrong block collapses the entire tower.
The Heresy of the Complete Snapshot
The counterintuitive alternative, one that feels almost wasteful and unsophisticated, is the practice of creating complete, independent snapshots. Instead of a chain, you maintain a collection of discrete, restorable moments. Each backup is an island, whole unto itself. It is undeniably more costly in terms of storage. It feels less clever. There is no elegant dependency graph, just a brute-force copy of a system’s state at a point in time.
Yet, this approach embraces a different kind of elegance: the elegance of operational simplicity. In a crisis, the recovery process is reduced to a single, terrifyingly simple question: “Which point in time do we want?” You choose a snapshot and you restore it. There are no prerequisites, no cascading dependencies. The failure domain is contained. If one snapshot is corrupt, the others remain untouched, viable, and ready. This method trades theoretical storage efficiency for the profound, tangible efficiency of a guaranteed one-step recovery.
For small services, where the total data volume is manageable and the cost of storage is trivial compared to the cost of extended downtime, this heresy is not just practical—it is wise. It acknowledges that our greatest vulnerability is often not the hardware or the software, but the frayed nerves and cognitive overload of the person holding the keys during an emergency. The goal of a backup is not to be a perfect specimen of data engineering; it is to be a guaranteed path home. Sometimes, the straightest, dullest, most redundant path is the one that actually gets you there. The chain promises a clever journey. The snapshot simply promises you’ll arrive.
Notes & further reading
A few pages I came back to while writing this: