The Flawed Gospel of the Single Source of Truth
It’s one of the most sacred incantations in our architecture documents, a phrase spoken with such reverence it might as well be italicized in holy script: The Single Source of Truth. It’s the promised land, the place where all data conflicts dissolve, where services bow before a central authority, and complexity is tamed. We’ve built entire philosophies, tools, and careers around chasing this particular dragon. But lately, I’ve come to a heretical conclusion, forged in the quiet, grimy reality of keeping small services running: the pursuit of a perfect, universal SSOT is not just a mirage—it’s actively harmful to the kind of boring, reliable technology we claim to love.
The problem isn’t the principle itself. Consistency is good. Knowing where to look for the canonical state of a customer or a transaction is essential. The problem is the architectural zealotry it inspires—the belief that this truth must be singular, instantaneous, and accessible to all. In our quest to eliminate all doubt, we build cathedrals. We reach for globally consistent databases, complex distributed transactions, and event-sourcing systems of such purity that they become the most fragile, complex, and central pieces of our entire operation. We’ve replaced the messy, understandable problem of “data is sometimes out of sync” with the catastrophic, opaque problem of “the source of truth is down.”
The Tyranny of the Central Nervous System
When you make one system the undisputed brain of your operation, you’ve also made it the single point of failure for your understanding of the world. That elegant, real-time API that serves the truth? When it stutters, everything that depends on it—from billing to diagnostics—goes blind or, worse, starts making faulty decisions based on cached lies. We traded the manageable, localized inconsistency of yesterday’s batch jobs for the systemic fragility of today’s always-on consensus.
This dogma also kills the simple, robust virtues we purport to uphold. It makes asynchronous, fire-and-forget logging feel dirty. It turns a service with its own durable, purpose-built storage into an architectural sin. It teaches us to distrust the humble, timestamped log file that has faithfully recorded what actually *happened*, in favor of querying the “truth,” which is merely a reconstructed *model* of what we believe the state should be. In an outage, I’ll take the messy, sequential log over the elegant, possibly-stale domain model every time.
Perhaps what we need is not a Single Source of Truth, but a Culture of Verifiable Truths. A world where services are allowed—encouraged, even—to keep their own authoritative records for their specific domain, built on technology they can actually operate. A world where reconciliation is an acknowledged, scheduled, and monitored process, not a hidden impossibility. Where we use logs as the immutable ledger of events, and treat derived state as just that: derived, useful, but fallible.
Truth, in the messy reality of networks and failovers and human operators, is not a point-in-time query. It’s a narrative, pieced together from multiple, sometimes conflicting, primary sources. Reliability comes not from enforcing a brittle singularity, but from designing for the graceful degradation of understanding, and from having clear, boring paths to rebuild it. Sometimes, the most reliable truth is the one you wrote down yourself, in your own log, at the moment it happened, before it had to journey through the fraught landscape of someone else’s perfect system.
Notes & further reading
A few pages I came back to while writing this: