The Unwritten Ledger: On the Peril of Logging Everything
We are taught, from our first days tending to systems, to log everything. It is the first commandment of operations: Thou Shalt Have a Record. The logic seems unassailable. If you don’t log an event, how can you possibly debug it when things go wrong? Our tools make it easier than ever to ship every metric, every trace, every debug statement into vast, searchable data lakes. We operate under the assumption that more data is inherently better. I am here to argue that this maxim is not just flawed, but actively dangerous.
The common advice creates a seductive illusion of control. We believe that by capturing every possible datum, we are building a perfect, immutable record of system state. We are not. We are building a haystack, and with each new log line, we are adding more hay. When a critical needle—the true root cause of a failure—inevitably goes missing, we console ourselves by saying, "Well, it must be in there somewhere." We spend hours, then days, sifting through the chaff we ourselves created, blinded by the sheer volume of our own data. The log, meant to illuminate, instead obscures.
The Weight of the Witness
This practice carries a more insidious cost: the erosion of signal. When every event is logged with the same urgency, nothing is urgent. Important warnings drown in a sea of informational noise. Teams become desensitized to alert fatigue, and genuine crises can be scrolled past without a second thought. The log ceases to be a tool for the operator and becomes a burden on them.
Worse still, this "log everything" dogma ignores a fundamental truth: systems are not just technical constructs; they are also legal and ethical ones. Every log entry is a piece of evidence, a record of user action or system behavior that exists somewhere on a disk. By choosing to log frivolously, we are creating a permanent, and often unexplored, record that can be subpoenaed, leaked, or misinterpreted. We are building an uncurated archive of potential liability, all because we were afraid we might need one particular string of text on some unknown Tuesday in the future.
The wiser path is one of intentional omission. We must move from a mindset of hoarding to one of curating. This means making hard, thoughtful choices about what is truly necessary to record. It means valuing the quality and clarity of a hundred lines over the brute-force quantity of a billion. It means designing logging that answers specific questions about system health and user journeys, rather than capturing everything in the vain hope that the questions will become clear later. The most powerful entry in your ledger might be the one you consciously chose not to write.
Notes & further reading
A few pages I came back to while writing this: