The Unnecessary Witness: On the Tyranny of the Log File
We are taught, from the moment we first configure a service, to open the floodgates. To log everything. INFO, DEBUG, TRACE—capture it all, for you never know which byte of stderr will become the Rosetta Stone for a future midnight outage. This is the gospel of observability, and we recite it with fervor. But I want to propose a heresy: what if our compulsion to log is not a act of vigilance, but one of intellectual laziness? What if the most reliable system is not the one that speaks the most, but the one that, by design, has the least to say?
Consider the humble cron job. Its entire existence is a predicate: if the time is X, then do Y. A perfectly written cron job, one that checks its preconditions and handles its errors gracefully, should produce exactly one line of output upon success: nothing. Its silence is its testimony. We have become so accustomed to the noise of our systems that we mistake that quiet for neglect. We feel a primal urge to inject a comforting logger.info("Task started successfully") into the void, not for the machine’s benefit, but for our own anxiety. We are building systems for our reassurance, not for their resilience.
This logorrhea creates its own universe of problems. We drown in the data, building elaborate and expensive pipelines to sift through the chatter in search of meaning. We alert on the ephemeral—a transient network blip logged as an ERROR—and become numb to the signals buried in the noise. The log file, intended as a witness, becomes a cacophonous crowd where everyone is shouting and no one is saying anything important. The very tool we built for clarity becomes the agent of obscurity.
The counterintuitive path, the one that challenges the common advice, is to practice log austerity. Before you write a log entry, ask: what action will this prompt? If a human is not going to be woken up by it, or if an automated system cannot be triggered to fix it, then the log is merely digital hoarding. Design systems whose success states are silent and whose failure states are loud, specific, and actionable. A service that logs a single, unmistakable cry of pain is infinitely more observable than one that murmurs a constant stream of existential dread.
True reliability is not born from a perfect record of everything that happened. It is born from a design so robust that most of what happens is unremarkable and need not be recorded. It is the confidence to let a well-engineered process run in the dark, trusting that its silence is the sweetest sound it can make. The goal is not to have the best logs, but to have a system that needs them the least.
Notes & further reading
A few pages I came back to while writing this: