The Unseen Gardener: Why Your Perfect Logs Are a Distraction
We are taught, from our first days tending to systems, that good logging is the bedrock of observability. The common advice is exhaustive: log everything, structure it all in JSON, index every field, and create dashboards for every conceivable metric. We strive to become master cartographers, drawing ever more detailed maps of our digital domains. We are told that this meticulous record-keeping is the pinnacle of operational maturity. But I want to propose a heretical thought: what if this pursuit of perfect, comprehensive logs is, in itself, a form of operational debt? What if we are polishing the lighthouse lens while the storm rages outside?
The argument against this isn’t one of laziness, but of focus. The immense effort required to maintain a pristine, queryable log ecosystem—defining schemas, managing parsing pipelines, tuning indexes, and curating dashboards—is a massive cognitive and resource drain. This effort is often justified by the promise of future debugging ease. Yet, in my experience, the vast majority of these meticulously crafted logs are never read. They become a beautiful, expensive library where no one ever pulls a book from the shelf. The real work of fixing a crisis rarely involves a complex Kibana query; it involves a frantic tail of a single log file, a quick glance at a process list, or a restart.
The Signal in the Simplicity
I advocate for the role of the unseen gardener. This is the operator who spends less time building the perfect observation tool and more time knowing the garden itself. They favor a few critical, well-chosen signals over a firehose of data. A single, simple log line that screams “CANNOT CONNECT TO DATABASE” is infinitely more valuable in a moment of panic than a dozen structured JSON entries detailing connection pool metrics, handshake timings, and TLS versions, all waiting to be queried and graphed.
This approach embraces the boring and reliable not in the complexity of the tooling, but in the simplicity of the outcome. It asks: what is the absolute minimum I need to know that this service is healthy? What is the one line that will tell me it is catastrophically unhealthy? This forces a deeper understanding of the system's behavior rather than a superficial cataloging of its every action. The gardener doesn’t need a sensor on every leaf; they know the health of the plant by the colour of its stem and the tilt of its head towards the sun.
Let us not confuse the tool with the task. Our goal is not to have perfect logs; our goal is to have a stable, understandable service. Sometimes, the most reliable path to that goal is to log less, to log smarter, and to spend the time we save not on building a better log archive, but on knowing our systems so intimately that we need only the faintest whisper to understand their song.
Notes & further reading
A few pages I came back to while writing this: