The Two Wells: On the Divergent Virtues of Logging and Metrics
There is a quiet, fundamental schism in how we understand our systems, a divide that runs deeper than the choice of tool or platform. It is the choice between two wells from which we draw our understanding: the deep, narrative well of logs, and the wide, statistical well of metrics. One tells a story, the other paints a landscape. Both are essential, yet they serve masters with entirely different appetites.
Logging is an act of preservation. Each line is a moment frozen in amber, a discrete event with a timestamp and a message. It is the scribe in the corner of the room, meticulously recording who did what and when. When a service stutters, we don our archaeologist's hat and sift through these strata of text. We look for the anomaly, the error message that doesn't belong, the sequence of events that led to the fault. This is a deep, narrow inquiry. We are reading a biography, searching for the single paragraph where the protagonist's fate was sealed.
Metrics, by contrast, are an act of distillation. They are not concerned with the individual event, but with its aggregate meaning. A metric takes a thousand login attempts and reduces them to a single number: success rate. It observes a hundred API calls and returns their average duration. This is a wide, shallow inquiry. It paints the weather map of our system's health—showing the high-pressure zones of normal operation and the gathering storms of an impending bottleneck. It answers the "what" and the "when" with startling clarity, but it is often silent on the "why."
The Choice of Thirst
The art of operations lies in knowing which well to approach for a drink. When the dashboard flashes red and the pager screams, the metrics well tells you *where* to look. The CPU graph is a spike; the error rate is a cliff. This is your compass. But it cannot tell you what caused the spike or who pushed the service off the cliff. For that, you must turn to the logs. You must follow the compass bearing provided by the metrics into the deep, textual record to unearth the specific failure, the misconfigured query, the unique user input that broke the pattern.
To rely solely on one is to see with a single eye. A world of only metrics is a world of symptoms without diagnosis, a beautiful map of a land you cannot truly navigate. A world of only logs is a library of every minute that ever passed, with no card catalog to find the one that matters. The steadfast reliability of our small services is born from drawing from both wells—using the wide-angle lens of metrics to know when something is wrong, and the telephoto lens of logs to understand precisely what it is.
Notes & further reading
A few pages I came back to while writing this: