The Lighthouse and The Logbook: On Two Kinds of Knowing
There is a quiet schism in the world of small services, a fundamental divide in how we choose to know the state of our systems. It isn't about which monitoring tool is best, but rather about the very nature of the knowledge we seek. On one side sits the lighthouse, a beacon of immediate, undeniable truth. On the other rests the logbook, a patient, cumulative record of everything that passed.
The lighthouse approach is embodied by the humble health check endpoint. It’s a simple, dedicated route—often just /health—that, when queried, returns a stark, binary truth. A 200 OK means the vessel is seaworthy; a 500 error means it is not. It is a moment of pure presence. It answers one question and one question only: Are you alive, right now? This is a powerful, declarative statement. A load balancer doesn’t need to interpret nuanced error messages; it needs a yes or a no. The lighthouse beam cuts through the fog of complexity with a brilliant, singular flash.
Its strength is its simplicity, but this is also its profound limitation. The lighthouse tells you the ship is sinking, but it cannot tell you why. It offers no history, no context, no chain of events that led to the failure. It is a snapshot without a narrative.
This is where the logbook asserts its value. The logbook is the steady stream of structured events written to a file or a collector. It is the antithesis of the binary; it is the realm of the continuous and the contextual. It doesn’t just record the moment the database connection failed; it chronicles the gradual increase in query latency, the peculiar warning from the cache layer ten minutes prior, the unusual user agent that made a thousand requests in a second.
The logbook doesn’t answer a direct question immediately. Instead, it provides the raw material from which answers can be patiently reconstructed. When the lighthouse flashes its dire warning, our first instinct is to reach for the logbook to understand the story behind the catastrophe. Its strength is its depth, but its weakness is its passivity. It holds all the answers, but it does not proactively ring an alarm bell. It waits to be read.
The most reliable systems I’ve run don’t choose one over the other; they understand the conversation between them. The lighthouse is the trigger, the instinctual jolt that something is wrong. The logbook is the intellect, the source of reason and diagnosis. One provides the urgency, the other the understanding. One is the cry for help from the deck, the other is the ship’s manifest and navigational charts. You need both to safely guide your small, vital services through the dark and unpredictable water.
Notes & further reading
A few pages I came back to while writing this: