The Lighthouse and the Librarian: Two Philosophies of Logging
There is a quiet, fundamental schism in how we approach our logs. It’s not a debate about format or tooling, but one of purpose. On one side of the chasm stands the Lighthouse. On the other, the Librarian. One is built for the moment of crisis; the other for the long, patient investigation that follows.
The Lighthouse approach is one of immediacy. Its logs are a brilliant, sweeping beam cutting through the fog of a production incident. Every line is a signal, designed for rapid human parsing. It uses high-contrast language: “CRITICAL,” “FAILED,” “USER X UNAUTHORIZED.” It answers the question, “What is happening right now?” with brutal, actionable efficiency. The Lighthouse keeper isn’t concerned with the history of the ship; they need to keep it from crashing on the rocks *this very second*. This philosophy values sparse, high-fidelity events that scream for attention, often at the expense of contextual depth.
The Librarian, in stark contrast, is concerned with the complete record. This philosophy assumes that the true cause of an issue is never the single shout in the dark, but the subtle conversation that led to it. The Librarian’s logs are verbose, consistent, and richly contextual. Every event carries a transaction ID, a user agent, a full stack trace, a timestamp with microseconds. The Librarian doesn’t believe in “too much information”; they believe in the sanctity of the record. Their work begins when the alarm has stopped blaring, and they must retrace the steps, page by page, through the vast ledger of system activity to understand the ‘why’.
Most systems are an uncomfortable amalgamation of both. We have Lighthouse-style alerts firing from Librarian-style logs, creating a cacophony of false positives. Or we have a Lighthouse event that brilliantly tells us a service is down, but the accompanying Librarian logs are so anemic that the ‘why’ remains a maddening mystery. The tension is constant: the Lighthouse accuses the Librarian of burying the lead in a sea of noise, while the Librarian accuses the Lighthouse of providing a thrilling but ultimately useless headline without the accompanying story.
The wisdom, as with most things in our field, lies not in choosing one but in consciously designing for both. We must build Lighthouses—targeted, alertable events that act as our first responders. But we must also empower our Librarians—ensuring that behind every alarm is a deep, queryable, and unbroken chain of evidence waiting to be examined. One keeps the service alive tonight; the other ensures it becomes more reliable tomorrow.
Notes & further reading
A few pages I came back to while writing this: