The Gardener and the Archivist: Two Approaches to Log Rotation

There is a quiet, fundamental task that underpins the reliability of any service: managing the logs it produces. It’s a chore as old as computing itself, and how we approach it reveals a great deal about our philosophy towards our systems. I’ve come to see two distinct archetypes in this space: the Gardener and the Archivist. One prunes, the other preserves. Both aim for order, but their methods and motivations could not be more different.

The Gardener approaches a log file like a well-tended hedge. Their primary tool is the rotation, a scheduled cut that keeps growth in check. A log file grows, reaches a certain size or age, and is gracefully closed. A new, empty file takes its place while the old one is renamed, perhaps compressed, and eventually trimmed away by a retention policy. The goal is not to keep everything, but to keep enough—enough to diagnose recent issues, to spot a trend, to understand the last few hours or days of a system’s life. The Gardener values clarity and manageability. They believe a system is healthier when its outputs are kept neat and current, preventing the chaos of a single, monstrous file that slows down queries and fills a disk.

The Archivist, by contrast, sees each log entry as a potential historical artifact. Their instinct is not to rotate and remove, but to ship and store. Tools are set to watch the active log file, streaming its new entries in near-real-time to a central repository—an object storage bucket, a dedicated log server, a vast indexed database. The local file on the machine is almost an ephemeral concern; its true purpose is to be a brief buffer before the data finds its permanent home in the archive. The Archivist’s goal is comprehensiveness. They want the complete record, believing that the value of a single log line is unknowable until someday, much later, it becomes the key to understanding a complex failure.

Neither approach is inherently wrong, but each carries its own consequences. The Gardener’s system is simple, lightweight, and low-cost. But it risks the irreversible loss of data. That crucial clue from three weeks ago has been trimmed away, leaving a mystery unsolved. The Archivist’s system is resilient and historically rich, but it introduces complexity, external dependencies, and ever-growing storage costs. It can also lead to a paradox of abundance: with everything saved, finding the right needle in the mountainous haystack becomes its own monumental task.

The most reliable systems I’ve seen often employ a careful hybrid, a mindful blend of both philosophies. They might let the Gardener handle local disk hygiene, performing the essential pruning to keep individual machines running smoothly. Meanwhile, the Archivist works in the background, ensuring a copy of every trimmed log is preserved elsewhere, just in case. It is a recognition that while we must tend to the immediate needs of our systems, we also owe a debt to the future, to the questions we haven't yet thought to ask.

Notes & further reading

A few pages I came back to while writing this: