The Two Rivers: On the Diverging Paths of Push and Pull
There is a fundamental schism in the way our small services communicate their state to the world, a quiet philosophical divide that dictates the flow of information. It is the difference between a whisper carried on the wind and a question asked in the stillness. In the realm of logging, metrics, and health checks, we are forever choosing between two rivers: the pushing stream and the pulling well.
One river, the push, is assertive and narrative. It is the service itself, conscious of its own being, deciding what is important and when to speak. It gathers its log lines, its heartbeat, its key measurements, and it actively dispatches them into the void, trusting they will find a home in a centralized log aggregator or a metrics database. This approach is born of a desire for immediacy and context. The event and its announcement are a single atomic unit in time. The service says, "This happened to me, now, and here is the exact state I was in." It feels proactive, a constant report from the front lines.
The other river, the pull, is patient and interrogative. It establishes a well of truth—the service’s own API endpoint, a metrics port, a status page. It sits in silence, waiting to be asked. The responsibility for knowledge shifts to an external collector, a monitoring agent that makes the rounds, peering into each well to draw out the current state. This method is one of delegation and standardization. The collector decides the rhythm and the questions: "Are you alive? What is your latency? How many connections do you have?" The service simply answers, truthfully and without embellishment, when called upon.
Neither path is inherently superior, but each carves a different landscape of complexity and failure. The push model can create a cacophony of data, a flood that must be managed and parsed. If the aggregator fails, messages are lost into the ether, and the service may be unaware, blissfully broadcasting to an empty room. The pull model, however, centralizes the point of failure on the collector. If it stumbles, the entire world goes dark and silent; every service could be thriving, but with no one to ask, they appear as dead stones. The pull requires a rigorous contract—a common language that every service must speak for the collector to understand.
Most of us operate a hybrid, a delta where the two rivers meet. Critical alerts might be pushed for urgency, while routine metrics are pulled for efficiency. The choice between them is less a technical mandate and more a reflection of temperament. Do we want our services to be storytellers, or do we want them to be librarians? The storyteller interrupts with news; the librarian offers wisdom only to those who seek it. Both are vital to knowing what is real.
Notes & further reading
A few pages I came back to while writing this: