The Ritual of the Echo Command: On Creating Your First, Imperfect Health Check

We speak of monitoring in grand terms: dashboards with a hundred flashing panels, intricate alerting rules, distributed tracing that maps the flow of a request across continents. It’s enough to paralyze anyone running a small, crucial service on a single virtual machine in the corner of a data center. The weight of "doing it right" can prevent us from doing anything at all. Today, I want to advocate for a different starting point: the ritual of the echo command.

The technique is laughably simple. Create a new cron job, perhaps one that runs every five minutes. Its sole duty is not to check disk space, or memory, or even if the database is up. Its duty is to simply be. It runs a script—a shell script, a Python one-liner, anything—whose primary function is to write a single line to a log file. Something like: echo "$(date) -- health_pulse -- OK" >> /var/log/heartbeat.log. Then, you set up a second, independent cron job, a sentinel. This one, running every ten minutes, reads the last line of that log file. It parses the timestamp. If that timestamp is older than, say, eight minutes, it triggers an alert. It can be as crude as sending an email, or a curl to a simple notification service.

The Philosophy of the Pulse

This is not a comprehensive health check. It is a pulse check. Its genius lies in its limitations. It doesn't tell you what is broken. It tells you, with stark and immediate clarity, that something fundamental is broken. The entire cron subsystem has frozen. The disk is full and can't write logs. The machine itself has locked up. The hypervisor has paused your instance. These are the silent, total failures that sophisticated application-level checks often miss because the agent running them is already dead.

By being separate from your application—running under the system's own scheduling, writing to a local file—this echo-pulse establishes an independent line of observation. It is the canary in a very basic coal mine. When this alert fires, you know the problem is environmental, systemic. It cuts through the noise of "high latency on endpoint /api/v2/widgets" and tells you the very ground your service stands on has shifted.

Implementing this takes minutes. The log file will grow, of course, which leads to the next beautiful, boring step: log rotation. You add a line to your logrotate.d config for /var/log/heartbeat.log, telling it to keep a week's worth of these pulses. Now you have a crude timeline. You can see when the machine was rebooted last, or if cron was restarted. The echo, the sentinel, and the rotation together form a tiny, complete, and robust system of observance.

It feels almost too trivial to document. Yet, in its triviality lies its power. It requires no new dependencies, no complex configuration, no understanding of a vendor's query language. It is a ritual that reinforces a vital ops principle: start by knowing if the thing is alive at all. Before you diagnose the patient's cholesterol, ensure the patient has a heartbeat. Let the sophisticated checks come later, built upon the certainty that this dull, regular echo is still sounding in the dark.

Notes & further reading

A few pages I came back to while writing this: