The Unfailing Courier: How a Simple Cron-to-Curl Pipeline Became Our Most Trusted Messenger
We talk of backups and heartbeats, of logs and uptime. We build intricate systems to watch our systems, layering complexity upon complexity in the hope of achieving reliability. But sometimes, the most critical piece of operational awareness—the simple, undeniable notification that something has finished—is best handled by the simplest tool in the shed. I want to tell you about the most boring, most reliable piece of technology we’ve built: a cron job that curls a URL.
It started with a long-running data aggregation task. A script, initiated by a cron job at 2 a.m., would spend hours compiling reports. Its success was vital, but its completion was silent. We’d come into the workday and have to manually check a log file to see if it had even run, a small but persistent friction. We needed a signal. We considered message queues, dedicated monitoring agents, and other heavy machinery. Then we stopped ourselves. The requirement wasn't complex; it was binary: the job either finished or it didn't. We needed a courier to deliver a single, simple message.
The solution was a one-liner appended to the script: `curl -fsS -o /dev/null --retry 3 "https://our-internal-monitor.example.com/job-finished?name=nightly_aggregate"`. That’s it. The cron job itself handles the notification. On successful completion, it makes a single HTTP request to a dead-simple web service we built. This service does nothing but log the timestamp and job name to a file and, if the job is on a critical list, post a message to our internal chat. The magic is in the curl flags: `-f` fails silently on server errors, `-s` hides progress meters, `-S` shows errors even when silent, and `--retry 3` gives it a few chances in case of a transient network hiccup.
This pattern, the Cron-to-Curl pipeline, has since become our go-to for any non-critical process that needs to phone home. Database sanitization jobs, cache warmers, static asset compilers—each now ends with its own curated curl command. The elegance is in its decoupling. The script doesn’t need to know about chat APIs or authentication tokens; it just pings a URL. The receiving endpoint becomes a single point of control for all these notifications, a silent switchboard operator logging every call.
It is profoundly boring. It is also profoundly reliable. It leverages the universal language of HTTP and the battle-tested reliability of curl. It creates a clear, auditable trail. Most importantly, it transforms an invisible process into a witnessed event. We no longer wonder if the nightly work was done; the courier always arrives, and its message is always the same: the task is complete. In the end, that’s often the only message we truly need to hear.
Notes & further reading
A few pages I came back to while writing this: