The Bell-Ringer and the Broken Wire

There is a wire, a single copper strand, that runs from the old community hall to Errol’s shed. It’s strung, somewhat precariously, from a series of weathered wooden poles, dipping low over the lane behind the butcher’s shop. For fifty-two years, this wire has been the town’s Saturday morning alarm clock. At 9:00 AM sharp, the bell in the hall’s modest steeple rings, calling the town to its weekend market. The system predates my existence, predates digital anything, and its keeper is Errol.

To call Errol a sysadmin would be to miss the point entirely. He is the bell-ringer. His control panel isn’t a screen filled with dashboards, but a heavy, brass-handled switch in his shed, connected to a car battery. His monitoring system is his own hearing, calibrated over a lifetime to the specific, resonant tone of that single bell. His logbook is a mental one, a tally of Saturdays, of clear rings and dull thuds, of the one winter when ice snapped the wire and he spent a frantic hour splicing it back together by feel in the freezing dark. There are no backups for this. There is only Errol, and his wire.

I found myself thinking about Errol this week after a cascading failure in our cloud logging pipeline. A dozen microservices, each dutifully vomiting terabytes of structured data into a Kafka queue, were suddenly silenced not by their own faults, but by a misconfigured certificate rotation in a service nobody remembered we relied on. We had redundancy, we had playbooks, we had SLOs. What we didn’t have was the equivalent of that single, critical wire. We had a network so complex that its breaking point was, by design, meant to be invisible, rerouted around, and logged somewhere else. The silence was deafening.

The Logic of Simplicity

Errol’s system embodies a terrifying and beautiful logic. There is one point of failure, and it is known. He walks the length of the wire twice a year, checking the insulators. He knows which owl likes to sit on the seventh pole and might fray the line with its claws. The entire mechanism is comprehensible to one person, held entirely within one person’s mind and muscle memory. There is no abstraction, no layer of indirection. When the bell doesn’t ring, the fault is either with the switch, the battery, the wire, or the bell. The troubleshooting path is starkly linear.

Our world is built on the opposite principle. We hide the wires. We bury them under layers of abstraction, virtualization, and orchestration. We prize systems that are resilient, but not necessarily understandable by any single individual. The failure is not a snapped wire you can see, but a ghost in the machine, a policy conflict in an identity and access management system, a quota limit in an object storage bucket you’ve never directly interacted with. The silence is the same, but the path to the noise is a labyrinth.

I’m not advocating we all rip out our Kubernetes clusters and replace them with car batteries. The scale demands complexity. But Errol’s tradition offers a vital lesson in observability. True reliability might not be about having no single points of failure, but about having as few as possible, and knowing each one intimately. It’s about having a wire you can walk, a component whose state you can assess with a glance or a touch. It’s about accepting that some things are too important to be left to a system so complex that its silence becomes a puzzle. Sometimes, you just need a bell, a wire, and someone who knows the sound of both when they are working as they should.

Notes & further reading

A few pages I came back to while writing this: