The Tinner's Rivets and the Cobbler's Wax
There are two ways to keep a thing from falling apart over time. One is to fix it so firmly that it cannot move. The other is to allow it to flex, knowing you will be there to mend the seam. In the quiet, unglamorous world of keeping services running, these philosophies manifest not as grand architectures, but as daily rituals. You are either a tinner, hammering rivets into a lasting hull, or a cobbler, applying wax to a thread that will need attention again tomorrow.
The Solid Hull
The tinner's approach is one of permanent fixtures. It's the meticulously versioned database schema, where every change is a migration, reviewed and tested, applied with a rollback plan. It's the service configuration baked into an immutable image, deployed as a unit. The rivet is the backup verification script that runs not just to see if files exist, but to restore a sample to a sandbox and run a checksum against known good data. The goal is a sealed system, a vessel that, once launched, should theoretically sail on without leaking. Its logs are the stamped serial numbers on the plates—a record of what was certified and when, useful mostly for proving the integrity of the original construction.
The Supple Seam
The cobbler, by contrast, works with softer materials. This is the approach of the cron job that has been `rsync`ing a directory to a backup server for a decade, watched over by a simple script that sends a message if the file count drops to zero. It's the log file that isn't ingested into a sprawling system, but is instead tailed by a human every morning, the patterns of its warnings and errors learned by heart. The wax is the daily, manual check. The seam is the understood, tolerated bit of drift—the knowledge that the backup server's clock is three minutes slow, that the old logging daemon occasionally duplicates entries. The system is not a sealed hull, but a well-worn boot; you know its creases, and you maintain it through attentive, recurring care.
Neither approach is inherently superior, but they demand different temperaments. The tinner spends their energy upfront, forging resilience into the design, trusting the rivets to hold. Their anxiety peaks at deployment, then (ideally) recedes. The cobbler's anxiety is lower at any single moment, but constant; their trust is placed not in the perfection of the artifact, but in the reliability of their own routine and attention. The tinner fears a single catastrophic failure of a rivet. The cobbler fears falling out of rhythm, missing a day of wax.
Most of us, in practice, are a blend. We rivet the core database, but wax the accessory scripts. We build an alerting system with solid joints, then develop a personal, habitual glance at a dashboard that tells us nothing the alerts haven't, simply because the rhythm of looking feels like maintenance. The lesson isn't in choosing a side, but in knowing which parts of your system are hull, and which are sole. Misapplying the rivet to something that needs to flex creates brittle points of failure. Applying wax to a critical structural joint is a recipe for a slow, fatal leak. The art is in knowing your materials, and your own capacity for care.
Notes & further reading
A few pages I came back to while writing this:
- Surprise, AZ
- The Lamplighter's Last Wick: On the Candle That Burns Before the Dawn
- Elk Grove, CA
- The Gardener's First Thaw: On the Roots That Drink From the Melt
- Pasadena, CA
- The Monitor's Unwinding Spool: On the Autoscaling That Unwinds the Thread It Holds
- New Haven, CT
- Stamford, CT
- Washington, DC
- one area's overview
- a practical rundown
- Little Rock, AR
- Gilbert, AZ