Heartbeat Monitoring: How to Catch Silent Cron Job Failures

The silent failure problem

HTTP monitoring works because you can poke a URL and see if it responds. But cron jobs, scheduled backups, ETL pipelines, and background workers don't have URLs to poke. They just… run. Or don't. And when they stop running, nothing tells you until you notice missing data days later.

Classic examples: nightly database backups that silently fail for a month, Kubernetes CronJobs stuck in a crash loop that no one checks, invoice generation scripts that skip a billing cycle. The job doesn't error loudly — it just doesn't happen.

How heartbeat monitoring works

Heartbeat monitoring inverts the check. Instead of the monitor asking 'are you up?', your job tells the monitor 'I ran successfully.' You register a heartbeat with an expected interval — say, every 24 hours with a 1-hour grace period — and add a single line to your job that pings a unique webhook URL when the job completes.

If the ping arrives on time, the heartbeat stays healthy. If the ping is late by more than the grace period, an alert fires. No ping means the job didn't finish — which is exactly what you want to know.

Setting up a heartbeat in five minutes

1. Create the heartbeat

In the PingHarbor portal, add a new heartbeat with a descriptive name (e.g. 'nightly-postgres-backup'), an expected interval (24h), and a grace period (1h). You'll get a unique webhook URL.

2. Ping from your job

Add a single curl call at the end of your successful job path:

curl -fsS -X POST https://api.pingharbor.com/heartbeat/<your-uuid>

For cron entries: chain it with && so it only fires on success. For scripts: put it after your final commit or backup verification step, not at the start.

3. Notify on failure

Wire the heartbeat to your existing notification channels — Slack, PagerDuty, email. When it misses, you get paged the same way as any other monitor.

What to monitor

Good candidates: database backups, log rotation, certificate renewal jobs, data sync pipelines, weekly report generators, cleanup scripts, queue consumers with expected throughput, deployment health checks. Basically anything that runs on a schedule and matters if it doesn't.

Bad candidates: jobs that legitimately have variable runtimes with no upper bound, and one-off scripts. Heartbeats work best when 'late' is a reliable signal that something's wrong.