Most backup mistakes I’ve run into don’t show up until someone actually needs to recover something. I’ve walked into more than one environment where backups were running fine — and nobody had ever tried to restore from them.
That’s always bothered me. A green backup job can make an environment look much safer than it actually is.
Over the years, I’ve run into the same three problems more than once: a backup that couldn’t produce a working system, servers that were never added to the backup policy, and a backup server that stopped doing its job without anyone noticing. Those experiences changed what I look at when I evaluate a backup setup.
Lesson 1: Backup Success ≠ Recovery Success
A backup job reporting “successful” tells you the data was copied. It says nothing about whether that data can actually bring a system back.
I found that out when a bare-metal restore didn’t boot even though the backup behind it had completed without a single error. The backup had done exactly what it was asked to do — capture the data — but nobody had verified that the restore path actually worked end to end.
The backup job itself wasn’t the problem. The problem was that we’d never tested the entire recovery path on that system. It was a useful distinction: a backup can be technically valid while the recovery process around it is not.
Lesson 2: A Policy Existing ≠ Being Protected
Backup policies get set up once, usually when a system is provisioned, and then quietly assumed to keep covering whatever gets added later.
That assumption broke when three new VMs went live and nobody had attached a backup policy to any of them — not because someone forgot to write one, but because the process for extending backup coverage to new systems wasn’t as automatic as everyone assumed.
A backup policy only protects what’s actually included in it. If a new server isn’t added, it isn’t protected — even if everyone assumes it is.
Lesson 3: The Backup Server Itself Can Fail
It’s easy to treat the backup server as the one piece of infrastructure that’s above the kind of failures it protects against. It isn’t.
Our backup server itself went down after one of its media disks filled up, and backups stopped running until someone noticed the problem.
Whatever monitors your production systems needs to be watching the backup infrastructure just as closely — disk space included.
What I Check Now
When I look at a backup environment now, I try to answer three questions:
- Can I actually restore a system from this backup?
- Are all production systems covered by a policy?
- Will someone know if the backup infrastructure itself stops working?
If any of those answers is unclear, I don’t consider the backup setup fully verified.
Why DR Testing Matters
Looking back, the common problem was pretty simple: we had tested parts of the backup process, but not always the entire recovery path.
That’s why regular DR testing matters. You don’t really know whether a recovery process works until you actually run it.
That’s also what led me to compare VMware snapshots with a full BMR restore — they solve different recovery problems, and “recoverable” means different things depending on what you’re trying to recover.
Related reading:
- The Backup Worked. The BMR Didn’t Boot.
- Three New Servers, One Missing Backup Policy
- The Day Our Backup Server Went Down
- VMware Snapshot vs BMR
- Why Every Disaster Recovery Plan Needs Regular Testing
Frequently Asked Questions
Is a successful backup enough proof that I can recover from it?
No. A backup job reporting success only confirms the data was copied — it doesn’t confirm the resulting backup can actually restore a working system. The only way to know is to test a restore, not just check the job log.
How do I make sure new servers get backup policy coverage automatically?
Don’t rely on it happening automatically. Add a backup-policy check to whatever process provisions a new server, and audit existing policies periodically against your actual server inventory. The mistake described above — three servers going live with no policy at all — happens exactly when that step is assumed rather than enforced.
What should I monitor on a backup server?
At minimum, I’d watch whether backup jobs are actually running, available disk space, and the health of the backup infrastructure itself. A monitoring setup that watches production systems but ignores the backup environment leaves an important gap.