This is the kind of failure that looks like a NetBackup problem — right up until you check vSAN datastore free space. It started with a single VM. Every night, the same backup job showed up in the NetBackup activity monitor as failed, while everything else on the same policy completed fine.
The failure log wasn’t vague about it — it pointed straight at snapshot creation. To rule out a one-off fluke, I triggered the same backup manually. Same failure, same point of breakdown.
Tracing the History
Before jumping to conclusions, I pulled up how far back this had actually been failing. It went back further than I expected — all the way to around when we’d taken a large batch of snapshots for a separate, unrelated project. That timing was the first real thread to pull on.
Moving Upstream
NetBackup’s VMware backups work by having vCenter create a snapshot of the VM first, then reading from that snapshot while the VM keeps running. With the failure consistently landing on snapshot creation, I went to vCenter to check the cluster this VM was sitting on.
I pulled up the vSAN capacity dashboard. Free space was under 1%.
Why It Mattered
A VMware snapshot needs room to write every change the VM makes after it’s taken, and vSAN reserves a slack margin for exactly that. With free space that close to zero, there wasn’t enough margin left — so vCenter had nothing to give NetBackup, and the backup failed before it ever started moving data.
vSAN free space under 1% → no room for a new snapshot → NetBackup can’t start → backup job fails.
Fixing It
The datastore had accumulated more than it should have: old snapshots that were never cleaned up after an earlier project, a handful of orphaned files, and a few images nobody was using anymore. Clearing all of it brought free space back from under 1% to around 9%.
This wasn’t a case of hunting down stale snapshots for their own sake — it surfaced because an unrelated system, NetBackup, started failing for a reason that traced back to the same root cause: not enough room left to create anything new.
No policy change. No client reinstall. No NetBackup configuration touched at all. Once the storage had room again, I reran the same job and it completed normally.
Lessons Learned
A backup failure isn’t always a backup problem. NetBackup was doing exactly what it was supposed to do — it failed, correctly, at the step it couldn’t complete. The actual cause was a layer underneath it that NetBackup depends on but doesn’t control.
A bulk operation on one project can quietly starve everything else sharing that datastore. The snapshots that ate into this cluster’s capacity had nothing to do with this VM or its backup policy. They were leftovers from separate work, sitting there long enough that nobody connected them to a nightly backup failure until the history was traced back.
Preventive Actions
After this, vSAN free space went on the list of things I check first whenever a VMware-based backup job fails — right alongside the policy and client checks. A capacity alert that only fires once storage is already critical isn’t much use here; by then the backup has probably already failed for a while. It’s the same reasoning behind wanting a dedicated threshold per resource instead of one general infrastructure alert — the lesson from the Zabbix alerting issue I ran into before.
Frequently Asked Questions
Why does NetBackup need a VMware snapshot to back up a VM?
NetBackup’s VMware backups work by taking a snapshot first, then reading from that snapshot while the VM keeps running normally. That’s what lets the backup capture a consistent copy without powering off or pausing the VM.
How much free space does vSAN need before it blocks new snapshots?
There’s no single official number to aim for. The important point is that 1% wasn’t a target or a recommended threshold — it was simply the number I saw when I investigated this particular incident. The lesson isn’t “keep it above 1%”; it’s that free space needs its own dedicated alert well before it gets anywhere close to critical.
Related reading:
- This wasn’t the media server disk running out of space that took down backups before — that was storage on the receiving end. This time the problem was upstream, on the source datastore itself.
- Some of what filled this datastore were the same kind of leftover snapshots covered in VMware Snapshots Nobody Deleted — this time surfaced through a backup failure instead of a routine environment review.
- The reasoning behind wanting a dedicated capacity threshold ties back to The Alert That Never Fired.