Rotating Proxmox VE snapshots with keep-N: a small script and its sharp edges

We should be upfront about where we stand. BookMyHost's production cloud is not Proxmox. It runs Apache CloudStack 4.22 on KVM, with VM disks on Ceph RBD (an erasure-coded 3+1 pool on Micron 7400 NVMe) in Noida and Chennai. We still spend most of our working day on KVM and QEMU, and a lot of the people we talk to (homelab owners, small MSPs, a two-person IT team at a factory) run Proxmox VE. The same question keeps coming up: "How do I take a snapshot every night and keep only the last seven?"

Proxmox has no built-in scheduler for snapshot rotation. It does schedule backups through vzdump and Proxmox Backup Server, and you should use them. Snapshots do a different job: they are the instant "undo" you want before apt full-upgrade or a schema migration. pve-snapshot-rotate.sh is the small, boring tool we wrote for that job. This post covers how it works and where it can still hurt you.

The problem, stated narrowly

On each run we want three things:

1. Create one new snapshot per selected guest, VM or container.
2. Delete the oldest rotation snapshots until only N are left.
3. Leave every other snapshot alone, along with any guest that is busy.

Point 3 is where home-grown cron one-liners usually go wrong. A loop over qm listsnapshot that deletes "everything older than a week" will also delete the snapshot a colleague took by hand before an upgrade. A snapshot attempt on a guest that vzdump is backing up fails, and if the next step runs anyway it deletes an old snapshot without having made a new one.

Design decisions

Ownership by name. Every snapshot the script creates is named PREFIX-YYYYmmdd-HHMMSS (default prefix auto). The script only counts and deletes snapshots whose names start with PREFIX-. That is the whole ownership model. There is no database and no description parsing, and nothing to get out of sync. The prefix is validated against ^[A-Za-z][A-Za-z0-9_]{0,20}$, so it is always a legal Proxmox snapshot name.

The timestamp format sorts correctly as plain text. On the qm/pct path the script sorts by name. On the REST API path it sorts by the snaptime field that the API returns. Both give oldest first.

Create first, then delete. The new snapshot is taken before anything is removed. If creation fails, the deletion step is skipped for that guest. You never end up with fewer rotation snapshots because of a failed run.

Skip, don't fight. Before touching a guest, the script reads its config. If there is a lock: line (backup, snapshot, migrate and so on), it logs a warning and moves on. Guests carrying a maintenance tag (change it with --maintenance-tag) are skipped as well. That gives you a per-guest off switch in the Proxmox UI without editing cron.

One selector at a time. You pass exactly one of --vmids, --all or --tag. Passing two is a usage error (exit 1). We did this on purpose. "Tag backup, plus these three IDs" sounds handy until someone has to work out at 2 a.m. why a VM was or was not included. --exclude works with any selector.

Two backends. By default the script uses qm, pct and pvesh on the node itself. With --api it uses curl against /api2/json with an API token, so you can run it from a management box or a CI runner.

Walkthrough

Install it:

git clone https://github.com/geetmangal/bmh-infra-scripts.git
sudo install -m 0755 bmh-infra-scripts/proxmox/pve-snapshot-rotate.sh /usr/local/sbin/
pve-snapshot-rotate.sh --help

It needs Bash 4.4+ and jq (apt install jq on the node). Tag the guests you want rotated. Proxmox tags are set per guest in the UI or with qm set 100 --tags backup. Then preview:

pve-snapshot-rotate.sh --tag backup --keep 7 --dry-run

A dry run logs, per guest, how many auto-* snapshots exist, the name it would create and which ones it would delete. It changes nothing and does not send the webhook.

The real run is the same command without --dry-run. A few variations:

# two specific VMs, include RAM state, keep three
pve-snapshot-rotate.sh --vmids 100,101 --keep 3 --vmstate

# everything on this node except VMID 999, JSON summary on stdout, logs to a file
pve-snapshot-rotate.sh --all --exclude 999 --json --log-file /var/log/pve-snap.log

# a separate rotation set with its own prefix (e.g. hourly)
pve-snapshot-rotate.sh --tag critical --prefix hourly --keep 24

Because each prefix is its own rotation set, --prefix hourly --keep 24 and --prefix auto --keep 7 can run on the same guests without counting or deleting each other's snapshots.

The core of the rotation is short enough to quote:

total=$(( ${#existing[@]} + 1 ))   # existing + the one just created
to_delete=$(( total - KEEP ))
if [[ $to_delete -gt 0 ]]; then
  for ((i=0; i<to_delete && i<${#existing[@]}; i++)); do
    s="${existing[$i]}"
    delete_snap "$vmid" "$type" "$s"
  done
fi

existing is the list of prefix-matched snapshots from before the new one was created, oldest first. The loop bound i<${#existing[@]} means the snapshot just taken is never deleted. One consequence is that --keep 0 behaves like --keep 1, because the fresh snapshot always survives.

The JSON summary

With --json, stdout gets one object and all logs go to stderr:

{"node":"pve1","snapshot":"auto-20261025-013000","dry_run":false,
 "ok":5,"failed":0,"skipped":1,
 "guests":[{"vmid":"100","name":"web","type":"qemu","result":"ok",
            "created":"auto-20261025-013000","deleted":["auto-20261018-013000"],"error":null},
           {"vmid":"104","name":"db","result":"skipped","reason":"lock:backup"}]}

That is enough for a monitoring check or a dashboard: jq '.failed' and alert when it is non-zero.

Using the REST API

Create a token for a dedicated user with only the VM.Snapshot and VM.Audit privileges on the guests involved. Put user@realm!tokenid=secret in a file with mode 0600. The script warns if the mode is looser.

PVE_API_HOST=https://pve1.example.net:8006 \
PVE_API_TOKEN_FILE=/etc/pve-snap.token \
pve-snapshot-rotate.sh --api --node pve1 --tag backup --keep 14

--node matters. The script lists /cluster/resources?type=vm and keeps only guests on that node. On a cluster, run it once per node. If a guest migrates, it is picked up by whichever node holds it at the next run. PVE_API_INSECURE=1 turns off TLS verification. Use it only in a lab with a self-signed certificate you have not replaced yet.

Failure modes worth knowing

Storage without snapshot support. Snapshots work on ZFS, LVM-thin, Ceph RBD and qcow2 files on directory or NFS storage. A raw disk on plain (thick) LVM or on a directory store cannot be snapshotted. That guest is counted as failed and the run exits with code 3. This is correct behaviour. Either move the disk or exclude the guest.

ZFS rollback only goes to the latest snapshot. On ZFS, Proxmox can only roll back to the most recent snapshot. Rolling back further means destroying the newer ones. Keep-7 on ZFS still gives you seven restore points, but restoring an old one costs you the ones after it.

RAM state is big. --vmstate (qm only, running VMs only) saves the guest's memory alongside the disk snapshot, so expect roughly the VM's RAM size in extra storage per snapshot. It also takes longer. The default per-operation --timeout is 600 seconds. On the qm/pct path it wraps the command, and on the API path it is curl's --max-time. If a large VM hits it, raise the timeout rather than retrying in a loop.

API calls are fire-and-forget. The Proxmox API runs snapshot create and delete as tasks and returns a task ID (UPID) straight away. In --api mode the script treats an accepted request as success and does not poll the task. If the task fails later on the node, for example because the guest became locked in between, the run still reports ok. When you use --api, check the node's task log now and then, or use the local qm/pct path, which waits for each command to finish.

Prefix collisions. If someone names a manual snapshot auto-before-upgrade, it matches auto- and gets rotated out like any other. If people on your team use "auto" in manual names, choose a prefix such as rot or nightly.

Long chains. Each snapshot is a point the storage has to track. On qcow2 in particular, a long chain of snapshots can cost you write performance and space. Keep N small. Seven nightly plus a few hourly is plenty for an undo button. For anything older, use your backups.

Overlapping runs. The script does not take a global lock. If a run can take longer than your cron interval, wrap it in flock. Per-guest locks still protect each VM, but two runs can race on deletion counts.

Exit codes

Without --strict, skipped guests do not change the exit code. That suits nightly runs, where a guest that is being backed up is normal.

Running it from cron

The simplest setup is a cron file on each node:

# /etc/cron.d/pve-snapshot-rotate
30 1 * * * root flock -n /run/pve-snap.lock /usr/local/sbin/pve-snapshot-rotate.sh --tag backup --keep 7 --log-file /var/log/pve-snapshot-rotate.log --notify-webhook "https://hooks.slack.com/services/XXX"

--notify-webhook POSTs {"text": ..., "content": ..., "username": ...}. Slack and Mattermost read text and Discord reads content, so one URL format works for all three. The message carries the ok, failed and skipped counts and the snapshot name.

Schedule the snapshot run so it does not overlap your vzdump window. Overlap will not break anything, because locked guests are skipped, but you will get a lot of "skipped" lines.

Or a systemd timer

If you prefer journald and systemctl list-timers:

# /etc/systemd/system/pve-snapshot-rotate.service
[Unit]
Description=Rotate Proxmox snapshots (keep 7)

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/pve-snapshot-rotate.sh --tag backup --keep 7 --json

# /etc/systemd/system/pve-snapshot-rotate.timer
[Unit]
Description=Nightly Proxmox snapshot rotation

[Timer]
OnCalendar=*-*-* 01:30:00
Persistent=true
RandomizedDelaySec=300

[Install]
WantedBy=timers.target

systemctl daemon-reload
systemctl enable --now pve-snapshot-rotate.timer
journalctl -u pve-snapshot-rotate.service -n 50

A oneshot service will not start a second copy while the first is still running, so you do not need flock here. Persistent=true makes up a run that was missed while the node was down.

What this script deliberately does not do

It does not take backups, copy snapshots off the node or prune by age. For daily and weekly sets, run it twice with different prefixes, for example --prefix daily --keep 7 nightly and --prefix weekly --keep 4 on Sundays.

If you would rather not run the hypervisor yourself, BookMyHost sells NVMe KVM VPS hosted in India.

Get the script

• Repository: https://github.com/geetmangal/bmh-infra-scripts
• File: https://github.com/geetmangal/bmh-infra-scripts/blob/main/proxmox/pve-snapshot-rotate.sh

MIT-licensed, ShellCheck-clean, one file. Read it before you run it. Pull requests are welcome.

Script on GitHub · Repository