Proxmox

The cluster of three: nodes, guests, updates and quorum.

Incident

pve-2 keeps losing its network link

Three times in ten hours pve-2 dropped off the network for a few seconds and left the cluster. Power saving on its network card is the likeliest cause.

Monitoring Minor

Incident

Backups stopped for 31 hours

The NAS ran out of space on Thursday night and restic failed at 04:00. The alert went to an address I stopped reading in spring.

Resolved Major

A terminal on the NAS: the three newest restic snapshots of pve-1, from 2026-11-03, 2026-11-04 and 2026-11-05 at 04:00, and df showing /srv at 7.3T of 7.3T, 100 % used.

PostMembers only

Before I update the cluster

Seven steps I go through before an update touches the cluster, one node at a time. For members, free.

1 min

Post

The VLAN weekend

The servers stay on VLAN 10, and phones, laptops and the TV moved to VLAN 20. From there only Jellyfin and DNS are reachable.

2 min

A network diagram: modem to opnsense, opnsense to the switch over VLAN 10 and 20, and the switch to pve-1 to pve-3, the NAS and the Wi-Fi.
Buy Logbuch