Tomasz Wieland

Tomasz Wieland

Runs three mini PCs and a NAS in the loft of a house in Ostheim and writes down what he set up and what broke. Builds CI pipelines for a living, which helps with none of this.

Incident

pve-2 keeps losing its network link

Three times in ten hours pve-2 dropped off the network for a few seconds and left the cluster. Power saving on its network card is the likeliest cause.

Monitoring Minor

TIL

lsblk shows the serial of the disk you are about to pull

lsblk takes a list of columns, and SERIAL is one of them. The serial is printed on the disk’s label, so it tells you which of four identical disks is sdc.

$ lsblk -d -o NAME,SIZE,SERIAL
NAME  SIZE SERIAL
sda   3.6T WX12D7A0K4L2
sdb   3.6T WX12D7A0K9F1
sdc   3.6T WX12D7A0J3N8
sdd   3.6T WX12D7A0K1C5

sdc reported 8 reallocated sectors this week. The new disk arrives on Monday, and I will pull the one labelled WX12D7A0J3N8, not the one in the third slot.

Changelog

v2.0.0: One inventory in YAML

The inventory is one YAML file, and failure alerts go to ntfy and a mail address set in one variable. Rename restic_repo before you upgrade.

Incident

Backups stopped for 31 hours

The NAS ran out of space on Thursday night and restic failed at 04:00. The alert went to an address I stopped reading in spring.

Resolved Major

A terminal on the NAS: the three newest restic snapshots of pve-1, from 2026-11-03, 2026-11-04 and 2026-11-05 at 04:00, and df showing /srv at 7.3T of 7.3T, 100 % used.

Post

88 watts less

The rack drew 62.5 watts on average in October, against 150 with the old tower in March. Over a year that is 767 kWh, about 245 euros.

1 min

A line chart of the rack’s power draw over a week: between 148 and 155 watts with the old tower, between 60 and 66 with the three mini PCs.

Post

The loft at night, after the fan curves

The NAS fans follow the disk temperature now instead of running flat out from 40 °C. Ten seconds of the loft at 23:00 are below.

1 min

PostMembers only

Before I update the cluster

Seven steps I go through before an update touches the cluster, one node at a time. For members, free.

1 min

Post

The VLAN weekend

The servers stay on VLAN 10, and phones, laptops and the TV moved to VLAN 20. From there only Jellyfin and DNS are reachable.

2 min

A network diagram: modem to opnsense, opnsense to the switch over VLAN 10 and 20, and the switch to pve-1 to pve-3, the NAS and the Wi-Fi.
Buy Logbuch