TPTito Pinardo Gutiérrez
Español

Articles

What broke, told in full

My homelab's failures written up in full (what was seen, the cause and the fix), a digest every Monday of what's out and affects my server, and what changed each month. No addresses or internal names.

Latest

Build · 1U · BACKUP

From «no backups» to encrypted backups the server itself can't delete

For weeks my homelab had an acknowledged gap: the code had two backups, but the data had no regular copies. In one day I set up Proxmox Backup Server with a disk of its own, encrypted backups of everything every night, a permission that can't delete and a test restore. Along the way, two surprises: a permission that wasn't enough and 158 GB that weren't in any backup.

  • Proxmox Backup Server
  • Proxmox VE
  • ZFS
  • Linux / SSH hardening
Read →

Archive

October 2026

  1. Near miss 1U · TRAEFIK

    CrowdSec behind Cloudflare: I nearly blocked everyone

    Before putting CrowdSec at the door, the access log said 95 % of traffic came from Cloudflare. Blocking those IPs meant blocking nearly every visitor.

    CrowdSecTraefikLinux / SSH hardening

  2. Near miss 1U · GIT

    Renovate at home: three upgrades that never applied

    I set up Renovate to open a PR for every new version. On day one it uncovered three Ansible roles that bumped the number and left the old version installed.

    RenovateAnsibleForgejo · Actions

  3. Build 1U · GIT

    My own CI/CD with Forgejo: a runner with no secrets, deploys by PR and this site publishing itself

    Why my homelab's code now lives in a Forgejo at home, what each runner does, the four stumbles of day one and how this site goes from a push to the internet on its own.

    Forgejo · ActionsGitTerraformAnsibleRust

  4. Measured improvement 1U · STORAGE

    SAS disks shipped with write cache off: ×13 on writes with ZFS

    How to check and enable the write cache (WCE) on SAS disks and SATA disks behind a shelf with sdparm, why it's safe with ZFS and how much it gains, measured with fio.

    ZFSLinux (Debian, Alpine, Arch)

  5. Measured improvement 1U · ZABBIX

    False load alerts in LXC containers: the load average is the host's

    Why Zabbix warned «load average per CPU above 3» on eight LXC containers at once with no real load, and how alerts went from 9,500 a week to about a hundred.

    ZabbixLXCWazuh

September 2026

  1. Monthly changes 1U · IAC

    What changed in September 2026: the whole homelab as code

    142 changes in a month: the whole homelab moves to Terraform and Ansible, panels close to the internet, the assistant gets a new model and tests.

    TerraformAnsibleProxmox VELinux / SSH hardeningWazuhPython

  2. Incident 1U · PROXMOX

    A Proxmox container firewall that does nothing: [OPTIONS] enable: 1 is missing

    Firewall rules on a Proxmox CT that are stored but never applied: without the [OPTIONS] section with enable: 1, firewall=1 on the interface isn't enough.

    Proxmox VELinux / SSH hardening

  3. Measured improvement 1U · HERMES

    qwen3:8b on an 8 GB GPU: from 17 to 73 tokens per second without changing model

    Why an 8B model was slow on Ollama with an 8 GB GPU: the desktop was eating the VRAM. q8_0 KV cache, flash attention and the whole model on the GPU.

    LLM locales (Ollama, Qdrant, RAG)Linux (Debian, Alpine, Arch)

  4. Measured improvement 1U · HERMES

    Reason only where it helps: a local 8B model, from 4.7 to 0.7 seconds when writing

    When to switch qwen3's reasoning (think) on and off in a tool-using assistant: choosing a tool needs it, writing with the data in front of it doesn't.

    LLM locales (Ollama, Qdrant, RAG)Python

  5. Near miss 1U · IAC

    Terraform and Proxmox: an SSH key change that was going to destroy four containers

    In the bpg/proxmox provider, changing user_account.keys forces the container to be recreated (ForceNew). How the plan caught it and what to do instead.

    TerraformProxmox VE

  6. Build 1U · OKD

    Installing OKD with the Agent-Based Installer on spinning disks: the four walls

    An ISO that wouldn't boot, a full RAM overlay, etcd with 6.6 s writes and a 5-minute systemd timeout: how a 3-node OKD cluster ended up healthy.

    OpenShift (OKD)Kubernetes / K3sProxmox VE

  7. Incident 1U · N8N

    2,104 orphaned headless Chromes: a memory leak with Puppeteer

    A container using 31 GB of 32 while n8n used 264 MB: the culprit was a Puppeteer API that dropped the browser without closing it on every error.

    JavaScript / TypeScriptDockerLinux (Debian, Alpine, Arch)

August 2026

  1. Incident 1U · PROXMOX

    Proxmox starts no containers after a power cut: LVM-thin left half-activated

    «Activation of logical volume pve/data is prohibited while pve/data_tmeta is active»: why LVM-thin breaks after a power cut and how to bring it up.

    Proxmox VELinux (Debian, Alpine, Arch)

  2. Incident 1U · GAMES

    17 days of empty backups: cron's PATH has no /usr/sbin

    A backup script that worked by hand and failed silently under cron: the minimal PATH lacks /usr/sbin, where pct lives. How it was found and how to validate every copy.

    Linux (Debian, Alpine, Arch)Proxmox VE

June 2026

  1. Incident 1U · ROUTER

    A self-signed certificate across the house: a MikroTik NAT rule with no inbound interface

    Why a dstnat rule for port 443 without in-interface-list=WAN intercepted HTTPS from inside the house and broke HSTS sites, and how to fix it.

    MikroTikTraefik