TPTito Pinardo Gutiérrez
← Back to the rack
Español
1U · N8Nn8n · headless browser · every 30 min In production

Slot 9 of 25 in the rack · In production

1U unit · Automation

n8n · what happens on its own

Recorded only after confirmation

Tasks I used to do by hand that are now flows: every 30 minutes one checks a website with a headless browser, compares with what it has already processed, fetches what's new and hands it to another service. The hard part isn't doing it, it's not losing anything when something fails halfway.

Why n8n
Flows look like a graph, every run is logged with its data and can be rerun. For scheduled tasks between services it's easier to debug than a loose script.
The rule
Something is only marked done when the target service confirms it got it. Otherwise it stays pending and the next run tries again.
How it's deployed
Terraform creates the container and an Ansible role brings up n8n, the headless browser and the flow. Shared folders are created with the right owner.
  • 2,104orphaned browser processes found
  • 28 GBof RAM freed by closing them
  • 51failed runs because of one permission
  • 30 minbetween runs

Its code, in the open Public version of the repo: no IPs or secrets.

Inside a flow

Every half hour, a flow looks for new items and hands them to another service. Pick a flow to see where something could get lost and how that's avoided.

Flows Pick one and follow the packet.

Outside Automation container · Proxmox n8n + headless browser The flow Fetch Record without losing anything Operations Source Target Me Zabbix Internal door Every 30 min New items Filter Deliver Browser File By its name Clean-up Confirmed? Record Pending History Ansible role Owner 1000 Memory Docker proxy
Hermesvisiting the unit

Click a part or pick a flow. If you came from a skill, the parts where I use it are already lit.

      A normal run
      1. Scheduled run — 30 minutes pass and the trigger fires.
      2. The source website — The source is checked.
      3. Candidate list — The list of new items arrives.
      4. Filter against the record — The filter compares it with the record.
      5. Processed record — What's already processed is dropped.
      6. Filter against the record — Two new items are left.
      7. Headless browser — The headless browser fetches each one's file.
      8. The fetched file — The API answers with each file's name.
      9. Read by its exact name — Each item reads its own by exact name. Before, with two at once, they could get crossed.
      10. Delivery — They're delivered to the target service, each with its own data.
      11. The target service — The target answers for each one.
      12. Confirmation — Responses are matched with their items: both accepted.
      13. Processed record — Only now are they written to the record.
      14. Clean-up — And the temporary files are cleaned up.
      Something fails halfway
      1. Delivery — An item is delivered.
      2. The target service — The target doesn't accept it.
      3. Confirmation — No confirmation arrives.
      4. Pending — The item stays pending, out of the record.
      5. Run history — The failure stays in the history to review.
      6. Scheduled run — On the next run it comes back as new.
      A browser that doesn't close
      1. Headless browser — A page takes too long and the API errors.
      2. Container memory — Before, the error dropped the reference to the browser but didn't close it: the process stayed alive.
      3. Monitoring — The container's memory climbs day after day.
      4. Me — Until I notice: 31 GB used out of 32.
      5. Headless browser — Now every error closes the browser before forgetting it.
      6. Container memory — Orphaned processes stay at zero.
      It gets deployed
      1. Ansible role — Ansible applies the role.
      2. Folder owner — Shared folders, owned by the n8n user.
      3. Docker socket proxy — Docker access, through the proxy.
      4. Scheduled run — n8n starts with its flow imported.
      5. Internal door — The panel, behind the internal door.
      6. Me — I check it from home or the VPN.

      1. The source website

        Where the new items come from.

      2. The target service

        Receives each item and confirms whether it accepted it.

      3. Me

        I check failed runs when something doesn't add up.

      4. Monitoring

        Watches the container's memory.

        • Zabbix

      5. Internal door

        The n8n panel only opens from home or the VPN, with two-factor. It was taken off the internet.

      6. Scheduled run

        The flow's trigger.

        • Docker

      7. Candidate list

        What's on the source right now.

        • Docker

      8. Filter against the record

        Keeps only what hasn't been processed yet.

        • JavaScript / TypeScript

      9. Delivery

        Sends each item to the target service, with that item's data and not the first one in the batch.

        • Docker

      10. Headless browser

        A small Puppeteer API that opens the site the way a person would and downloads the file.

        • Docker
        • JavaScript / TypeScript

      11. The fetched file

        The API answers with its name; the content stays on disk.

        • Docker

      12. Read by its exact name

        Each item reads its file by the name the API returned, not «whatever is first in the folder».

        • JavaScript / TypeScript

      13. Clean-up

        Deletes the temporary files at the end.

        • Linux (Debian, Alpine, Arch)

      14. Confirmation

        Each response from the target is matched with its item. Only what was accepted goes into the record.

        • JavaScript / TypeScript

      15. Processed record

        Avoids repeating work. If something gets in here without being done, it's lost for good: nobody tries it again.

      16. Pending

        Whatever failed stays out of the record and comes back on the next run.

      17. Run history

        Every run, with each node's data. That's where diagnoses come from.

      18. Ansible role

        Brings up the stack, imports the flow and creates the shared folders.

        • Ansible

      19. Folder owner

        n8n runs without root, as its own user. Every folder it has to write to or delete from is created with that owner.

        • Linux (Debian, Alpine, Arch)
        • Docker

      20. Container memory

        Where it showed that something wasn't closing what it opened.

        • Linux (Debian, Alpine, Arch)

      21. Docker socket proxy

        n8n doesn't touch the Docker socket directly: it goes through a proxy that only allows what's needed.

        • Docker
        • Linux / SSH hardening

      What I use from my skill tree

      Pick a skill: the parts where I use it light up on the map and I tell you what for.

      Terraform
      Creates the automation container.
      Ansible
      The role that brings up n8n, the headless browser and the flow, with the folders and their owners.
      LXC
      Runs in its own LXC container.
      Docker
      n8n, the browser API and the socket proxy, in one stack.
      Linux (Debian, Alpine, Arch)
      Users and owners of the shared folders, and processes that really get closed.
      Zabbix
      Watches the container's memory.
      Linux / SSH hardening
      Panel only behind the internal door, n8n without root and Docker through a proxy.
      JavaScript / TypeScript
      The headless browser API and the flow's code nodes, in JavaScript.

      Incident log

      Real failures. Read the symptom, pick the cause you think it was and see how it was fixed.

      1. #1

        Items that vanished without a trace

        June 2026 · flow

        A run found three new items. Only one reached the target. The other two were never tried again.

        What do you think was going on?

        Show the fix
        Cause
        Two bugs together. The record was built from everything the filter saw as new, not from what the target confirmed. And with several items in one batch, file reading took «whatever was there», not each item's own.
        Fix
        Only what the target confirms is recorded, matching each response with its item. Each item reads its file by the name the API returned and is sent with its own data.
        Lesson
        A «done» record is written after doing, never before. Otherwise a failure stops being temporary and becomes permanent and silent.

      2. #2

        31 GB of 32 in use

        05-09-2026 · memory

        The container was about to run out of memory. n8n itself was using 264 MB.

        What do you think was going on?

        Show the fix
        Cause
        On any error, the API dropped its reference to the browser without closing it. The process lived forever: 2,104 browsers piled up over 22 days.
        Fix
        Close the browser on every error before forgetting it. Killing the orphans freed about 28 GB at once and, since the fix, the count stays at zero even when the site fails.
        Lesson
        Dropping a reference doesn't free what's behind it. Whatever is opened gets closed on the error path too.

      3. #3

        The same job, 51 times

        18-09-2026 · permissions

        For 28 hours the flow failed every half hour and repeated the same work over and over.

        What do you think was going on?

        Show the fix
        Cause
        After moving the service to code, the role created that folder without an explicit owner. Clean-up failed with «Permission denied», cut the flow before writing the record and the next run saw the same items as new again.
        Fix
        Create that folder in the same task as the others, owned by the n8n user, and rerun the role: only that task changed.
        Lesson
        When moving something to code, whatever used to be fine «just because» has to be written down: here, a folder's owner.

      Told in full in an article:

      How it was built

      1. June 2026

        The flow

        • First scheduled flow with a headless browser.
        • Record fix: only after confirmation.

      2. 16-09-2026

        To code

        • A new container with Terraform and an Ansible role for the whole stack.
        • The old one, stopped as a fallback.
        • Panel off the internet on 26-09.

      Decisions (and why)

      • When something is marked done

        When the target confirms it

        Marking it earlier is simpler and works 99 % of the time. The other 1 % is lost without anyone seeing.

      • Docker access

        Through a limited proxy

        Giving an automation tool the whole Docker socket is giving it the server. The proxy only allows the operations it needs.

      What I learned

      • Record after confirming, not before.
      • Whatever is opened gets closed on errors too.
      • When moving something to code, write down even what seemed obvious.

      What's next

      • A warning when a flow fails several times in a row, instead of finding out from the history.

      Shall we talk?

      I can walk through any of these decisions in detail in an interview.

      Download CV Get in touch ← Back to the rack

      Told generically on purpose: no sources, target services or the content being moved.