TPTito Pinardo Gutiérrez
← Articles
Español

Near miss Published on · 1U · TRAEFIK

CrowdSec behind Cloudflare: I nearly blocked everyone

I wanted the homelab's public door to recognise scans and brute force and block them, with CrowdSec and its community list. Before switching it on I looked at what it would see: nearly every request came from a few Cloudflare IPs. CrowdSec would have blocked all of Cloudflare. The fix was teaching Traefik whom to trust, and checking that a fake header doesn't fool it.

  • CrowdSec
  • Traefik
  • Linux / SSH hardening

What I wanted

All web traffic from the internet comes in through Traefik, behind Cloudflare. CrowdSec reads its access log and recognises scenarios: scans of well-known paths, hunts for sensitive files, known exploits and brute force. It shares what it sees with its community network and gets back the list of malicious IPs others have seen. A Traefik plugin enforces its decisions at the entrance.

What it would have seen

Before installing anything I looked at the log: 95 % of requests came from a handful of IPs, all Cloudflare's. It makes sense: behind Cloudflare, the connection always comes from Cloudflare, and the visitor's IP travels in a header. Traefik didn't trust that header, so it logged Cloudflare's.

With that log, CrowdSec would have seen «Cloudflare» doing everything, blocked it, and the site would have gone dark for nearly everyone.

Whom to trust

Traefik now trusts the header only when the connection comes from Cloudflare's published ranges. If someone connects directly carrying that header, Traefik drops it and logs the real IP.

One trick remains: sending a fake header through Cloudflare. Cloudflare doesn't remove it, it appends the real IP. That's why CrowdSec and the plugin take the last IP in the chain, the one Cloudflare adds, not the first. I checked it with a line carrying a fake header: it doesn't fool them.

As a bonus, the services behind it now get the real IP too.

visitor → Cloudflare → Traefik
X-Forwarded-For: 203.0.113.9, 198.51.100.7
                 └ set by anyone  └ added by Cloudflare: this is the one that counts

Look before blocking

Even with the real IP, blocking someone legitimate (a friend on the game servers, me from outside) is worse than letting through a scan that finds nothing. So, a week of only watching: every morning a summary arrives with what it detected, how many community-list IPs came by and, separately, the IPs it would have blocked that got successful responses, the false-positive candidates.

And along the way, Traefik's logs rotate daily again: the access log was already at 570 MB.

Three days later

In 72 hours of only watching, CrowdSec would have blocked 7,875 requests from 143 IPs: probes, WordPress and panel hunts, sensitive files. The community list added another 17,451 known IPs. The 71 flagged IPs that had received some successful response were cloud scanners visiting public pages: not one known false positive. Since 18-10, it blocks at the door.