By the end you have DNS proxied through Cloudflare, TLS validated end to end with a real origin certificate, a WAF, admin surfaces behind Access, and a packet filter that leaves the origin reachable only from Cloudflare and your tailnet. The last part is the point: everything before it is a suggestion an attacker walks past if port 443 answers on the public IP.
What the edge does, and what it does not
Cloudflare terminates TLS, absorbs floods, applies WAF and bot rules, caches, and hides the origin address. It does not protect an origin anyone can reach, and hiding the IP is not a control: certificate transparency and DNS history leak it.
DNS: one proxied record per name
One proxied record per hostname, no wildcards: wildcards make cache and WAF rules impossible to reason about and issue certificates for names you did not intend.
app.example.com. 300 IN A ${VPS_IP}
app.example.com. 300 IN AAAA <origin IPv6>
Publish IPv6 and skip the ip6tables half of the lockdown, and v6 is the hole.
dig +short app.example.com
# 104.x.x.x or 172.6x.x.x => proxied, good
# ${VPS_IP} => DNS only, the origin is exposed
Keep _acme-challenge free of CNAMEs: it is the one name the certificate pipeline owns, and a wildcard there is the usual way DNS-01 stops working months later.
TLS modes: Full strict or nothing
The modes that matter: Flexible sends Cloudflare to your origin over HTTP: redirect loops and plaintext, never use it. Full encrypts but does not validate the origin certificate, so a self-signed one passes. Full (strict) requires a certificate valid for the hostname, signed by a public CA or Cloudflare Origin CA; anything else is a 526.
Strict is the only mode where the padlock means anything end to end. It is easy to end up on Full for months: ACME had never worked, Traefik served CN=TRAEFIK DEFAULT CERT, and Cloudflare was talking to a certificate nobody validated. Strict would have turned every request into a 526. The lesson is not to relax TLS, it is to fix issuance and then tighten.
# Run on the VPS. Loopback is the one place you can always talk TLS to the origin.
openssl s_client -connect 127.0.0.1:443 -servername app.example.com </dev/null 2>/dev/null \
| openssl x509 -noout -issuer -subject -dates
# issuer shows O=Let's Encrypt => ready for Full (strict)
# issuer=CN=TRAEFIK DEFAULT CERT => stay on Full, fix ACME first
The alternative is a Cloudflare Origin CA certificate: signed by Cloudflare, valid up to fifteen years, trusted only by the edge, installed as a static file in Traefik. It buys Full (strict) in ten minutes, no ACME, no DNS token. The costs: browsers do not trust it, so the tailnet path shows a warning, and you reissue on every hostname change. DNS-01 is the better fit when the dashboard is also reached over the tailnet.
ACME DNS-01 with a scoped token
HTTP-01 needs inbound port 80 from Let's Encrypt, which the lockdown forbids. DNS-01 needs only the ability to write a TXT record, so it survives the lockdown and works for wildcards.
The scope is exact: Zone:Read plus Zone:DNS:Edit, on the zones that hold your records. Zone:Read resolves the zone ID, Zone:DNS:Edit writes and deletes the challenge record. No account scope, no zone settings, no cache purge.
The 403 we hit: with Zone:Read alone, acme.json stayed empty and Traefik logged cannot get ACME client cloudflare, with 403 10000 Authentication error when lego tried to create the TXT record. The token was valid and could list zones, which is what makes it confusing: authentication succeeds and the write fails.
The second failure is packaging. The token was a Docker secret at /run/secrets/CF_API_TOKEN, but lego reads the environment, and the name is provider-specific: CLOUDFLARE_DNS_API_TOKEN. Without it, Traefik reports missing credentials and never attempts a challenge.
# /srv/traefik/.env, mode 0600 root:root. Compose reads this file automatically.
CF_API_TOKEN=<scoped token with Zone:Read + Zone:DNS:Edit>
services:
traefik:
environment:
- CLOUDFLARE_DNS_API_TOKEN=${CF_API_TOKEN}
certificatesResolvers:
cloudflare:
acme:
email: [email protected]
storage: /letsencrypt/acme.json
dnsChallenge:
provider: cloudflare
resolvers:
- "1.1.1.1:53"
- "8.8.8.8:53"
delayBeforeCheck: 10
docker compose up -d traefik
docker logs traefik --since 5m 2>&1 | grep -i -E 'acme|certificate' | tail -20
docker cp traefik:/letsencrypt/acme.json /tmp/acme.json
jq '.cloudflare.Certificates | length' /tmp/acme.json
# > 0 means certificates exist
docker exec traefik ls -l /letsencrypt/acme.json
# -rw------- root root
rm -f /tmp/acme.json
Traefik refuses an acme.json whose permissions are too open, and the error mentions permissions, not certificates. It lives in a named volume, so the daemon manages ownership. Renewal matters more than issuance: Traefik renews at thirty days left, and a token rotated without updating .env fails silently until expiry. Alert on notAfter from outside, not on the ACME log.
Cache: rules versus the origin's headers
Cache rules decide eligibility and can override the TTL; the origin's Cache-Control is the only thing that knows whether a response is user-specific. The origin is authoritative, rules only narrow eligibility.
Hashed assets get public, max-age=31536000, immutable on a filename that changes every build, so a deploy never needs a purge. HTML gets private, no-cache. s-maxage is the edge-only TTL: Cloudflare honors it, browsers ignore it, which is how you say ten minutes at the edge and zero in the browser.
Cache Everything on HTML is how one visitor's page reaches another and how a deploy appears to take an hour. Cache /build/*, /storage/*, fonts and images. Not documents.
ASSET="$(curl -s https://app.example.com/ | grep -o '/build/assets/[^"]*\.js' | head -1)"
curl -sD - -o /dev/null "https://app.example.com${ASSET}" | grep -i -E '^(cache-control|cf-cache-status|age|etag)'
# cache-control: public, max-age=31536000, immutable
# cf-cache-status: HIT on the second request
curl -sD - -o /dev/null https://app.example.com/ | grep -i -E '^(cache-control|cf-cache-status)'
# cache-control: no-cache, private
# cf-cache-status: DYNAMIC
Use GET, never HEAD: Laravel's SetCacheHeaders only applies on GET, so a HEAD check reports no-cache, private on a correctly cached asset.
WAF, bots and Access
Enable the Cloudflare Managed Ruleset and the OWASP Core Ruleset, then add what you need: rate limits on /login and /api by IP and path, not globally. Add narrow skip rules with a written reason.
Bot Fight Mode challenges anything that does not look like a browser, including your crawler and every API client. A crawler you run can be whitelisted by user agent on the app side, which is spoofable by design; do not repeat it at the edge. Verified Bots relies on reverse DNS, and an address with no PTR cannot validate. Use a skip rule on a secret header plus a source IP, and rotate it when a contractor leaves.
Access covers admin surfaces: the dashboard, /admin, queue UIs. One Zero Trust application per hostname or path, a policy by IdP group, sessions in hours, service tokens for CI.
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://traefik.example.com/
# 302 https://<team>.cloudflareaccess.com/login => Access is in front
curl -s -o /dev/null -w '%{http_code}\n' \
-H "CF-Access-Client-Id: ${CF_ACCESS_CLIENT_ID}" \
-H "CF-Access-Client-Secret: ${CF_ACCESS_CLIENT_SECRET}" \
https://traefik.example.com/
# 401 => through Access, stopped by Traefik's basic auth: two layers, as designed
Access, basic auth and the tailnet allowlist are three independent gates: we accept any two, never one.
Locking the origin to Cloudflare
Two layers: the cloud firewall, which survives a broken host, and the host's DOCKER-USER chain, because Docker publishes ports past ufw and firewalld. A host rule you can flush with a careless command is not a control, and a cloud rule alone leaves every container reachable when the panel is wrong.
In the cloud firewall, allow TCP 80 and 443 from the fifteen IPv4 and seven IPv6 Cloudflare ranges, nothing else inbound, and leave outbound open for Tailscale, the registry, ACME and mail. If your provider caps rules and you allow 80/443 from anywhere, the cloud layer is no longer a control.
A Cloudflare Tunnel is the stronger variant: no inbound port at all, at the cost of another daemon and token to keep alive.
#!/usr/bin/env bash
# /usr/local/sbin/cf-only-web.sh
# Restrict inbound HTTP/HTTPS on published Docker ports to Cloudflare edge + tailnet.
# Idempotent: rebuilds the DOCKER-USER chains from scratch on every run.
set -euo pipefail
CF4=(
173.245.48.0/20 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22
141.101.64.0/18 108.162.192.0/18 190.93.240.0/20 188.114.96.0/20
197.234.240.0/22 198.41.128.0/17 162.158.0.0/15 104.16.0.0/13
104.24.0.0/14 172.64.0.0/13 131.0.72.0/22
)
CF6=(
2400:cb00::/32 2606:4700::/32 2803:f800::/32 2405:b500::/32
2405:8100::/32 2a06:98c0::/29 2c0f:f248::/32
)
# 172.18.0.1 is the gateway of our traefik network. Yours may differ:
# docker network inspect traefik -f '{{(index .IPAM.Config 0).Gateway}}'
# ---------- IPv4 ----------
iptables -N DOCKER-USER 2>/dev/null || true
iptables -F DOCKER-USER
iptables -A DOCKER-USER -m conntrack --ctstate ESTABLISHED,RELATED -j RETURN
iptables -A DOCKER-USER -s 100.64.0.0/10 -j RETURN # tailnet
iptables -A DOCKER-USER -s 172.16.0.0/12 -j RETURN # docker networks
iptables -A DOCKER-USER -s 127.0.0.1/32 -j RETURN
for r in "${CF4[@]}"; do
iptables -A DOCKER-USER -s "$r" -p tcp -m multiport --dports 80,443 -j RETURN
done
iptables -A DOCKER-USER -p tcp -m multiport --dports 80,443 -j DROP
# ---------- IPv6 ----------
ip6tables -N DOCKER-USER 2>/dev/null || true
ip6tables -F DOCKER-USER
ip6tables -A DOCKER-USER -m conntrack --ctstate ESTABLISHED,RELATED -j RETURN
ip6tables -A DOCKER-USER -s fd7a:115c:a1e0::/48 -j RETURN # tailnet ULA
ip6tables -A DOCKER-USER -s ::1/128 -j RETURN
for r in "${CF6[@]}"; do
ip6tables -A DOCKER-USER -s "$r" -p tcp -m multiport --dports 80,443 -j RETURN
done
ip6tables -A DOCKER-USER -p tcp -m multiport --dports 80,443 -j DROP
echo "cf-only-web: applied ($(date -u +%FT%TZ))"
# /etc/systemd/system/cf-only-web.service
[Unit]
Description=Restrict published HTTP/HTTPS ports to Cloudflare edge and tailnet
Requires=docker.service
After=docker.service
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/cf-only-web.sh
ExecReload=/usr/local/sbin/cf-only-web.sh
[Install]
WantedBy=multi-user.target
# /etc/systemd/system/docker.service.d/zz-cf-only-web.conf
[Service]
# Re-apply the Cloudflare-only rules after every docker start.
ExecStartPost=/usr/local/sbin/cf-only-web.sh
sudo install -m 0755 cf-only-web.sh /usr/local/sbin/cf-only-web.sh
sudo systemctl daemon-reload
sudo systemctl enable --now cf-only-web.service
sudo systemctl restart docker
sudo iptables -S DOCKER-USER | wc -l
The chain is consulted per packet in FORWARD, so ordering against container start does not matter: re-applying after the daemon is up is enough. It is rebuilt from scratch on every run.
# From a machine outside the tailnet:
curl -s -o /dev/null -w 'cloudflare:%{http_code}\n' https://app.example.com/
# cloudflare:200
curl -s -m 5 -o /dev/null -w 'direct:%{http_code}\n' "http://${VPS_IP}/" ; echo "curl exit=$?"
# direct:000, curl exit=28 (timeout) is the pass condition
# From a machine on the tailnet:
curl -s -o /dev/null -w 'tailnet:%{http_code}\n' --resolve "app.example.com:443:${TAILNET_IP}" https://app.example.com/
# tailnet:200
Run the direct test from another machine: local traffic to the published port traverses OUTPUT, not DOCKER-USER, so it connects while the internet cannot, and the false negative looks like a broken firewall.
The consequences are permanent. HTTP-01 is impossible, so DNS-01 is the only challenge type. Health checks from outside Cloudflare move to the tailnet. GitHub-hosted runners cannot reach the origin, which is why deploys run on a runner on the machine. The Host header stops being an access control, because no unapproved peer can send one.
Failure modes
- 526 on every request -> origin certificate self-signed or wrong for the hostname -> issue via DNS-01, verify on loopback, retest.
- 521 or 522 -> Cloudflare cannot reach the origin -> Traefik down, a range missing from
DOCKER-USER, or an unsaved rule. acme.jsonempty, credentials missing -> token supplied as a file, not an env var -> setCLOUDFLARE_DNS_API_TOKEN.403 10000 Authentication erroron TXT creation -> token hasZone:Readonly -> recreate withZone:DNS:Edittoo.- Direct IP answers 200 ->
DOCKER-USERflushed, drop-in missing -> install both, restart docker, retest from outside. - Assets
DYNAMICor repeatedMISS-> cache rule mismatch, or origin sendsno-store/Set-Cookie-> fix, verify with GET. - Admin reachable without Access -> no Access app for that hostname -> add app and policy, verify with a bare
curl.
Checklist
- Proxied A and AAAA per hostname, no wildcards,
_acme-challengefree of CNAMEs. - Full (strict), origin issuer verified from loopback,
acme.jsonmode 0600 with certificates. - DNS-01 configured, token scoped to
Zone:ReadplusZone:DNS:Editin a 0600 env file. - Cache rules cover hashed assets only; HTML is
DYNAMIC, verified with GET. - WAF rules on, rate limits on login and API, skip rules documented.
- Access application and policy on every admin surface, service tokens for CI.
- Cloud firewall allows 80/443 from Cloudflare ranges only.
-
cf-only-web.serviceenabled, Docker drop-in installed, both chains populated. - Cloudflare 200, direct IP timeout, tailnet 200, each tested from the right network.
- Certificate expiry alert outside the host.
Proof you are done
- - `dig +short app.example.com` returns Cloudflare anycast, not `${VPS_IP}`.
- - `openssl s_client -connect 127.0.0.1:443 -servername app.example.com` shows a public CA or Origin CA issuer, never `TRAEFIK DEFAULT CERT`.
- - Full (strict), and no HTTP-01 resolver exists in the config.
- - `jq '.cloudflare.Certificates | length'` on a copy of `acme.json` is greater than zero.
- - `curl -m 5 http://${VPS_IP}/` times out; `--resolve` against `${TAILNET_IP}` returns 200.
- - `sudo iptables -S DOCKER-USER | grep -c DROP` is 1; `RETURN` count is at least 18.
- - `systemctl is-enabled cf-only-web.service` prints `enabled`; `systemctl show docker` lists the drop-in.
- - `/build/*` returns `cf-cache-status: HIT` on the second GET; `/` returns `DYNAMIC`.
- - `curl https://traefik.example.com/` returns 302 to Access, or 401 with valid service tokens.