djuntgen@juntgen.com
← all posts

Building a Homelab with AI · part 5

Moving Caddy to the DMZ: Isolating Your Reverse Proxy


This is Part 5 of the Building a Homelab with AI series. Previously: GitOps with Portainer | Next: Security Review

Why This Is the Most Important Change

The reverse proxy is the single most security-critical service in a homelab. It’s the only thing facing the public internet. Every HTTPS request to *.example.com flows through it. If an attacker compromises the reverse proxy, the blast radius matters a lot — and when that proxy sits on the same LAN as your Docker host, database server, and home automation, the blast radius is everything.

Previously, Caddy ran as a Docker container on docker-host (10.0.0.12), sharing the LAN with every other service. That had to change.

The Architecture

The new setup:

Internet
  |
Cloudflare (DNS proxy, DDoS protection)
  |
Edge firewall (port forward 80/443)
  |
DMZ subnet (10.0.10.0/28)
  |
caddy-dmz VM (10.0.10.10) -- Ubuntu 24.04, Caddy as systemd service
  |
  +---> LAN backends (10.0.0.x)
  +---> VLAN 20 backends (10.0.20.x)

A few deliberate decisions here:

VM, not LXC. LXC containers share the host kernel. A kernel exploit in an LXC container could potentially escape to the Proxmox host. A VM has its own kernel, adding a hypervisor isolation boundary. For the one service that faces the internet, that extra isolation is worth the overhead.

Ubuntu, not Alpine. Alpine is smaller, sure. But I run Ubuntu everywhere else. Operational consistency matters when you’re the only SRE. I don’t want to context-switch to a different package manager and init system at 2 AM when something breaks.

Systemd service, not Docker. Caddy is a single Go binary. Wrapping it in Docker adds a container runtime, an overlay filesystem, and additional attack surface — all for zero benefit. The Ansible role downloads a custom Caddy build (with the Cloudflare DNS plugin), drops it in /usr/bin/caddy, and manages it with a systemd unit file.

The Ansible Caddy Role

The roles/caddy/ role handles everything:

  • Downloads the custom caddy-cloudflare binary from GitHub
  • Creates a dedicated caddy system user (no shell, no home directory)
  • Deploys the systemd service unit
  • Templates the Caddyfile from templates/Caddyfile.j2
  • Creates <caddy-env-file> with the Cloudflare API token (from Ansible Vault)
  • Sets permissions: root-owned binary, caddy-owned config, 600 on the env file

Ansible Vault for Secrets

The Cloudflare API token can’t live in plaintext in the repo. Ansible Vault handles this:

# Create a random vault password
openssl rand -base64 32 > <vault-password-file>
chmod 600 <vault-password-file>

# Configure ansible.cfg to use it automatically
# vault_password_file = <vault-password-file>

# Encrypt the token
ansible-vault encrypt_string 'cf-token-here' --name 'vault_cloudflare_api_token'

There’s a subtlety with group_vars here. You can’t mix encrypted and plaintext variables in the same YAML file without encrypting the whole file. The solution: turn group_vars/all.yml into a directory group_vars/all/ with two files — vars.yml for plaintext and vault.yml for encrypted values. Ansible loads both automatically.

The Caddyfile

The Caddyfile uses a (local_only) snippet to restrict internal services to RFC1918 addresses:

(local_only) {
    @blocked not remote_ip 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
    respond @blocked "Access denied" 403
}

portainer.example.com {
    import local_only
    reverse_proxy https://10.0.0.12:9443 {
        transport http {
            tls_insecure_skip_verify
        }
    }
}

Public services like example.com and app.example.com omit the local_only import. The tls_insecure_skip_verify is necessary because backends like Portainer and Proxmox use self-signed certificates — Caddy needs to trust them for the reverse proxy to work. This is acceptable risk for internal traffic.

The Cutover

The migration was a multi-step process:

  1. Deploy the new Caddy VM on the DMZ with Ansible
  2. Test with direct requests to 10.0.10.10
  3. Update Cloudflare DNS: *.example.com now resolves to the DMZ
  4. Update the edge firewall port forwarding: 80/443 to 10.0.10.10
  5. Verify all services work through the new proxy
  6. Delete the old Docker Caddy stack via Portainer API
  7. Remove stacks/caddy/ from the repo
  8. Stop, back up, and destroy the old Nginx Proxy Manager LXC

I kept the old NPM container stopped (not destroyed) for a week as a safety net before finally destroying it. Always have a rollback plan.

The Result

The reverse proxy now runs on an isolated VM in a dedicated DMZ subnet. Even if an attacker exploits a vulnerability in Caddy, they land on a minimal VM with no Docker, no other services, and firewall rules restricting lateral movement. That’s a dramatically smaller blast radius than a container on the LAN.

This was the single biggest security improvement of the entire IaC project. And it was deployed entirely through Ansible — reproducible, documented, and version-controlled.

Next: reviewing the security posture of the whole setup.