djuntgen@juntgen.com
← all posts

Building a Homelab with AI · part 17

OpenClaw Round Two: LAN Deployment, Linger Lessons, and Caddy Integration


The Goal

When we first deployed OpenClaw (post 14), the gateway was bound to 127.0.0.1:18789 and accessible only via SSH tunnel. That was the right call at the time — minimal exposure while we learned the tool. After a few sessions of tunneling in, the friction was real enough to justify doing it properly: Caddy reverse proxy, LAN-only TLS, no tunnel required.

This post covers taking a fresh cloud-init VM all the way to https://openclaw.example.com — and the four bugs we hit along the way.


Starting Point: Fresh VM, Existing Role

The openclaw VM (10.0.0.14) had been rebuilt from the Proxmox cloud-init template. The Ansible role existed from previous sessions. On paper: run two playbooks, done.

In practice, four things broke in sequence. Each one taught us something.


Bug 1: Linger

Symptom: OpenClaw works fine while SSHed in. Disconnect the session, wait a minute, refresh the UI — 502 Bad Gateway. SSH back in: service is dead.

Cause: OpenClaw’s gateway runs as a systemd user service under the <user> user. By default, a user’s systemd instance only lives while that user has an active login session. When the last SSH session closes, systemd tears down the user instance — taking every user service with it.

Fix: loginctl enable-linger <user>

This creates a file at /var/lib/systemd/linger/<user> that instructs systemd to keep the user instance alive permanently, regardless of active sessions. The service survives reboots without requiring a login.

This was missing from the Ansible role. We added it to tasks/service.yml:

- name: Enable systemd linger for <user> (service persists without active session)
  ansible.builtin.command: loginctl enable-linger <user>
  args:
    creates: /var/lib/systemd/linger/<user>
  become: true

The creates argument makes this idempotent — Ansible skips the command if the file already exists. The task requires become: true because only root can enable linger for other users.


Bug 2: openclaw daemon status Triggers Restarts

Symptom: The Ansible playbook ran to completion. Then the gateway restarted itself 30 seconds later, clearing the auth session.

Cause: The original service.yml included a task that ran openclaw daemon status as a final verification step. It turns out this is a known upstream bug in OpenClaw — calling daemon status triggers an unintended restart of the gateway service. Our “check if it worked” step was breaking the thing it was checking.

Fix: Remove the daemon status task from the role. Use the underlying systemd command to check service state instead:

systemctl --user status openclaw-gateway.service

This never triggers a restart. The openclaw daemon status command should be treated as broken until upstream fixes it.


Bug 3: Gateway Not Reachable from Caddy

After adding openclaw.example.com to the Caddyfile and deploying it, Caddy returned 503. The gateway was running but not answering requests from 10.0.10.10.

Cause: By default, OpenClaw sets gateway.bind to loopback — it only listens on 127.0.0.1. Caddy (on 10.0.10.10) can’t reach 127.0.0.1 on the VM. It needs the gateway to listen on a real interface.

Fix: Two changes together:

  1. Change the bind mode:

    openclaw config set gateway.bind lan

    This sets the gateway to listen on all LAN interfaces (0.0.0.0:18789).

  2. Open the firewall for Caddy only:

    - name: Allow OpenClaw gateway from Caddy DMZ (10.0.10.10)
      community.general.ufw:
        rule: allow
        port: '18789'
        proto: tcp
        src: 10.0.10.10
        comment: OpenClaw gateway - Caddy reverse proxy only

The UFW rule is critical. Binding to lan exposes port 18789 to the entire LAN subnet — without the firewall rule, anything on 10.0.0.0/24 could hit the gateway directly (without going through Caddy). The UFW rule locks it down to 10.0.10.10 only. The Caddyfile (local_only) snippet handles the LAN-vs-public distinction at the Caddy layer.

We added gateway.bind lan to tasks/configure.yml:

- name: Set gateway.bind to lan (LAN accessible via Caddy)
  ansible.builtin.command: openclaw config set gateway.bind lan
  become: false
  become_user: <user>
  changed_when: true
  notify: restart openclaw

Bug 4: “Origin Not Allowed” in the Browser

Caddy was returning 200. The page loaded. Then the browser console showed:

WebSocket connection to 'wss://openclaw.example.com/...' failed:
Error: HTTP Authentication failed; no valid credentials available

And the UI showed “Origin not allowed” or “Pairing required.”

Cause: OpenClaw validates the Origin header on WebSocket connections. When the browser connects via https://openclaw.example.com, the origin header is https://openclaw.example.com. The gateway’s default allowedOrigins list only contained the localhost origin — it rejected everything coming through Caddy.

Fix:

openclaw config set gateway.controlUi.allowedOrigins '["https://openclaw.example.com"]'

In tasks/configure.yml:

- name: Allow Control UI from Caddy reverse proxy origin
  ansible.builtin.command: >
    openclaw config set gateway.controlUi.allowedOrigins
    '["https://openclaw.example.com"]'
  become: false
  become_user: <user>
  changed_when: true
  notify: restart openclaw

The Caddyfile Entry

The Caddyfile change was the simplest part. One block in ansible/roles/caddy/templates/Caddyfile.j2:

openclaw.example.com {
    import local_only
    reverse_proxy 10.0.0.14:18789
}

The (local_only) snippet (defined earlier in the Caddyfile) returns 403 for any request not originating from an RFC1918 address. Deployed via:

ansible-playbook playbooks/caddy.yml

Token Authentication

The OpenClaw gateway uses a token to authenticate the browser. The token is stored in vault as openclaw_gateway_token, deployed to ~/.openclaw/.env (mode 600) on the VM, and sourced as an EnvironmentFile by the systemd unit.

To access the Control UI, navigate to:

https://openclaw.example.com/#token=<token>

The #token= fragment is consumed by the browser — it is never sent to the server in the URL. To retrieve the token value:

cd ~/homelab/ansible
ansible-vault view inventory/group_vars/all/vault.yml | grep openclaw_gateway_token

End State

After all four fixes:

CheckResult
systemctl --user status openclaw-gateway.serviceactive (running)
`loginctl show-user grep Linger`
sudo ufw status verbosePort 22 from LAN, port 18789 from 10.0.10.10
https://openclaw.example.com/#token=<token>Control UI loads, auth succeeds

The SSH tunnel approach still works as a fallback. But it is no longer the primary access path.


Lessons Learned

1. Systemd linger is not optional for user services that need to survive reboots. If your service runs under a user account (not root), loginctl enable-linger <user> must be part of the deployment. Without it, the service is effectively “works while you’re watching it.” This is easy to miss because it works fine during initial setup when you’re actively SSHed in.

2. Debug tools that break what they measure are worse than no debug tools. openclaw daemon status looks useful. It causes a restart. If we had not spotted the pattern — service up, run status, service restarts — we would have chased it for much longer. When something restarts itself without a clear reason, look at every command being run immediately before the restart.

3. Binding to LAN without firewalling is incomplete. gateway.bind lan is necessary for Caddy to reach the gateway, but it also exposes port 18789 to the entire LAN. The UFW rule that restricts it to 10.0.10.10 is not a nice-to-have — it is the control that makes the broader exposure acceptable. The two changes must be made together.

4. allowedOrigins must match the exact origin the browser presents. The origin is scheme + hostname + port. https://openclaw.example.com (no port, HTTPS) is different from http://10.0.0.14:18789. When a reverse proxy changes the scheme or hostname, any origin-validation in the backend must be updated to match.


What’s Next

  • The bootstrap command no longer needs -e ansible_user=root — cloud-init creates <user> directly. That simplification is reflected in the updated docs/openclaw-setup.md.
  • OpenClaw capabilities are still minimal (research/sandbox phase). As trust is established, skills and tool access can be expanded through the vault-backed config.
  • The openclaw daemon status upstream bug should be tracked — if it gets fixed, the removed status-check task can be restored.