Skip to content

Runbook 06 — Firewall rules (per-zone allow-lists, default deny)

Goal: implement the inter-zone policy matrix. Every zone defaults to deny; only the rules below are allowed. Rules are evaluated top-down per interface; first match wins.

Prereqs: runbooks 03–05 validated (interfaces, aliases, gateways exist).

Companion layer: this runbook is the OPNsense / inter-zone (north-south) firewall. Each fleet host also runs its own host firewall (firewalld, default-deny inbound) for east-west containment of same-VLAN traffic OPNsense never sees — see runbook 23. The two are complementary.

Conventions

26.1 note (source-checked against opnsense/core stable/26.1): two rule UIs coexist — legacy Firewall → Rules → (per-interface tabs, as written below; pages firewall_rules.php/firewall_rules_edit.php) and Firewall → Rules [new] (one unified MVC list at /ui/firewall/filter/; pick the zone via the interface filter). Same ruleset either way — use whichever. The Gateway field for policy routing, including gateway groups (PROTONVPN_LB), is present in both (confirmed intact) — this whole design depends on it. - Label deltas if you use Rules [new] (MVC): the rule's on/off is "Enabled" (legacy uses "Disabled" — opposite sense), and the IP-version field is "Version" (legacy "TCP/IP Version"). Source/Destination port are "Source Port"/ "Destination Port" (legacy "… port range"). - Per-interface rules are top-down, first-match ("quick"); the floating section evaluates before interface rules. - Destination NAT rules no longer auto-create filter rules — on the DNAT rule set "Firewall rule" = Pass (that's the field's label in 26.1), or leave it Manual and add the matching rule here.

  • Add rules under Firewall → Rules → .
  • "fw" = "This Firewall (self)" destination.
  • Each zone starts with an anti-lockout / housekeeping pattern, then allow rules, then an implicit default-deny (no explicit block needed unless noted).
  • Use aliases from runbook 03 (RFC1918, SERVERS_NET, IPA_PORTS, etc.). Before relying on the identity rules below, re-open IPA_PORTS and confirm it contains 636 (LDAPS), not 635 — a typo there silently blocks secure LDAP to FreeIPA. Also confirm every zone interface is …/24 (a /32 left from runbook 03 makes the zone's rules match traffic that can never arrive).
  • Policy: all egress via Proton. The per-zone VPN_BYPASS → WAN_GW rule is present as a safety valve but VPN_BYPASS is empty by default (runbook 05 §6) — so it matches nothing and everything goes to PROTONVPN_LB. It only diverts traffic to the real WAN once you add a destination (when Proton's exit gets blocked somewhere).

A good first rule on every client/server zone (block inter-zone leakage before allowing internet): - Block source=<zone> net dest=RFC1918 (place AFTER the specific inter-zone allows you want, BEFORE the internet allow).

Port aliases (the GUI can't take a comma-list in one port field). The "to fw" rules below reference three port aliases — create them in Firewall → Aliases (type Port(s)), or they're already in the pre-built config: - FW_PORTS_MGMT = 22, 53, 123, 443 (MGMT only) - FW_PORTS_STD = 53, 123, 443 (SERVERS, WORKSTATIONS) - FW_PORTS_DNSNTP = 53, 123 (DMZ, IOT, GUEST) Plus host aliases: JELLYFIN (10.20.20.30) for the IOT→Jellyfin rule, HOMEASSISTANT (10.20.20.45) for the HA control/UI rules, TVHEADEND (10.20.20.31) for the WORKSTATIONS Live-TV rule (runbook 21), INVIDIOUS (10.20.30.70) + KODI_BOX (10.20.40.101) for the WORKSTATIONS→DMZ Invidious-direct rule (WS 2c), and OBS (10.20.20.50) for the workstation→obs Alloy-push rule (WS 2b). The obs push uses two ports (8428 metrics, 3100 logs) → either a OBS_PUSH port alias (8428, 3100) in one rule, or two rules (the live config uses two).

Read each zone's intent below, then build it in Firewall → Rules → (or confirm it there if you loaded the pre-built config).

MGMT (10.20.10.0/24)

  1. Pass: TCP/UDP, src MGMT net, dst fw, ports 443,22,53,123 (GUI/SSH/DNS/NTP).
  2. Pass: src MGMT net, dst RFC1918, gateway default (administer all zones).
  3. Pass: src MGMT net, dst VPN_BYPASS, gateway = WAN_GW.
  4. Pass: src MGMT net, dst any, gateway = PROTONVPN_LB (internet via Proton).

MGMT is entered only from the WG admin peer (runbook 07) — there are no rules on other zones permitting traffic into MGMT.

SERVERS (10.20.20.0/24)

  1. Pass: src SERVERS net, dst fw, ports 53,123 (DNS/NTP only — not 443). SERVERS is a data/identity tier, not an admin zone, so it must not reach the OPNsense web GUI. OPNsense's implicit "allow this-firewall" self-rule would otherwise let SERVERS → (self):443 through, so a firewall-automation block (sequence 90) denies SERVERS → (self):443 on every firewall IP ahead of that self-rule — codified in provision/opnsense/fw-automation-rules.json and asserted by the fw-conformance harness (see Firewall automation rules below). 1a. Pass: src HOMEASSISTANT (10.20.20.45), dst IOT net, gateway default — Home Assistant controls the IP IoT devices (the Zigbee/Z-Wave devices ride the USB coordinator, not the network). Scope to the HA host, not all of SERVERS. (Place BEFORE the RFC1918 block below.) See runbook 21; mDNS discovery of IP IoT is handled by extending the reflector (runbook 08) to SERVERS⇄IOT.
  2. Block: src SERVERS net, dst RFC1918 — servers answer connections; they don't initiate into other zones. (Intra-SERVERS traffic is switched, not filtered here, so this only stops server→other-zone. The HA→IOT allow in 1a sits above this block.)
  3. Pass: src SERVERS net, dst VPN_BYPASS, gateway = WAN_GW.
  4. Pass: src SERVERS net, dst any, gateway = PROTONVPN_LB — server egress is via Proton too. (Inbound to the reverse proxy is unaffected: those rules are on WAN and use reply-to. ACME DNS-01/API calls ride Proton fine; HTTP-01 inbound rides WAN.)
  5. Block: src SERVERS net, dst any (kill-switch).

DMZ (10.20.30.0/24) — edge-facing/internal (origin Caddy + hidden Bind primary)

  1. Pass: src DMZ net, dst fw, ports 53,123 only.
  2. Pass: src DMZ net, dst SERVERS_NET, only the specific backend port(s) the origin Caddy needs (e.g. Authelia forward-auth, app backends, Nextcloud→DB 5432/3306, LDAP to IPA). Keep this tight — list exact hosts/ports. Add (runbook 21): origin Caddy → JELLYFIN:8096 and origin Caddy → HOMEASSISTANT:8123 (it reverse-proxies jellyfin.${domain} and ha.${domain}, both now in SERVERS). (Invidious is intra-DMZ — no rule.) 2a. Pass: src JUMP (10.20.30.5 — the SSH jump host only), dst 10.20.0.0/16, port 22, gateway default. This is the single DMZ→fleet:22 exception that powers the no-VPN admin relay (edge :443 → jump → any fleet host, runbook 20); it reaches every zone incl. MGMT (so ssh to the GPU host works this way). Must sit ABOVE the RFC1918 block below (first-match), and is scoped to the jump's single IP — every other DMZ host stays blocked from the fleet on :22.
  3. Block: src DMZ net, dst RFC1918 (DMZ must not reach MGMT/WS/IOT — except the JUMP→:22 exception in 2a above, which is matched first).
  4. Pass: src DMZ net, dst VPN_BYPASS, gateway = WAN_GW.
  5. Pass: src DMZ net, dst any, gateway = PROTONVPN_LB — outbound updates (ACME, package pulls) via Proton.
  6. Block: src DMZ net, dst any (kill-switch).

No WAN port-forwards into DMZ. Under edge/origin (runbook 17) the public reaches the origin Caddy + answers off the hidden Bind primary only via the edge WireGuard tunnels — which terminate on the WG-edge interface, not WAN. The pass rules permitting edge tunnel → origin Caddy :443 and edge tunnel → hidden primary :53 (AXFR/IXFR, TSIG) live on that WG-edge interface (runbook 17), source-restricted to the per-VPS tunnel /32s.

WORKSTATIONS (10.20.40.0/24)

  1. Pass: TCP/UDP, src WS net, dst fw, ports FW_PORTS_STD (53/123/443).
  2. Pass: src WS net, dst SERVERS_NET, ports IPA_PORTS (identity + NFSv4 2049) — gateway default (local, untunneled — NFS stays LAN-speed). 2a. Pass: src WS net, dst HOMEASSISTANT:8123, JELLYFIN:8096, and TVHEADEND:9981,9982, gateway default — HA dashboard + Jellyfin + tvheadend Live-TV/HTSP (runbook 21). (The LibreELEC/Kodi box does NOT live here — it stayed on IOT by the 2026-08-02 decision: it runs untrusted third-party plugins, and WORKSTATIONS membership grants effective read of every sec=sys NFS home. See §IOT.) (HA/Jellyfin are also reachable via ha/jellyfin.${domain} through the origin Caddy; this is the fast local path.) 2b. Pass: TCP, src WS net, dst OBS (10.20.20.50):8428,3100, gateway default — Grafana Alloy push (node metrics → VictoriaMetrics :8428, journald → Loki :3100) so workstation agents report to the obs sink like the EL fleet (esp. the GPU host, the AI host; a workstation). Without this the WS zone can't reach SERVERS beyond IPA_PORTS. (MGMT workstations like the admin laptop already reach obs via the MGMT→all admin path.) 2c. RETIRED (2026-08-08). The former KODI_BOX → INVIDIOUS:3000 WS→DMZ exception assumed the Kodi box lived in WORKSTATIONS; the 2026-08-02 decision kept it on IOT (untrusted plugins). The same host-scoped rule now lives in §IOT (rule 2b, automation seq 541) — direct to the backend, never via apps-proxy. The KODI_BOX alias points at the IOT address (10.20.60.100). The old GUI rule "Workstations Kodi to Invidious" (opt4) is inert with the repointed alias — delete it in the GUI when convenient.
  3. Pass: src WS net, dst IOT net (cast/print; tighten to specific ports in runbook 08) — gateway default.
  4. Block: src WS net, dst RFC1918 — isolates every other internal zone (MGMT, DMZ, GUEST). Must come AFTER the SERVERS/IOT allows (2–3), BEFORE the internet rule.
  5. Block: TCP/UDP 853 (DoT) any (anti-DNS-bypass).
  6. Pass: src WS net, dst VPN_BYPASS, gateway = WAN_GW (safety-valve hook; empty by default → matches nothing → all egress stays on Proton).
  7. Pass: src WS net, dst any, gateway = PROTONVPN_LB (internet via the Proton load-balance group).
  8. Block: src WS net, dst any (kill-switch / fail-closed).

Why block RFC1918, not pass it. Allowing the specific local destinations (SERVERS in 2, IOT in 3) on the default gateway keeps that traffic local/untunneled; the RFC1918 block (4) then isolates all other internal zones. (Passing RFC1918 as a "local bypass" before the mgmt/dmz blocks would silently open WS→MGMT/DMZ, since those sit inside 10.x — so block after the allows; that order is fail-safe.) Only real internet (non-RFC1918) reaches the Proton rule (7); the final block (8) fails closed if the whole group drops.

IOT (10.20.60.0/24)

  1. Pass: src IOT net, dst fw, ports 53,123 only (DNS via intercept + NTP).

    ⚠️ The DNS-intercept port-forward MUST carry destination port 53 (NAT rdr: src IOT net, dst !IOT address port 53, tcp/udp → IOT address:53). A portless rdr matches every IOT tcp/udp flow and rewrites it to Unbound:53 — this silently killed ALL IOT internet/NTP/media for ~2 months (2026-06-11 → 2026-08-08; the tell: pf states all show dst <gw>:53 with nat_addr = the real destination, and "NTP replies" are 12-byte DNS FORMERRs arriving in microseconds). Now codified: the rdr lives in provision/opnsense/fw-automation-rules.json (nat section; opnsense-fw.py apply recreates it on rebuild) and its port-53 shape is asserted by the fw-conformance natshape check.

  2. Pass: src IOT net, dst SERVERS_NET host Jellyfin, Jellyfin port 8096 (Roku→Jellyfin) — pin host alias in runbook 11/09. 2a. Pass: TCP, src KODI_BOX (10.20.60.100, host-scoped), dst TVHEADEND (10.20.20.31):9981-9982 — the LibreELEC/Kodi box's pvr.hts Live-TV/HTSP path (operator-approved 2026-08-08). Lives in the automation layer (provision/opnsense/fw-automation-rules.json, seq 540), evaluated before the GUI zone rules. Host-scoped via the KODI_BOX alias (re-IP = one edit; the IP is pinned by a Kea reservation on the box's MAC). The box stays on IOT (untrusted plugins — see the 2026-08-02 decision). 2b. Pass: TCP, src KODI_BOX, dst 10.20.20.7 (the invidious bound front door):443 — automation seq 543, operator-approved 2026-08-09. The one Tier-2 name granted to the box, made expressible as an L3 rule by giving the name its own IP: apps-proxy holds a second address — a /32 alias on eth0 (proxy-front-addr.service, role-managed — a RemainAfterExit oneshot, so the role restarts it whenever its address list changes: started on an already-active oneshot is a no-op, which left the .8 alias unapplied and Caddy crash-looping on bind: cannot assign requested address 2026-09-05; a same-subnet second NIC was tried and reverted live 2026-08-09: its kernel route flipped Caddy's outbound source to .7 and broke HA's trusted_proxies) — with an exact-host Caddy site serving only invidious.${domain} bound to it, while the shared wildcard plane is pinned to .6 (proxy_bind) — so .7 serves nothing else (asserted by the fw-conformance http checks: invidious@.7 = 200, any other name @.7 = 421 from the explicit catch-all). Split-horizon resolves invidious.${domain} → .7 for everyone (workstations/WG-laptops already reach any SERVERS IP:443 via rule 2's IPA_PORTS; phones get companion automation rule seq 241). A shared-.6:443 grant stays declined — that would expose the whole Tier-2 plane (HA/auth/git/cloud/…).

    RETIRED seq 541 (backend-direct KODI_BOX → INVIDIOUS:3000, 2026-08-08 → 2026-08-09): listings worked but playback cannot — invidious runs local: true with domain: baked into every stream/DASH URL, so playback always bounces to https://invidious.${domain}. The front door serves the real name; the add-on's instance URL is simply https://invidious.${domain}. The box's media surface: Jellyfin + tvheadend + the invidious front door; still no NFS, no shared apps-proxy.

  3. Block: src IOT net, dst RFC1918 (no other internal access).
  4. Pass: src IOT net, dst VPN_BYPASS, gateway = WAN_GW — safety-valve hook (empty by default). If the Roku's YouTube ever gets blocked on the Proton exit, add Google/YouTube ranges to VPN_BYPASS and it'll exit the real WAN.
  5. Pass: src IOT net, dst any, gateway = PROTONVPN_LB — all other IOT internet via Proton; tighten to specific devices/aliases for chatty gadgets.
  6. Block: src IOT net, dst any (kill-switch).

mDNS reflection (runbook 08) handles discovery; the unicast allow rules live on WS (toward IOT) and here (IOT→Jellyfin).

GUEST (10.20.90.0/24)

  1. Pass: src GUEST net, dst fw, ports 53,123 only.
  2. Block: src GUEST net, dst RFC1918 (total internal isolation).
  3. Block DoT 853.
  4. Pass: src GUEST net, dst VPN_BYPASS, gateway = WAN_GW.
  5. Pass: src GUEST net, dst any, gateway = PROTONVPN_LB. (Guests share your Proton egress/quota — if you'd rather keep guests off your VPN, point this at WAN_GW instead.)
  6. Block: src GUEST net, dst any (kill-switch).

(Also enable client isolation on the guest SSID at the AP.)

WAN (inbound) — deny all except the WG admin endpoint

Home is inbound-silent on the Comcast link: the only thing that may arrive from the internet is the road-warrior WireGuard dial-in. There are no service port-forwards (the public-facing layer is on the VPS edges, runbook 17, reached over home-dialed outbound tunnels).

  1. Pass: proto UDP, src any, dst .69 (WAN WG-admin VIP), dst port 51820 — the WireGuard admin endpoint (runbook 07). reply-to WAN_DHCP is automatic.
  2. (implicit) default-deny everything else inbound on WAN. OPNsense blocks un-matched inbound by default; the bogon/private block (Interfaces → WAN, and Firewall → Settings → Advanced) stays on.

Do not add 80/443/53/22 forwards here. If you later need to verify the posture: an external scan of the Comcast IP block should show only UDP 51820 on .69 open — nothing else (runbook 14).

The edge tunnels do not appear here: home dials out to each VPS, so those handshakes leave as outbound UDP and their return traffic is stateful — no WAN inbound rule is required for them.

Break-glass: direct WAN egress (DISABLED by default)

A disabled override for when all of Proton is down — flip it on to restore internet via Comcast, flip it off when Proton's back. Manual on purpose: auto-failing-over to WAN would silently leak your traffic to the ISP (the thing you're avoiding).

Firewall → Rules → Floating → +: - Disabled: yes (this is the key — it sits inert until you need it) - Action Pass, quick on, Interface(s) = the internal zones (or all), Direction in, Source = RFC1918 (or ZONES_ALL), Destination any, Gateway = WAN_GW/WAN_DHCP - Description: BREAK-GLASS: direct WAN egress — enable only when Proton is down

Because it's a floating quick rule it evaluates before the per-zone rules, so enabling it overrides the Proton routing + kill-switch for everyone at once. (Scope the source to just MGMT net if you only want recovery access, not whole-network internet, during an outage.)

Floating / housekeeping

  • Block private networks / Block bogon networks: these are per-interface toggles on Interfaces → WAN (not on Firewall → Settings → Advanced) — keep both enabled on WAN.
  • Anti-lockout: management is via MGMT/WG only (the WG admin peer enters MGMT, runbook 07); the Proxmox console of the OPNsense VM is the break-glass fallback.

Firewall automation rules (as code)

A handful of cross-zone service grants (and one deny) don't fit the per-zone GUI ruleset above cleanly — they're host-scoped passes added over the API into OPNsense's MVC Automation → Filter list. Those live as OPNsense runtime state and can be lost on a rebuild, so they are codified in the repo and reconciled by a script:

  • provision/opnsense/fw-automation-rules.json — the declarative source of truth. Each rule is matched by its exact description. Interface ids: opt1=MGMT opt2=SERVERS opt3=DMZ opt4=WORKSTATIONS opt6=IOT opt7=GUEST opt12=WGRW. It carries, e.g., the sequence-90 SERVERS → (self):443 and sequence-91 WORKSTATIONS → (self):443 blocks (referenced by the SERVERS rule 1 above), the wg-admin → UniFi UI and registry → hidden-primary :53 grants, the origin → radio bound-listener :443 relay (seq 645 — the Tier-1 radio path; DMZ→WORKSTATIONS stays closed, so the origin reaches adac only via the apps-proxy's radio-only .8 address), the two DMZ → fileserver :8081 shelf-map grants (seq 545 tenant-proxy, the tenant file path — runbook 55 §11; seq 546 origin Caddy, the Tier-1 sync.${domain} face added 2026-09-13 — runbook 52 §The Tier-1 face), and the runner-pub → fileserver :22 shelf deploy grant (seq 547, 2026-09-13 — runbook 52 §Push-to-deploy: the public CI rings a doorbell, and the account it reaches is bound by a certificate force-command + an sshd ForceCommand to one program that verifies the release's signature itself, so the grant carries no shell). The :8081 pair are single-host sources rather than the DMZ /24 for the same reason, and it is the reason worth remembering: shelf-map trusts Remote-User absolutely (it reads that header to pick which tenant's socket to serve), so anything that can reach the port can name itself any tenant — only a proxy that authenticates first may be a source. The fleet-side half of the same grant is the fileserver's own host_firewall_source_ports; neither layer is decorative. And it carries the per-proxy_vhosts apps-proxy → backend passes that scripts/apps-frontdoor-sync.py emits (SERVERS→DMZ, and SERVERS→WORKSTATIONS since the radio.${domain} → adac:8073 OpenWebRX front-door, 2026-09-02 — the script keys the zone pair off the backend's subnet prefix; fw-conformance asserts the allow stays apps-proxy-scoped).
  • scripts/opnsense-fw.py apply [--dry-run] — reconcile the live firewall to the JSON: it only adds missing rules (idempotent, matched by description) and never deletes/edits live rules, so an emergency hand-added rule is never clobbered; reconcile removals by hand. scripts/opnsense-fw.py list shows the live automation rules read-only.
  • tests/fw-conformance/conformance.py — the assertion harness: it probes the expected pass/block outcomes (including SERVERS → 10.20.10.1:443 = block) so drift from the design is caught.

Automation-filter passes evaluate ahead of the classic per-interface rules, so a scoped pass here (e.g. WG_LAPTOPS → SERVERS_NET:22) correctly precedes a classic block → RFC1918 on the same interface — that's why these grants work.

After an OPNsense rebuild: re-add the aliases + GUI ruleset per runbooks 02–08, then scripts/opnsense-fw.py apply to restore these automation rules, then run the conformance harness. Editing policy = edit the JSON, then apply. See provision/opnsense/README.md.

Validation

  • [ ] From a WS test client: internet works and shows Proton egress IP; can reach SERVERS on IPA/NFS ports; cannot reach MGMT or DMZ-internal; cannot reach other client zones.
  • [ ] From GUEST test client: internet only; no RFC1918 reachable.
  • [ ] From IOT test client: only DNS/NTP to fw + Jellyfin; RFC1918 otherwise blocked; DNS intercept still forces Unbound.
  • [ ] Kill-switch verified again (disable Proton → WS internet stops).
  • [ ] Firewall → Log Files → Live View shows expected passes/blocks, no surprise denies for legit flows.

Then → runbook 07 (road-warrior WireGuard + WG-only MGMT).