Skip to content

Runbook 06 — Firewall rules (per-zone allow-lists, default deny)

Goal: implement the inter-zone policy matrix. Every zone defaults to deny; only the rules below are allowed. Rules are evaluated top-down per interface; first match wins.

Prereqs: runbooks 03–05 validated (interfaces, aliases, gateways exist).

Companion layer: this runbook is the OPNsense / inter-zone (north-south) firewall. Each fleet host also runs its own host firewall (firewalld, default-deny inbound) for east-west containment of same-VLAN traffic OPNsense never sees — see runbook 23. The two are complementary.

Conventions

26.1 note (source-checked against opnsense/core stable/26.1): two rule UIs coexist — legacy Firewall → Rules → (per-interface tabs, as written below; pages firewall_rules.php/firewall_rules_edit.php) and Firewall → Rules [new] (one unified MVC list at /ui/firewall/filter/; pick the zone via the interface filter). Same ruleset either way — use whichever. The Gateway field for policy routing, including gateway groups (PROTONVPN_LB), is present in both (confirmed intact) — this whole design depends on it. - Label deltas if you use Rules [new] (MVC): the rule's on/off is "Enabled" (legacy uses "Disabled" — opposite sense), and the IP-version field is "Version" (legacy "TCP/IP Version"). Source/Destination port are "Source Port"/ "Destination Port" (legacy "… port range"). - Per-interface rules are top-down, first-match ("quick"); the floating section evaluates before interface rules. - Destination NAT rules no longer auto-create filter rules — on the DNAT rule set "Firewall rule" = Pass (that's the field's label in 26.1), or leave it Manual and add the matching rule here.

  • Add rules under Firewall → Rules → .
  • "fw" = "This Firewall (self)" destination.
  • Each zone starts with an anti-lockout / housekeeping pattern, then allow rules, then an implicit default-deny (no explicit block needed unless noted).
  • Use aliases from runbook 03 (RFC1918, SERVERS_NET, IPA_PORTS, etc.). Before relying on the identity rules below, re-open IPA_PORTS and confirm it contains 636 (LDAPS), not 635 — a typo there silently blocks secure LDAP to FreeIPA. Also confirm every zone interface is …/24 (a /32 left from runbook 03 makes the zone's rules match traffic that can never arrive).
  • Policy: all egress via Proton. The per-zone VPN_BYPASS → WAN_GW rule is present as a safety valve but VPN_BYPASS is empty by default (runbook 05 §6) — so it matches nothing and everything goes to PROTONVPN_LB. It only diverts traffic to the real WAN once you add a destination (when Proton's exit gets blocked somewhere).

A good first rule on every client/server zone (block inter-zone leakage before allowing internet): - Block source=<zone> net dest=RFC1918 (place AFTER the specific inter-zone allows you want, BEFORE the internet allow).

Port aliases (the GUI can't take a comma-list in one port field). The "to fw" rules below reference three port aliases — create them in Firewall → Aliases (type Port(s)), or they're already in the pre-built config: - FW_PORTS_MGMT = 22, 53, 123, 443 (MGMT only) - FW_PORTS_STD = 53, 123, 443 (SERVERS, WORKSTATIONS) - FW_PORTS_DNSNTP = 53, 123 (DMZ, IOT, GUEST) Plus host aliases: JELLYFIN (10.20.20.30) for the IOT→Jellyfin rule, HOMEASSISTANT (10.20.20.45) for the HA control/UI rules, TVHEADEND (10.20.20.31) for the WORKSTATIONS Live-TV rule (runbook 21), INVIDIOUS (10.20.30.70) + KODI_BOX (10.20.40.101) for the WORKSTATIONS→DMZ Invidious-direct rule (WS 2c), and OBS (10.20.20.50) for the workstation→obs Alloy-push rule (WS 2b). The obs push uses two ports (8428 metrics, 3100 logs) → either a OBS_PUSH port alias (8428, 3100) in one rule, or two rules (the live config uses two).

Read each zone's intent below, then build it in Firewall → Rules → (or confirm it there if you loaded the pre-built config).

MGMT (10.20.10.0/24)

  1. Pass: TCP/UDP, src MGMT net, dst fw, ports 443,22,53,123 (GUI/SSH/DNS/NTP).
  2. Pass: src MGMT net, dst RFC1918, gateway default (administer all zones).
  3. Pass: src MGMT net, dst VPN_BYPASS, gateway = WAN_GW.
  4. Pass: src MGMT net, dst any, gateway = PROTONVPN_LB (internet via Proton).

MGMT is entered only from the WG admin peer (runbook 07) — there are no rules on other zones permitting traffic into MGMT.

SERVERS (10.20.20.0/24)

  1. Pass: src SERVERS net, dst fw, ports 53,123 (DNS/NTP only — not 443). SERVERS is a data/identity tier, not an admin zone, so it must not reach the OPNsense web GUI. OPNsense's implicit "allow this-firewall" self-rule would otherwise let SERVERS → (self):443 through, so a firewall-automation block (sequence 90) denies SERVERS → (self):443 on every firewall IP ahead of that self-rule — codified in provision/opnsense/fw-automation-rules.json and asserted by the fw-conformance harness (see Firewall automation rules below). 1a. Pass: src HOMEASSISTANT (10.20.20.45), dst IOT net, gateway default — Home Assistant controls the IP IoT devices (the Zigbee/Z-Wave devices ride the USB coordinator, not the network). Scope to the HA host, not all of SERVERS. (Place BEFORE the RFC1918 block below.) See runbook 21; mDNS discovery of IP IoT is handled by extending the reflector (runbook 08) to SERVERS⇄IOT.
  2. Block: src SERVERS net, dst RFC1918 — servers answer connections; they don't initiate into other zones. (Intra-SERVERS traffic is switched, not filtered here, so this only stops server→other-zone. The HA→IOT allow in 1a sits above this block.)
  3. Pass: src SERVERS net, dst VPN_BYPASS, gateway = WAN_GW.
  4. Pass: src SERVERS net, dst any, gateway = PROTONVPN_LB — server egress is via Proton too. (Inbound to the reverse proxy is unaffected: those rules are on WAN and use reply-to. ACME DNS-01/API calls ride Proton fine; HTTP-01 inbound rides WAN.)
  5. Block: src SERVERS net, dst any (kill-switch).

DMZ (10.20.30.0/24) — edge-facing/internal (origin Caddy + hidden Bind primary)

  1. Pass: src DMZ net, dst fw, ports 53,123 only.
  2. Pass: src DMZ net, dst SERVERS_NET, only the specific backend port(s) the origin Caddy needs (e.g. Authelia forward-auth, app backends, Nextcloud→DB 5432/3306, LDAP to IPA). Keep this tight — list exact hosts/ports. Add (runbook 21): origin Caddy → JELLYFIN:8096 and origin Caddy → HOMEASSISTANT:8123 (it reverse-proxies jellyfin.${domain} and ha.${domain}, both now in SERVERS). (Invidious is intra-DMZ — no rule.) 2a. Pass: src JUMP (10.20.30.5 — the SSH jump host only), dst 10.20.0.0/16, port 22, gateway default. This is the single DMZ→fleet:22 exception that powers the no-VPN admin relay (edge :443 → jump → any fleet host, runbook 20); it reaches every zone incl. MGMT (so ssh to the GPU host works this way). Must sit ABOVE the RFC1918 block below (first-match), and is scoped to the jump's single IP — every other DMZ host stays blocked from the fleet on :22.
  3. Block: src DMZ net, dst RFC1918 (DMZ must not reach MGMT/WS/IOT — except the JUMP→:22 exception in 2a above, which is matched first).
  4. Pass: src DMZ net, dst VPN_BYPASS, gateway = WAN_GW.
  5. Pass: src DMZ net, dst any, gateway = PROTONVPN_LB — outbound updates (ACME, package pulls) via Proton.
  6. Block: src DMZ net, dst any (kill-switch).

No WAN port-forwards into DMZ. Under edge/origin (runbook 17) the public reaches the origin Caddy + answers off the hidden Bind primary only via the edge WireGuard tunnels — which terminate on the WG-edge interface, not WAN. The pass rules permitting edge tunnel → origin Caddy :443 and edge tunnel → hidden primary :53 (AXFR/IXFR, TSIG) live on that WG-edge interface (runbook 17), source-restricted to the per-VPS tunnel /32s.

WORKSTATIONS (10.20.40.0/24)

  1. Pass: TCP/UDP, src WS net, dst fw, ports FW_PORTS_STD (53/123/443).
  2. Pass: src WS net, dst SERVERS_NET, ports IPA_PORTS (identity + NFSv4 2049) — gateway default (local, untunneled — NFS stays LAN-speed). 2a. Pass: src WS net, dst HOMEASSISTANT:8123, JELLYFIN:8096, and TVHEADEND:9981,9982, gateway default — HA dashboard + Jellyfin + tvheadend Live-TV/HTSP (runbook 21). The LibreELEC/Kodi box lives in this zone: its Jellyfin add-on hits JELLYFIN:8096, pvr.hts hits TVHEADEND:9982 (HTSP), and it reads NFS media via IPA_PORTS 2049 (rule 2). (HA/Jellyfin are also reachable via ha/jellyfin.${domain} through the origin Caddy; this is the fast local path.) 2b. Pass: TCP, src WS net, dst OBS (10.20.20.50):8428,3100, gateway default — Grafana Alloy push (node metrics → VictoriaMetrics :8428, journald → Loki :3100) so workstation agents report to the obs sink like the EL fleet (esp. the GPU host, the AI host; a workstation). Without this the WS zone can't reach SERVERS beyond IPA_PORTS. (MGMT workstations like the admin laptop already reach obs via the MGMT→all admin path.) 2c. Pass: TCP, src KODI_BOX (10.20.40.101, host-scoped), dst INVIDIOUS (10.20.30.70):3000, gateway default. Kodi's Invidious add-on hits the backend directly at http://10.20.30.70:3000 (by IP — the invidious.${domain} name now resolves via split-horizon to apps-proxy :443, so the add-on targets the backend IP, not the FQDN; Invidious itself takes no edge/Authelia on this path). The lone WORKSTATIONS→DMZ exception — deliberately host-scoped to the Kodi box via the KODI_BOX alias (so a re-IP is one edit), not zone-wide, since DMZ is the restricted edge-facing zone. Must come BEFORE the RFC1918 block (4).
  3. Pass: src WS net, dst IOT net (cast/print; tighten to specific ports in runbook 08) — gateway default.
  4. Block: src WS net, dst RFC1918 — isolates every other internal zone (MGMT, DMZ, GUEST). Must come AFTER the SERVERS/IOT allows (2–3) and the host-scoped 2c, BEFORE the internet rule.
  5. Block: TCP/UDP 853 (DoT) any (anti-DNS-bypass).
  6. Pass: src WS net, dst VPN_BYPASS, gateway = WAN_GW (safety-valve hook; empty by default → matches nothing → all egress stays on Proton).
  7. Pass: src WS net, dst any, gateway = PROTONVPN_LB (internet via the Proton load-balance group).
  8. Block: src WS net, dst any (kill-switch / fail-closed).

Why block RFC1918, not pass it. Allowing the specific local destinations (SERVERS in 2, IOT in 3) on the default gateway keeps that traffic local/untunneled; the RFC1918 block (4) then isolates all other internal zones. (Passing RFC1918 as a "local bypass" before the mgmt/dmz blocks would silently open WS→MGMT/DMZ, since those sit inside 10.x — so block after the allows; that order is fail-safe.) Only real internet (non-RFC1918) reaches the Proton rule (7); the final block (8) fails closed if the whole group drops.

IOT (10.20.60.0/24)

  1. Pass: src IOT net, dst fw, ports 53,123 only (DNS via intercept + NTP).
  2. Pass: src IOT net, dst SERVERS_NET host Jellyfin, Jellyfin port 8096 (Roku→Jellyfin) — pin host alias in runbook 11/09.
  3. Block: src IOT net, dst RFC1918 (no other internal access).
  4. Pass: src IOT net, dst VPN_BYPASS, gateway = WAN_GW — safety-valve hook (empty by default). If the Roku's YouTube ever gets blocked on the Proton exit, add Google/YouTube ranges to VPN_BYPASS and it'll exit the real WAN.
  5. Pass: src IOT net, dst any, gateway = PROTONVPN_LB — all other IOT internet via Proton; tighten to specific devices/aliases for chatty gadgets.
  6. Block: src IOT net, dst any (kill-switch).

mDNS reflection (runbook 08) handles discovery; the unicast allow rules live on WS (toward IOT) and here (IOT→Jellyfin).

GUEST (10.20.90.0/24)

  1. Pass: src GUEST net, dst fw, ports 53,123 only.
  2. Block: src GUEST net, dst RFC1918 (total internal isolation).
  3. Block DoT 853.
  4. Pass: src GUEST net, dst VPN_BYPASS, gateway = WAN_GW.
  5. Pass: src GUEST net, dst any, gateway = PROTONVPN_LB. (Guests share your Proton egress/quota — if you'd rather keep guests off your VPN, point this at WAN_GW instead.)
  6. Block: src GUEST net, dst any (kill-switch).

(Also enable client isolation on the guest SSID at the AP.)

WAN (inbound) — deny all except the WG admin endpoint

Home is inbound-silent on the Comcast link: the only thing that may arrive from the internet is the road-warrior WireGuard dial-in. There are no service port-forwards (the public-facing layer is on the VPS edges, runbook 17, reached over home-dialed outbound tunnels).

  1. Pass: proto UDP, src any, dst .69 (WAN WG-admin VIP), dst port 51820 — the WireGuard admin endpoint (runbook 07). reply-to WAN_DHCP is automatic.
  2. (implicit) default-deny everything else inbound on WAN. OPNsense blocks un-matched inbound by default; the bogon/private block (Interfaces → WAN, and Firewall → Settings → Advanced) stays on.

Do not add 80/443/53/22 forwards here. If you later need to verify the posture: an external scan of the Comcast IP block should show only UDP 51820 on .69 open — nothing else (runbook 14).

The edge tunnels do not appear here: home dials out to each VPS, so those handshakes leave as outbound UDP and their return traffic is stateful — no WAN inbound rule is required for them.

Break-glass: direct WAN egress (DISABLED by default)

A disabled override for when all of Proton is down — flip it on to restore internet via Comcast, flip it off when Proton's back. Manual on purpose: auto-failing-over to WAN would silently leak your traffic to the ISP (the thing you're avoiding).

Firewall → Rules → Floating → +: - Disabled: yes (this is the key — it sits inert until you need it) - Action Pass, quick on, Interface(s) = the internal zones (or all), Direction in, Source = RFC1918 (or ZONES_ALL), Destination any, Gateway = WAN_GW/WAN_DHCP - Description: BREAK-GLASS: direct WAN egress — enable only when Proton is down

Because it's a floating quick rule it evaluates before the per-zone rules, so enabling it overrides the Proton routing + kill-switch for everyone at once. (Scope the source to just MGMT net if you only want recovery access, not whole-network internet, during an outage.)

Floating / housekeeping

  • Block private networks / Block bogon networks: these are per-interface toggles on Interfaces → WAN (not on Firewall → Settings → Advanced) — keep both enabled on WAN.
  • Anti-lockout: management is via MGMT/WG only (the WG admin peer enters MGMT, runbook 07); the Proxmox console of the OPNsense VM is the break-glass fallback.

Firewall automation rules (as code)

A handful of cross-zone service grants (and one deny) don't fit the per-zone GUI ruleset above cleanly — they're host-scoped passes added over the API into OPNsense's MVC Automation → Filter list. Those live as OPNsense runtime state and can be lost on a rebuild, so they are codified in the repo and reconciled by a script:

  • provision/opnsense/fw-automation-rules.json — the declarative source of truth. Each rule is matched by its exact description. Interface ids: opt1=MGMT opt2=SERVERS opt3=DMZ opt4=WORKSTATIONS opt6=IOT opt7=GUEST opt12=WGRW. It carries, e.g., the sequence-90 SERVERS → (self):443 and sequence-91 WORKSTATIONS → (self):443 blocks (referenced by the SERVERS rule 1 above), the wg-admin → UniFi UI and registry → hidden-primary :53 grants, and the per-proxy_vhosts apps-proxy → DMZ-backend passes that scripts/apps-frontdoor-sync.py emits.
  • scripts/opnsense-fw.py apply [--dry-run] — reconcile the live firewall to the JSON: it only adds missing rules (idempotent, matched by description) and never deletes/edits live rules, so an emergency hand-added rule is never clobbered; reconcile removals by hand. scripts/opnsense-fw.py list shows the live automation rules read-only.
  • tests/fw-conformance/conformance.py — the assertion harness: it probes the expected pass/block outcomes (including SERVERS → 10.20.10.1:443 = block) so drift from the design is caught.

Automation-filter passes evaluate ahead of the classic per-interface rules, so a scoped pass here (e.g. WG_LAPTOPS → SERVERS_NET:22) correctly precedes a classic block → RFC1918 on the same interface — that's why these grants work.

After an OPNsense rebuild: re-add the aliases + GUI ruleset per runbooks 02–08, then scripts/opnsense-fw.py apply to restore these automation rules, then run the conformance harness. Editing policy = edit the JSON, then apply. See provision/opnsense/README.md.

Validation

  • [ ] From a WS test client: internet works and shows Proton egress IP; can reach SERVERS on IPA/NFS ports; cannot reach MGMT or DMZ-internal; cannot reach other client zones.
  • [ ] From GUEST test client: internet only; no RFC1918 reachable.
  • [ ] From IOT test client: only DNS/NTP to fw + Jellyfin; RFC1918 otherwise blocked; DNS intercept still forces Unbound.
  • [ ] Kill-switch verified again (disable Proton → WS internet stops).
  • [ ] Firewall → Log Files → Live View shows expected passes/blocks, no surprise denies for legit flows.

Then → runbook 07 (road-warrior WireGuard + WG-only MGMT).