OPNsense / pfSense config.xml Data Structure Research#

Research conducted Feb 2025 by analyzing upstream OPNsense and pfSense source code, config.xml samples, and the Go schema package. This document informs parsing accuracy and identifies gaps in our data model.

1. XML Boolean Patterns#

OPNsense/pfSense uses two distinct patterns for boolean-like values. Understanding these is critical for correct Go type selection.

1a. Presence-Based Booleans (Self-Closing Tags)#

Element existing = true, absent = false. Content is irrelevant.

Upstream PHP pattern: isset($rule['disabled']) or !empty($rule['disabled'])

Go type: BoolFlag (custom type in pkg/schema/opnsense/common.go)

pfSense Bug #6893 note: Prior to pfSense 2.3.3, some code produced <tag/> while other code produced <tag></tag>. Both forms are valid XML and our *string / BoolFlag types handle both correctly via Go's encoding/xml.

pfSense presence-based enable note: pfSense correctly parses Interface and DhcpdInterface <enable/> elements as presence-based using BoolFlag types. The public API converts to string "1" for backward compatibility.

1b. Value-Based Booleans#

Element contains 1, yes, or a specific value. Absent or empty = false.

Upstream PHP pattern: $config['system']['ipv6allow'] == "1"

Go type: string with value check, or BoolFlag (which now delegates non-empty body content to shared.IsValueTrue)

1c. Design Rationale#

The distinction is not arbitrary:

• Presence-based = flags typically absent (disabled, negation, special modes)
• Value-based = feature toggles typically enabled (with explicit 1)

Note: As of PR #577, BoolFlag now delegates non-empty element bodies to shared.IsValueTrue, so int or string toggle fields can migrate to BoolFlag when the semantics allow. This addresses issue #558 where OPNsense 26.1 emits on into fields previously typed as int, causing parse failures. The canonical truthy/falsy parsers are implemented in pkg/schema/shared/ (see §8b).

BoolFlag is NOT a drop-in for every value-based boolean. BoolFlag.MarshalXML emits a self-closing element for true and emits nothing at all for false. That means a field whose on-wire representation is always <tag>0</tag> or <tag>1</tag> (and must remain so on round-trip) cannot migrate to BoolFlag — the false case would disappear from the output. Criteria for safe migration:

1. Absence of the element must be semantically equivalent to false in the target device's behavior (OPNsense/pfSense treat most toggles this way — absent = disabled).
2. No round-trip consumer must depend on the literal <tag>0</tag> form when the underlying value is false.

If either criterion fails, prefer shared.FlexBool (always emits <tag>0</tag> or <tag>1</tag>) or keep the field as string and call shared.IsValueTrue at the converter.

2. Source/Destination Structure#

The <source> and <destination> elements are the most complex sub-structures in firewall rules. Understanding their design is critical.

2a. XML Structure Examples#

<!-- Match any address -->
<source><any/></source>

<!-- Match interface subnet -->
<source><network>lan</network></source>

<!-- Match specific IP/CIDR -->
<source><address>192.168.1.0/24</address></source>

<!-- Match alias -->
<source><address>MyAlias</address></source>

<!-- Negated match -->
<source><not/><network>lan</network></source>

<!-- With port (TCP/UDP only) -->
<destination><network>wan</network><port>443</port></destination>

<!-- Port range -->
<destination><any/><port>8000-9000</port></destination>

2b. Mutual Exclusivity#

<any>, <network>, and <address> are mutually exclusive. Resolution priority (from OPNsense legacyMoveAddressFields):

1. <network> (highest priority)
2. <address>
3. <any> / implicit any (when none present)

2c. Valid <network> Values#

• Interface names: lan, wan, opt1, opt2, ...
• Interface IP: lanip, wanip, opt1ip
• Special: (self) (all local IPs)
• Interface group names
• VIP names

2d. Port Range Delimiters#

• In config.xml: hyphen (80-443)
• In pf rules: colon (80:443)
• OPNsense/pfSense code handles the conversion

3. Filter Rule Fields Reference#

3a. Currently Modeled (in Rule struct)#

3b. Rate-Limiting and Advanced Fields (Added in Phase 3)#

3c. Remaining Missing Fields (LOW importance)#

4. Schema Gaps: Source and Destination#

RESOLVED (Phase 1): All missing fields added directly to Source/Destination structs. Fields were added directly rather than via RuleLocation embedding to preserve the *string Any field and maintain backward compatibility with existing consumers.

4a. Current Implementation (Post-Phase 1)#

// security.go
type Source struct {
    Any *string `xml:"any,omitempty"`
    Network string `xml:"network,omitempty"`
    Address string `xml:"address,omitempty"`
    Port string `xml:"port,omitempty"`
    Not BoolFlag `xml:"not,omitempty"`
}

type Destination struct {
    Any *string `xml:"any,omitempty"`
    Network string `xml:"network,omitempty"`
    Address string `xml:"address,omitempty"`
    Port string `xml:"port,omitempty"`
    Not BoolFlag `xml:"not,omitempty"`
}

4b. Previously Missing Fields (Now Resolved)#

Helper methods IsAny(), EffectiveAddress(), and Equal() were added to both types for safe comparison and address resolution following OPNsense priority rules.

4c. Existing RuleLocation Type#

pkg/schema/opnsense/common.go defines RuleLocation with similar fields. It remains available for other use cases but was not embedded into Source/Destination to avoid complexity with the *string Any field and XML marshaling behavior.

5. Type Mismatches: string vs BoolFlag#

Resolution Status#

Converted to BoolFlag (presence-based — isset() in PHP):

• Rule: Disabled, Quick, Log (security.go) — Phase 2
• NATRule: Disabled, Log (security.go) — Phase 2
• InboundRule: Disabled, Log (security.go) — Phase 2
• System: DisableConsoleMenu (system.go) — Phase 3
• Firmware: Type, Subscription, Reboot (system.go) — Phase 3
• User: Expires, AuthorizedKeys, IPSecPSK, OTPSeed (system.go) — Phase 3
• System.RRD: Enable (system.go) — Phase 3
• Rrd: Enable (services.go) — Phase 3
• OpnSenseDocument: TriggerInitialWizard (opnsense.go) — Phase 3

Kept as string (value-based — == "1" in PHP):

The following fields use OPNsense MVC value-based semantics where <field>0</field> is valid and distinct from absent. BoolFlag would incorrectly treat <field>0</field> as true (element present), breaking the == "1" / == "0" distinction. These remain string:

5a. Security (security.go) — value-based, kept as string#

5b. Services (services.go) — value-based, kept as string#

5c. OPNsense module (opnsense.go) — value-based, kept as string#

5d. Interfaces and DHCP (interfaces.go, dhcp.go) — platform-specific#

pfSense and OPNsense use different boolean semantics for interface and DHCP scope enable flags:

Implementation: The pfSense parser uses an intermediate decodeDocument type (pkg/parser/pfsense/decode_types.go) that decodes XML with BoolFlag-aware pfsense.Interface and pfsense.DhcpdInterface types, then converts to backward-compatible opnsense.Interfaces/opnsense.Dhcpd with string Enable fields set to "1" (enabled) or "" (disabled).

6. NAT Structure Issues#

RESOLVED (Phase 4): All missing NAT outbound and inbound fields have been added.

6a. NAT XML Path#

In OPNsense config.xml, inbound NAT (port forward) rules are at <nat><rule>, not <nat><inbound><rule>. Our schema uses xml:"inbound>rule" path mapping.

6b. NAT Outbound Rule Fields (Added in Phase 4)#

6c. NAT Inbound Rule Fields (Added in Phase 4)#

6d. Outbound NAT Mode Values#

7. Key XML Design Decisions#

7a. Dynamic Interface Keys#

The <interfaces> section uses dynamic element names (<wan>, <lan>, <opt0>) rather than repeated <interface name="wan">. This requires custom unmarshaling in Go. Our map-based approach in Interfaces and Dhcpd types is correct.

7b. Comma-Separated Lists#

Several fields pack multiple values into a single element:

• <interface>wan,lan,opt1</interface> (floating rules)
• <icmptype>3,11,0</icmptype> (ICMP type list)
• Space-separated: <timeservers>0.ntp.org 1.ntp.org</timeservers>

Our InterfaceList custom type correctly handles the comma-separated case.

7c. UUID vs Tracker#

• OPNsense: uuid attribute on rule elements (<rule uuid="...">)
• pfSense: <tracker> element (integer, auto-generated from microtime)
• Both may be present in migrated configs

7d. Legacy vs MVC Model Split#

OPNsense maintains two parallel systems:

• Legacy: Rules in <filter><rule> and <nat><rule> (pfSense-compatible)
• MVC/New-style: Rules in <OPNsense><Firewall><Filter> with <rules>, <snatrules>, etc.

Both are loaded by pf_firewall(). Our schema currently only models the legacy format.

8. Correctly Implemented Patterns#

8a. Source.Any / Destination.Any as *string#

Go's encoding/xml produces "" for both <any/> and absent elements when using plain string. Using *string correctly distinguishes presence (non-nil) from absence (nil). Validated against FilterRule.php's isset() pattern.

8b. BoolFlag Custom Type#

Correctly implements presence-based boolean semantics: absent element → false, <tag/> (empty body) → true, <tag>body</tag> → delegates to shared.IsValueTrue(body). This means <tag>0</tag>, <tag>off</tag>, and <tag>no</tag> correctly parse as false, fixing a latent bug where any element presence previously returned true regardless of content.

Shared boolean vocabulary: The pkg/schema/shared/ package defines the canonical liberal parsers:

• IsValueTrue(s) / IsValueFalse(s) — recognizes "on"/"off", "yes"/"no", "1"/"0", "true"/"false", "enable"/"disable", "enabled"/"disabled" (case-insensitive)
• FlexBool — value-level liberal boolean for fields where element presence is not the signal (always emitted, content carries the boolean value)
• FlexInt — liberal int parser sibling (delegates to IsValueTrue for non-numeric content)

BoolFlag.UnmarshalXML layers presence semantics over shared.IsValueTrue — absent → false, <tag/> → true, <tag>body</tag> → IsValueTrue(body). FlexBool delegates to the same IsValueTrue helper for its body parsing, so the two types share the liberal vocabulary without one depending on the other. pfSense converters use shared.IsValueTrue directly (retired pfsense.isPfSenseValueTrue).

8c. InterfaceList Custom Type#

Correctly handles comma-separated interface lists for floating rules.

8d. Interfaces and Dhcpd Map Types#

Correctly handle dynamic interface element names via custom UnmarshalXML/MarshalXML.

8e. decodeDocument Intermediate Layer (pfSense parser)#

The pfSense parser uses a two-stage architecture for correct presence-based boolean handling:

1. Decode stage: XML is decoded into an intermediate decodeDocument type (defined in pkg/parser/pfsense/decode_types.go) that uses platform-specific types with BoolFlag-aware Enable fields:

   • pfsense.Interface with Enable opnsense.BoolFlag (from pkg/schema/pfsense/interfaces.go)
   • pfsense.DhcpdInterface with Enable opnsense.BoolFlag (from pkg/schema/pfsense/dhcp.go)

2. Conversion stage: The intermediate document is converted to a pfsense.Document with backward-compatible opnsense types via toDocument():

   • BoolFlag Enable fields are transformed to string: true → "1", false → ""
   • This preserves API compatibility with existing consumers expecting string Enable fields

This architecture resolves the type mismatch issue described in Section 5 by using platform-specific type forks at the decode layer while maintaining a unified public API. It correctly distinguishes pfSense's presence-based <enable/> (element present = enabled) from absent elements (disabled), which was previously mishandled when using string types that cannot distinguish <enable/> from <enable></enable> from absent elements.

9. Action Items (Priority Order)#

1. CRITICAL: Add Address field to Source and Destination structs — COMPLETE
2. HIGH: Add Not field (BoolFlag) to Source and Destination — COMPLETE
3. HIGH: Add Port field to Source struct — COMPLETE
4. HIGH: Add Log field (BoolFlag) to Rule struct — COMPLETE
5. HIGH: Convert Rule.Disabled, NATRule.Disabled, InboundRule.Disabled from string to BoolFlag — COMPLETE
6. HIGH: Add Floating, Gateway, Direction fields to Rule — COMPLETE
7. MEDIUM: Convert struct{} fields in system.go to BoolFlag — COMPLETE
8. MEDIUM: Add rate-limiting fields to Rule (max-src-nodes, etc.) — COMPLETE (Phase 3: 14 fields added — rate-limiting, TCP/ICMP, state/advanced)
9. MEDIUM: Convert IDS/IPsec Enabled fields from string to BoolFlag
10. LOW: Add remaining missing filter rule fields (tag, tagged, DSCP, etc.)
11. LOW: Add missing NAT rule fields — COMPLETE (Phase 4: NATRule +4 fields, InboundRule +5 fields)

10. Remaining Work#

10a. Medium Priority#

10b. Low Priority#

10c. Phase Summary#