OPNsense / pfSense config.xml Data Structure Research#

Research conducted Feb 2025 by analyzing upstream OPNsense and pfSense source code, config.xml samples, and the Go schema package. This document informs parsing accuracy and identifies gaps in our data model.

1. XML Boolean Patterns#

OPNsense/pfSense uses two distinct patterns for boolean-like values. Understanding these is critical for correct Go type selection.

1a. Presence-Based Booleans (Self-Closing Tags)#

Element existing = true, absent = false. Content is irrelevant.

Upstream PHP pattern: isset($rule['disabled']) or !empty($rule['disabled'])

Go type: BoolFlag (custom type in pkg/schema/opnsense/common.go)

pfSense Bug #6893 note: Prior to pfSense 2.3.3, some code produced <tag/> while other code produced <tag></tag>. Both forms are valid XML and our *string / BoolFlag types handle both correctly via Go's encoding/xml.

1b. Value-Based Booleans#

Element contains 1, yes, or a specific value. Absent or empty = false.

Upstream PHP pattern: $config['system']['ipv6allow'] == "1"

Go type: string with value check, or potentially a custom type

1c. Design Rationale#

The distinction is not arbitrary:

• Presence-based = flags typically absent (disabled, negation, special modes)
• Value-based = feature toggles typically enabled (with explicit 1)

2. Source/Destination Structure#

The <source> and <destination> elements are the most complex sub-structures in firewall rules. Understanding their design is critical.

2a. XML Structure Examples#

Match any address:

<source>
  <any />
</source>

Match interface subnet:

<source>
  <network>lan</network>
</source>

Match specific IP/CIDR:

<source>
  <address>192.168.1.0/24</address>
</source>

Match alias:

<source>
  <address>MyAlias</address>
</source>

Negated match:

<source>
  <not />
  <network>lan</network>
</source>

With port (TCP/UDP only):

<destination>
  <network>wan</network>
  <port>443</port>
</destination>

Port range:

<destination>
  <any />
  <port>8000-9000</port>
</destination>

2b. Mutual Exclusivity#

<any>, <network>, and <address> are mutually exclusive. Resolution priority (from OPNsense legacyMoveAddressFields):

1. <network> (highest priority)
2. <address>
3. <any> / implicit any (when none present)

2c. Valid <network> Values#

• Interface names: lan, wan, opt1, opt2, ...
• Interface IP: lanip, wanip, opt1ip
• Special: (self) (all local IPs)
• Interface group names
• VIP names

2d. Port Range Delimiters#

• In config.xml: hyphen (80-443)
• In pf rules: colon (80:443)
• OPNsense/pfSense code handles the conversion

3. Filter Rule Fields Reference#

3a. Currently Modeled (in Rule struct)#

3b. Rate-Limiting and Advanced Fields (Added in Phase 3)#

3c. Remaining Missing Fields (LOW importance)#

4. Schema Gaps: Source and Destination#

RESOLVED (Phase 1): All missing fields added directly to Source/Destination structs. Fields were added directly rather than via RuleLocation embedding to preserve the *string Any field and maintain backward compatibility with existing consumers.

4a. Current Implementation (Post-Phase 1)#

// security.go
type Source struct {
    Any *string `xml:"any,omitempty"`
    Network string `xml:"network,omitempty"`
    Address string `xml:"address,omitempty"`
    Port string `xml:"port,omitempty"`
    Not BoolFlag `xml:"not,omitempty"`
}

type Destination struct {
    Any *string `xml:"any,omitempty"`
    Network string `xml:"network,omitempty"`
    Address string `xml:"address,omitempty"`
    Port string `xml:"port,omitempty"`
    Not BoolFlag `xml:"not,omitempty"`
}

4b. Previously Missing Fields (Now Resolved)#

Helper methods IsAny(), EffectiveAddress(), and Equal() were added to both types for safe comparison and address resolution following OPNsense priority rules.

4c. Existing RuleLocation Type#

pkg/schema/opnsense/common.go defines RuleLocation with similar fields. It remains available for other use cases but was not embedded into Source/Destination to avoid complexity with the *string Any field and XML marshaling behavior.

5. Type Mismatches: string vs BoolFlag#

Resolution Status#

Converted to BoolFlag (presence-based — isset() in PHP):

• Rule: Disabled, Quick, Log (security.go) — Phase 2
• NATRule: Disabled, Log (security.go) — Phase 2
• InboundRule: Disabled, Log (security.go) — Phase 2
• System: DisableConsoleMenu (system.go) — Phase 3
• Firmware: Type, Subscription, Reboot (system.go) — Phase 3
• User: Expires, AuthorizedKeys, IPSecPSK, OTPSeed (system.go) — Phase 3
• System.RRD: Enable (system.go) — Phase 3
• Rrd: Enable (services.go) — Phase 3
• OpnSenseDocument: TriggerInitialWizard (opnsense.go) — Phase 3

Kept as string (value-based — == "1" in PHP):

The following fields use OPNsense MVC value-based semantics where <field>0</field> is valid and distinct from absent. BoolFlag would incorrectly treat <field>0</field> as true (element present), breaking the == "1" / == "0" distinction. These remain string:

5a. Security (security.go) — value-based, kept as string#

5b. Services (services.go) — value-based, kept as string#

5c. OPNsense module (opnsense.go) — value-based, kept as string#

5d. DHCP (dhcp.go) — value-based, kept as string#

Note on value-based fields: These use OPNsense MVC pattern <field>1</field> / <field>0</field>. Converting to BoolFlag would break semantics because BoolFlag.UnmarshalXML treats any present element as true, regardless of content — so <enabled>0</enabled> would incorrectly become true.

6. NAT Structure Issues#

RESOLVED (Phase 4): All missing NAT outbound and inbound fields have been added.

6a. NAT XML Path#

In OPNsense config.xml, inbound NAT (port forward) rules are at <nat><rule>, not <nat><inbound><rule>. Our schema uses xml:"inbound>rule" path mapping.

6b. NAT Outbound Rule Fields (Added in Phase 4)#

6c. NAT Inbound Rule Fields (Added in Phase 4)#

6d. Outbound NAT Mode Values#

7. Key XML Design Decisions#

7a. Dynamic Interface Keys#

The <interfaces> section uses dynamic element names (<wan>, <lan>, <opt0>) rather than repeated <interface name="wan">. This requires custom unmarshaling in Go. Our map-based approach in Interfaces and Dhcpd types is correct.

7b. Comma-Separated Lists#

Several fields pack multiple values into a single element:

• <interface>wan,lan,opt1</interface> (floating rules)
• <icmptype>3,11,0</icmptype> (ICMP type list)
• Space-separated: <timeservers>0.ntp.org 1.ntp.org</timeservers>

Our InterfaceList custom type correctly handles the comma-separated case.

7c. UUID vs Tracker#

• OPNsense: uuid attribute on rule elements (<rule uuid="...">)
• pfSense: <tracker> element (integer, auto-generated from microtime)
• Both may be present in migrated configs

7d. Legacy vs MVC Model Split#

OPNsense maintains two parallel systems:

• Legacy: Rules in <filter><rule> and <nat><rule> (pfSense-compatible)
• MVC/New-style: Rules in <OPNsense><Firewall><Filter> with <rules>, <snatrules>, etc.

Both are loaded by pf_firewall(). Our schema currently only models the legacy format.

8. Correctly Implemented Patterns#

8a. Source.Any / Destination.Any as *string#

Go's encoding/xml produces "" for both <any/> and absent elements when using plain string. Using *string correctly distinguishes presence (non-nil) from absence (nil). Validated against FilterRule.php's isset() pattern.

8b. BoolFlag Custom Type#

Correctly implements presence-based boolean semantics: UnmarshalXML sets true on any element presence, regardless of content.

8c. InterfaceList Custom Type#

Correctly handles comma-separated interface lists for floating rules.

8d. Interfaces and Dhcpd Map Types#

Correctly handle dynamic interface element names via custom UnmarshalXML/MarshalXML.

8e. RuleLocation Struct#

Already has all needed fields for complete source/destination modeling. Just needs to be connected to the actual Source/Destination types.

9. Action Items (Priority Order)#

1. CRITICAL: Add Address field to Source and Destination structs — COMPLETE
2. HIGH: Add Not field (BoolFlag) to Source and Destination — COMPLETE
3. HIGH: Add Port field to Source struct — COMPLETE
4. HIGH: Add Log field (BoolFlag) to Rule struct — COMPLETE
5. HIGH: Convert Rule.Disabled, NATRule.Disabled, InboundRule.Disabled from string to BoolFlag — COMPLETE
6. HIGH: Add Floating, Gateway, Direction fields to Rule — COMPLETE
7. MEDIUM: Convert struct{} fields in system.go to BoolFlag — COMPLETE
8. MEDIUM: Add rate-limiting fields to Rule (max-src-nodes, etc.) — COMPLETE (Phase 3: 14 fields added — rate-limiting, TCP/ICMP, state/advanced)
9. MEDIUM: Convert IDS/IPsec Enabled fields from string to BoolFlag
10. LOW: Add remaining missing filter rule fields (tag, tagged, DSCP, etc.)
11. LOW: Add missing NAT rule fields — COMPLETE (Phase 4: NATRule +4 fields, InboundRule +5 fields)

10. Remaining Work#

10a. Medium Priority#

10b. Low Priority#

10c. Phase Summary#

11. Kea DHCP4 XML Structure#

This section documents the full <Kea><dhcp4> XML structure in OPNsense config.xml as parsed by the Go schema package (pkg/schema/opnsense/kea.go).

11a. <dhcp4> Top-Level Structure#

<dhcp4 version="1.0.4">
  <general>
    <enabled>1</enabled>
    <interfaces>lan</interfaces>
    <fwrules>1</fwrules>
    <valid_lifetime>4000</valid_lifetime>
  </general>
  <ha>
    <enabled>0</enabled>
    <this_server_name />
    <max_unacked_clients>2</max_unacked_clients>
  </ha>
  <subnets>
    <subnet4 uuid="..."> ... </subnet4>
  </subnets>
  <reservations>
    <reservation uuid="..."> ... </reservation>
  </reservations>
  <ha_peers />
</dhcp4>

Go type mapping:

11b. KeaSubnet (<subnet4> element)#

Each <subnet4> element carries a uuid attribute and the following child elements:

Pools format note: The <pools> field stores entries from OPNsense's KeaPoolsField. Each newline-separated entry is either a range (192.168.1.100-192.168.1.200) or CIDR notation. The converter uses the first entry as DHCPScope.Range; additional entries emit a SeverityInfo warning.

11c. KeaOptionData (<option_data> element)#

Used on both <subnet4> and <reservation> elements.

11d. KeaReservation (<reservation> element)#

11e. Converter Behavior (convertKeaDHCPScopes)#

convertKeaDHCPScopes() in pkg/parser/opnsense/converter_subsystems.go converts Kea subnets into []common.DHCPScope entries and appends them to CommonDevice.DHCP alongside ISC scopes (which carry Source: "isc"). Key behaviors:

• Produces no scopes when <subnets> is empty. Scopes are emitted even when Kea is disabled (Enabled reflects the server state).
• Each DHCPScope has Source: "kea" and Enabled set from the Kea general enabled flag.
• Option data is mapped: Routers → Gateway, DomainNameServers → DNSServer, NTPServers → NTPServer. For comma-separated values, only the first entry is used.
• Reservations are grouped by their <subnet> UUID and attached as StaticLeases on the matching scope. Reservations referencing a nonexistent subnet UUID are not attached and emit a conversion warning.
• ISC DHCP scopes (OPNsense and pfSense) carry Source: "isc". An empty Source is treated as "isc" for backward compatibility.