opnDossier Architecture Overview#

Document Purpose#

This document provides a high-level architectural overview of opnDossier, a CLI-based multi-device firewall configuration processor. It covers the key architectural components including parsers, schemas, the CommonDevice model, report plugins, and audit plugins.

1. System Overview#

opnDossier is an offline-first, operator-focused CLI application designed to transform complex XML configuration files from multiple firewall platforms (OPNsense and pfSense) into human-readable markdown documentation, with security analysis and compliance checking capabilities. The system follows a security-first design with complete air-gap compatibility.

Core Design Principles#

1. Offline-First: Zero external dependencies, complete air-gap compatibility
2. Operator-Focused: Built for network administrators and operators
3. Framework-First: Leverages established Go libraries (Cobra, Viper, Charm ecosystem)
4. Structured Data: Maintains configuration hierarchy and relationships
5. Security-First: No telemetry, comprehensive input validation, secure processing

Architecture Pattern#

opnDossier follows a monolithic CLI application pattern with clear separation of concerns:

• Single binary distribution for easy deployment
• Local processing only with no external network calls
• Streaming data pipeline from XML input to various output formats

CLI Command Architecture#

The system provides multiple first-class commands for different operational workflows:

• opndossier convert: Configuration transformation and documentation generation
• opndossier audit: Dedicated security audit and compliance checking command with support for blue mode (defensive audit) and red mode (attack surface analysis)
• opndossier display: Interactive viewing of configurations
• opndossier diff: Configuration comparison and change detection
• opndossier validate: Configuration syntax and semantic validation

The audit command provides a focused interface specifically designed for security and compliance assessments, with two audit modes: blue (defensive audit with security findings and recommendations) and red (attacker-focused recon report).

2. Parsers and Schemas#

2.1 XML Parser Architecture#

The XML parser is implemented in internal/cfgparser/xml.go using a streaming token-based approach rather than DOM parsing for better memory efficiency and security.

Parser Structure#

The XMLParser struct processes XML using streaming tokens:

• Streaming Processing: Reads XML tokens individually rather than loading entire documents into memory
• Character Encoding Support: Supports UTF-8, US-ASCII, ISO-8859-1, and Windows-1252
• Element Routing: The handleStartElement function routes XML elements to corresponding schema fields, handling 30+ configuration sections

Security Features#

The parser implements multiple layers of security protection:

1. XXE Prevention: Sets dec.Entity to an empty map to disallow all entity declarations, preventing XML External Entity attacks
2. XML Bomb Protection:
   • 10MB maximum input size limit
   • Streaming architecture prevents memory exhaustion
   • io.LimitReader enforces size limits at I/O level
3. Context-Based Cancellation: Supports timeouts and cancellation to prevent long-running operations

2.2 Schema Structure (XML DTOs)#

opnDossier provides device-specific schema packages for each supported platform:

OPNsense Schema#

The OPNsense schema is defined in pkg/schema/opnsense/ with XML Data Transfer Objects (DTOs) organized across multiple files. These DTOs mirror the OPNsense XML configuration structure with xml: tags.

Root Document#

The OpnSenseDocument struct in pkg/schema/opnsense/opnsense.go serves as the root, containing 30+ configuration sections including system, interfaces, DHCP, firewall, NAT, VPN, certificates, and services.

Key Schema Files#

• common.go: Common types like BoolFlag (custom XML marshaling for OPNsense boolean values)
• system.go: System configuration including hostname, users, groups, web GUI, SSH
• interfaces.go: Network interfaces with custom XML marshaling for map-based interface representation
• security.go: Firewall rules, NAT, IDS/IPS, IPsec
• network.go: Gateways, gateway groups, static routes
• Additional files: dhcp.go, vpn.go, certificates.go, services.go, high_availability.go, packages.go, revision.go

XML-Specific Characteristics#

Schema DTOs handle OPNsense XML quirks:

• Presence-based booleans: Empty element means true, absent means false
• Pointer-heavy fields: Uses *string to distinguish unset vs. empty values
• Map-keyed collections: Interfaces use map[string]InterfaceItem for XML unmarshaling
• String-encoded values: Many numeric/boolean values stored as strings in XML

pfSense Schema#

The pfSense schema is defined in pkg/schema/pfsense/ following a copy-on-write pattern: reuse OPNsense types where XML structures are identical, fork locally at divergence points.

Root Document#

The Document struct in pkg/schema/pfsense/document.go serves as the root with <pfsense> XML tag (vs OPNsense's <opnsense>). The document reuses 25+ OPNsense types for identical sections (interfaces, dhcpd, snmpd, openvpn, rrd, load_balancer, staticroutes, ppps, gateways, ca, cert, vlans, revision) while forking pfSense-specific types for divergent sections.

Key Structural Differences#

pfSense XML diverges from OPNsense in several areas:

• NAT port forwards: pfSense uses <nat><rule> (direct child) with <target> field for redirect IP, while OPNsense uses <nat><inbound><rule> (nested) with <internalip> field
• User authentication: pfSense uses <bcrypt-hash> for passwords and per-user <priv>[] arrays for privileges, while OPNsense uses <password> (SHA-based) with group-based privilege model
• DNS servers: pfSense uses <dnsserver>[] (repeating XML elements parsed as array), while OPNsense uses single <dnsserver> string
• DHCPv6: pfSense includes RAMode and RAPriority fields not present in OPNsense
• System tunables: Both platforms support kernel tunables, but pfSense stores them at <system><sysctl><item>[] with tunable/value/descr fields

Schema Reuse Strategy#

The pfSense schema imports pkg/schema/opnsense types extensively to avoid duplication. For example:

• Document.Interfaces uses opnsense.Interfaces (identical map-based structure)
• Document.Dhcpd uses opnsense.Dhcpd (identical per-interface DHCP configuration)
• OutboundNAT reuses opnsense.Outbound (identical structure)
• Only fork types where XML structure differs (e.g., InboundRule with Target field, FilterRule with additional fields, User with bcrypt authentication)

IPsec VPN Schema#

The pfSense IPsec schema is defined in pkg/schema/pfsense/vpn.go with dedicated types for IPsec tunnel configuration:

• IPsec: Top-level container with Phase1[], Phase2[], Client, Logging
• IPsecPhase1: IKE Phase 1 (SA) configuration with 27 fields including identity (MyIDType/Data, PeerIDType/Data), crypto (Encryption algorithms with key lengths), timing (Lifetime, RekeyTime, ReauthTime, RandTime), NAT traversal settings, MOBIKE, DPD, certificate references, custom ports, split connection
• IPsecPhase2: Phase 2 (child SA) configuration with 20 fields including network identity (LocalID, RemoteID with Type/Address/Netbits), NATLocalID for BINAT, encryption/hash algorithms with key lengths, PFS group, lifetime, ping host
• IPsecClient: Mobile/road-warrior client pool with 13 fields including IPv4/IPv6 pools, DNS/WINS servers, split DNS, login banner, password persistence
• IPsecLogging: Per-subsystem log level configuration

BoolFlag patterns: Uses presence-based BoolFlag for disabled, mobile, enable, save_passwd elements. All structs with BoolFlag fields implement MarshalXML via type alias pattern (GOTCHAS.md §15.1) to ensure addressable fields during marshaling.

See pkg/schema/pfsense/README.md for comprehensive pfSense schema reference including listtag arrays, config version mapping, and detailed field inventories.

Shared XML Security Hardening#

The pkg/parser/xmlutil.go provides shared XML parsing utilities used by both OPNsense and pfSense parsers:

• NewSecureXMLDecoder: Creates an *xml.Decoder with security hardening (input size limit, XXE prevention via empty entity map, charset reader for UTF-8/US-ASCII/ISO-8859-1/Windows-1252)
• CharsetReader: Handles XML charset declarations for multiple encodings, with normalization for charset name variations

Both device-specific parsers delegate to this shared infrastructure to avoid duplicating security hardening logic.

2.3 Device Parser Factory and Auto-Detection#

The factory in pkg/parser/factory.go provides automatic device type detection and parser delegation using a thread-safe DeviceParserRegistry pattern analogous to the format handling registry in the converter subsystem (see section 4.5.1).

DeviceParserRegistry Pattern#

The factory uses a thread-safe DeviceParserRegistry (pkg/parser/registry.go) to look up parsers instead of hardcoded switch statements. The registry follows the database/sql driver pattern with self-registration:

• Self-Registration: Parsers register themselves via init() functions (e.g., pkg/parser/opnsense/parser.go calls parser.Register("opnsense", NewParserFactory))
• Registry API: Register(deviceType, fn), Get(deviceType), List(), DefaultRegistry()
• Thread-Safe: All operations protected by sync.RWMutex, singleton initialization via sync.Once
• Dynamic Discovery: List() enables CLI completions and error messages to enumerate supported device types
• Test Isolation: NewFactoryWithRegistry() accepts custom registries for test isolation without global pollution

Critical: Blank Import Requirement

Code using parser.NewFactory() must include blank imports for all desired parsers, or the registry will be empty:

_ "github.com/EvilBit-Labs/opnDossier/pkg/parser/opnsense"
_ "github.com/EvilBit-Labs/opnDossier/pkg/parser/pfsense"

Device-specific parsers only register when their init() runs. Missing blank imports cause "unsupported device type" errors with empty supported lists. The canonical blank imports live in cmd/root.go, but test files using the factory must add their own.

Auto-Detection Mechanism#

The peekRootElementBounded function performs device type detection:

1. Uses bounded LimitReader (10MB cap) for safety
2. Employs TeeReader to buffer consumed bytes
3. Runs detection in a goroutine with context cancellation support
4. Returns a combined reader (buffered bytes + remainder) for full parsing

The createWithAutoDetect method uses registry-based dispatch:

fn, ok := f.registry.Get(rootElem)
if !ok {
    return nil, nil, fmt.Errorf(
        "unsupported device type: root element <%s> is not recognized; supported: %s",
        rootElem, strings.Join(f.registry.List(), ", "),
    )
}
return parseDevice(ctx, fn(f.xmlDecoder), fullReader, validateMode)

Error messages dynamically list supported device types from registry.List(), eliminating hardcoded device lists.

Device-Specific Parsers#

Device-specific parsers implement the DeviceParser interface and register themselves with the global registry via init() functions:

• pkg/parser/opnsense/parser.go: Wraps the XML parser and converts OPNsense schema DTOs to the CommonDevice model. Uses the exported parser.XMLDecoder interface (the local xmlDecoder interface was removed in favor of the public API).

• pkg/parser/pfsense/parser.go: Manages its own XML decoding via parser.NewSecureXMLDecoder because the shared parser.XMLDecoder interface returns *schema.OpnSenseDocument which is incompatible with pfsense.Document. The parser decodes directly into pfsense.Document and converts to CommonDevice via the pfSense-specific converter.

DeviceParser Interface: The DeviceParser interface defines the contract for device-specific parsers:

• Parse(ctx, r) returns (*CommonDevice, []ConversionWarning, error) for structural parsing
• ParseAndValidate(ctx, r) returns (*CommonDevice, []ConversionWarning, error) with additional semantic validation

The three-value return signature allows parsers to surface non-fatal conversion issues alongside the parsed device model.

Implementation Reference: See docs/solutions/architecture-issues/pluggable-deviceparser-registry-pattern.md for comprehensive implementation details, testing strategy, and extensibility guidance.

3. CommonDevice Model#

3.1 Architecture and Purpose#

The CommonDevice struct in pkg/model/device.go is the platform-agnostic, normalized representation of firewall configurations. It transforms XML-specific data into clean Go structs suitable for analysis, reporting, and multi-device support.

3.2 Key Design Differences from XML DTOs#

The CommonDevice model differs fundamentally from XML schema DTOs:

3.3 Converter Pattern#

The Converter in pkg/parser/opnsense/converter.go implements the transformation from schema DTOs to CommonDevice:

• Main Conversion: The ToCommonDevice method orchestrates 30+ domain-specific converters and returns (*CommonDevice, []ConversionWarning, error) to capture both fatal errors and non-fatal issues
• Domain Converters: Each domain has dedicated methods like convertSystem, convertInterfaces, convertFirewallRules, and convertVPN
• Normalization: Converts XML quirks (BoolFlag, string booleans, int-as-bool) to idiomatic Go types
• Type Safety: Uses typed enum constants instead of string literals throughout conversion (e.g., common.RuleTypePass, common.OutboundAutomatic, common.LAGGProtocolLACP), eliminating magic strings at compile boundaries
• Map-to-Slice: Converts XML maps to sorted slices for deterministic output
• Warning Accumulation: Uses an addWarning() method to record non-fatal issues such as missing firewall rule fields, empty gateway addresses, or incomplete NAT configurations

pfSense IPsec Converter#

The pfSense converter in pkg/parser/pfsense/converter_services.go includes dedicated IPsec conversion functions:

• convertVPN: Now includes IPsec conversion alongside OpenVPN
• convertIPsec: Returns zero-value when no tunnels or mobile client exist, sets Enabled: true otherwise
• convertIPsecPhase1Tunnels: Maps 27 Phase 1 fields including identity, crypto algorithms with key lengths, timing parameters (rekey/reauth/rand), NAT-T, MOBIKE, DPD, certificate references
• convertIPsecPhase2Tunnels: Maps 20 Phase 2 fields including network identity with netbits, NATLocalID, encryption/hash algorithms with key lengths, PFS group, ping host
• convertIPsecMobileClient: Maps 13 mobile client fields including IPv4/IPv6 pools, DNS/WINS servers, split DNS, login banner, password persistence
• convertIPsecEncryptionAlgorithms: Shared helper extracting algorithm names with optional key lengths, used by both Phase 1 and Phase 2

Pre-shared key handling: The PreSharedKey field intentionally remains in the schema layer and is not mapped to the common model, relying on the sanitizer to handle sensitive data redaction.

ConversionWarning Type#

The ConversionWarning type in pkg/model/warning.go represents non-fatal issues encountered during schema-to-CommonDevice conversion:

• Field: Dot-path of the problematic field (e.g., "FirewallRules[0].Type")
• Value: The problematic value encountered
• Message: Human-readable description of the issue
• Severity: Importance level using the analysis.Severity type (critical, high, medium, low, info)

Warnings are generated for incomplete or problematic data that doesn't prevent conversion but may indicate configuration issues, including:

• Firewall rules with missing type, source, destination, or interface
• NAT rules without internal IP or interface assignment
• Gateways with empty addresses or names
• Users missing name or UID fields
• Certificates with empty PEM data
• High Availability sync targets with missing credentials

3.4 Domain Organization#

CommonDevice organizes configuration into four primary domains:

System Domain#

Defined in pkg/model/system.go:

• Hostname, domain, DNS servers, time servers
• Web GUI configuration (protocol, port, certificate)
• SSH configuration
• Firmware settings
• Hardware offloading options
• System-level toggles (IPv6, console menu, NAT reflection)

Network Domain#

Defined in pkg/model/network.go:

• Interfaces: Physical/logical interfaces with IP configuration
• VLANs: 802.1Q VLAN tagging
• Bridges: Network bridges with STP
• LAGGs: Link aggregation with typed protocol enum (common.LAGGProtocolLACP, common.LAGGProtocolFailover, common.LAGGProtocolLoadBalance, common.LAGGProtocolRoundRobin)
• Virtual IPs: CARP, IP alias, proxy ARP with typed mode enum (common.VIPModeCarp, common.VIPModeIPAlias, common.VIPModeProxyARP)
• Tunnels: GIF/GRE interfaces
• Routing: Gateways, gateway groups, static routes

Security Domain#

Defined in pkg/model/firewall.go and related files:

• FirewallRules: Firewall rules with typed actions (common.RuleTypePass, common.RuleTypeBlock, common.RuleTypeReject), typed direction (common.DirectionIn, common.DirectionOut, common.DirectionAny), typed IP protocol (common.IPProtocolInet, common.IPProtocolInet6), and normalized RuleEndpoint structure. Rule types use typed string enums instead of bare string literals for improved type safety.
• NAT: Outbound NAT with typed mode enum (common.OutboundAutomatic, common.OutboundHybrid, common.OutboundAdvanced, common.OutboundDisabled), and inbound NAT (port forwards) with typed IP protocol enums
• IDS: Suricata IDS/IPS configuration
• VPN: OpenVPN, WireGuard, IPsec configurations with platform-agnostic tunnel definitions:
  • IPsecPhase1Tunnel: 27 fields covering IKE identity, crypto algorithms with key lengths, timing (rekey/reauth/rand), NAT-T, MOBIKE, DPD, certificate references, custom ports, split connection
  • IPsecPhase2Tunnel: 20 fields covering network identity with netbits, NATLocalID for BINAT, encryption/hash algorithms with key lengths, PFS group, lifetime, ping host
  • IPsecMobileClient: 13 fields covering IPv4/IPv6 virtual address pools, DNS/WINS servers, split DNS, login banner, password persistence

Services Domain#

Defined in pkg/model/services.go:

• DHCP: DHCP server per interface with static leases. Advanced configuration fields now organized into separate DHCPAdvancedV4 and DHCPAdvancedV6 structs for better type organization.
• DNS: Unbound + dnsmasq configuration
• NTP: Time synchronization
• SNMP: Monitoring
• Syslog: Remote logging
• Monit: Process monitoring
• Netflow: NetFlow/IPFIX accounting

Compliance Domain#

Defined in pkg/model/enrichment.go, the compliance domain contains audit results:

• ComplianceResults: Top-level audit results container with mode, findings, per-plugin results, summary, and metadata
• ComplianceFinding: Individual compliance findings with type, severity, title, description, recommendation, component, and references
• PluginComplianceResult: Per-plugin results including plugin info, findings, summary, controls, and per-control compliance status
• ComplianceControl: Control definitions with ID, title, description, category, severity, rationale, remediation, references, tags, and metadata. Each control includes a Status field populated from the Compliance map during mapping, with values ControlStatusPass ("PASS") or ControlStatusFail ("FAIL"). This makes exports self-describing.
• ComplianceResultSummary: Aggregate statistics including total findings, findings by severity level, plugin count, and compliant/non-compliant control counts

These types mirror the structure of audit.Report / analysis.Finding / audit.ComplianceResult / compliance.Control but live in the pkg/model package (public API) to avoid circular dependencies with internal/audit/compliance packages. CommonDevice.ComplianceChecks uses *ComplianceResults and is populated by mapAuditReportToComplianceResults() in cmd/audit_handler.go.

4. Report Plugins#

4.1 Report Builder Architecture#

The report generation system in internal/converter/builder/builder.go provides programmatic markdown generation with compile-time type safety.

Core Interfaces#

The builder system follows the Interface Segregation Principle, organizing functionality into three focused sub-interfaces:

• SectionBuilder (9 methods): Defines methods for building individual configuration sections (BuildSystemSection, BuildNetworkSection, BuildSecuritySection, BuildServicesSection, BuildIPsecSection, BuildOpenVPNSection, BuildHASection, BuildIDSSection, BuildAuditSection) and configuration (SetIncludeTunables)

• TableWriter (11 methods): Defines methods for writing formatted data tables (WriteFirewallRulesTable, WriteInterfaceTable, WriteUsersTable, WriteNATRulesTable, WriteStaticRoutesTable, WriteVLANTable, WriteGatewaysTable, WriteOpenVPNTable, WriteIPsecTable, WriteDHCPSummaryTable, WriteDHCPStaticLeasesTable)

• ReportComposer (2 methods): Defines methods for assembling complete reports (BuildStandardReport, BuildComprehensiveReport)

• ReportBuilder interface: Composes all three sub-interfaces (SectionBuilder, TableWriter, ReportComposer) for full backward compatibility. Consumers can depend on only the methods they need by accepting narrower interfaces.

• MarkdownBuilder struct: Concrete implementation satisfying the full ReportBuilder interface via compile-time assertion

• SectionWriter interface: Provides streaming output for memory efficiency

Consumer-Local Interface Narrowing#

The HybridGenerator demonstrates consumer-local interface narrowing: its internal builder field uses a private reportGenerator interface (composed of auditBuilder and ReportComposer) that exposes only the 4 methods HybridGenerator actually calls (SetIncludeTunables, BuildAuditSection, BuildStandardReport, BuildComprehensiveReport). The public API (SetBuilder/GetBuilder) still accepts and returns the full ReportBuilder interface for backward compatibility, with GetBuilder using a two-value type assertion to recover the broad interface.

4.2 Report Types#

The MarkdownBuilder generates two primary report types:

Standard Report#

BuildStandardReport includes:

• Header with system information and metadata
• Table of contents
• System, network, security, and services sections
• System tunables

Comprehensive Report#

BuildComprehensiveReport includes all standard sections plus:

• VLANs and static routes
• IPsec VPN configuration
• OpenVPN configuration
• High Availability and CARP

4.3 Section Builders#

Each major configuration area has dedicated build methods:

• BuildSystemSection: System configuration, users, groups
• BuildNetworkSection: Network interfaces and configuration
• BuildSecuritySection: NAT, firewall rules, IDS configuration
• BuildServicesSection: DHCP, DNS, SNMP, NTP, load balancers
• BuildIPsecSection: IPsec VPN
• BuildOpenVPNSection: OpenVPN servers and clients
• BuildHASection: High Availability and CARP
• BuildAuditSection: Compliance audit results from ComplianceChecks

Audit Section Rendering#

The BuildAuditSection method in internal/converter/builder/builder.go renders compliance audit results from CommonDevice.ComplianceChecks:

• Returns empty string when ComplianceChecks is nil (safe to call unconditionally)
• Generates summary table with mode and total findings count
• Renders per-plugin compliance results with severity breakdowns
• Unified controls table: When Controls data is available for a plugin, renders a "Plugin Results" table with columns: Control ID, Title, Severity, Category, Status (PASS/FAIL). Replaces the legacy findings-only table.
• Failures-only filtering: When builder.failuresOnly is true, filters the unified controls table to show only FAIL rows. Displays "All controls compliant" message when all rows are filtered.
• Legacy fallback: When Controls is empty but Findings exist, renders the legacy findings table without Status column
• Finding partitioning: Partitions findings by Type field — findings with Type: "inventory" are rendered in a separate "Configuration Notes" section with a different table structure (Component, Title, Details columns) vs. security findings (Severity, Component, Title, Recommendation columns)
• Displays audit metadata in sorted order
• Uses helper functions: EscapePipeForMarkdown() for pipe-only escaping, TruncateString() for rune-aware exact-position truncation
• Plugin names and metadata keys iterated in sorted order via slices.Sorted(maps.Keys(...))

The corresponding WriteAuditSection method in internal/converter/builder/writer.go provides streaming output for the audit section.

4.4 Formatting Utilities#

The internal/converter/formatters package provides comprehensive formatting:

• FormatBool: Checkmarks (✓/✗)
• FormatBoolStatus: "Enabled"/"Disabled"
• FormatInterfacesAsLinks: Markdown links to sections
• EscapeTableContent: Markdown special character escaping
• TruncateDescription: Word-boundary-aware truncation
• FormatBytes: Human-readable byte formatting

4.5 Multi-Format Output#

The HybridGenerator in internal/converter/hybrid_generator.go provides unified output generation using handler dispatch via handlerForFormat() and the DefaultRegistry. Rather than using switch statements for format selection, the generator delegates to FormatHandler implementations registered in DefaultRegistry, which serves as the single source of truth for supported formats, aliases, file extensions, and validation.

Supported formats:

• Markdown: Via ReportBuilder, with audit section appended when ComplianceChecks is present
• JSON: Direct serialization, automatically includes ComplianceChecks via struct tags
• YAML: Direct serialization, automatically includes ComplianceChecks via struct tags
• Plain Text: Stripped markdown using StripMarkdownFormatting()
• HTML: Rendered via goldmark using RenderMarkdownToHTML()

The GenerateToWriter method enables memory-efficient streaming for large configurations.

4.5.1 FormatRegistry Pattern#

The FormatRegistry in internal/converter/registry.go provides centralized format handling through the DefaultRegistry singleton:

Core Architecture:

• FormatHandler interface: Defines Generate(), GenerateToWriter(), FileExtension(), and Aliases() methods
• DefaultRegistry: Package-level singleton pre-populated with all built-in format handlers (markdown, json, yaml, text, html)
• Handler implementations: Each format has a dedicated handler (markdownHandler, jsonHandler, yamlHandler, textHandler, htmlHandler) that delegates to the corresponding HybridGenerator methods

Key Features:

• Single source of truth: All format names, aliases, file extensions, and validation logic centralized in one location
• Alias resolution: Canonical() method resolves aliases (e.g., "md" → "markdown", "yml" → "yaml", "txt" → "text", "htm" → "html")
• Format validation: Get() method provides authoritative validation, returning ErrUnsupportedFormat for unknown formats
• Extensibility: Adding a new format requires only registering a FormatHandler in newDefaultRegistry() — all validation, completion, and dispatch automatically adopt it
• Panic-on-duplicate: Follows database/sql driver pattern with init-time panics for duplicate format names, aliases, or nil handlers

Integration Points:

• CLI commands (cmd/convert.go, cmd/shared_flags.go) use the registry for format validation, completions, and file extension mapping
• Options.Format.Validate() delegates to DefaultRegistry.Get()
• processor.Transform() uses DefaultRegistry.Canonical() for alias resolution
• Config validation derives ValidFormats from DefaultRegistry.ValidFormats()

See AGENTS.md §5.9b for detailed implementation guidance.

Audit Section Integration#

The HybridGenerator integrates audit sections based on output format:

• Markdown: generateMarkdown and generateMarkdownToWriter append audit sections via BuildAuditSection() / WriteAuditSection() when ComplianceChecks is present
• JSON/YAML: Audit data serialized automatically through CommonDevice.ComplianceChecks field with no format-specific code needed

4.6 Data Enrichment#

The prepareForExport function in internal/converter/enrichment.go enriches CommonDevice before JSON/YAML export:

• Statistics: Interface counts, rule stats, user/group counts (computed via analysis.ComputeStatistics)
• Analysis: Security findings, configuration insights (computed via analysis.ComputeAnalysis)
• SecurityAssessment: Computed security scores
• Redaction: Replaces sensitive data (passwords, keys, SNMP community strings) with [REDACTED]
• ComplianceChecks: Pass-through field populated by audit handler (cmd/audit_handler.go), not modified during enrichment

Shared Analysis Package#

The converter delegates to internal/analysis/ for statistics computation and detection logic:

• ComputeStatistics(): Computes interface, rule, user, group, service, gateway, and system statistics
• ComputeAnalysis(): Orchestrates all detection functions and returns a complete Analysis
• DetectDeadRules(): Identifies unreachable rules (after block-all) and duplicates
• DetectUnusedInterfaces(): Finds enabled interfaces not used in rules or services
• DetectSecurityIssues(): Detects insecure configurations (HTTP web GUI, default SNMP community, overly permissive WAN rules)
• DetectPerformanceIssues(): Identifies performance impacts (disabled offloading, high rule counts)
• DetectConsistency(): Validates gateway formats, DHCP-interface consistency, user-group relationships
• RulesEquivalent(): Compares firewall rules for functional equivalence (including Disabled field)

The shared internal/analysis/ package was created in PR #409 to eliminate duplicated logic between the converter enrichment pipeline and the processor's analyze phase. Both subsystems now import and delegate to the same detection functions for consistent behavior.

5. Audit Plugins#

5.1 Audit Plugin Architecture#

The audit plugin system in internal/compliance/interfaces.go provides a plugin-based compliance checking framework for auditing firewall configurations against industry security standards.

Core Interface#

All compliance plugins implement the Plugin interface:

type Plugin interface {
    Name() string
    Version() string
    Description() string
    RunChecks(device *common.CommonDevice) []Finding
    GetControls() []Control
    GetControlByID(id string) (*Control, error)
    ValidateConfiguration() error
}

Plugin Execution Model: Each plugin's RunChecks() method is executed within a panic recovery boundary to provide isolation between plugins and the audit engine. This ensures that a misbehaving plugin (especially dynamically-loaded) cannot crash the entire audit process. The audit engine requires a *slog.Logger parameter to log panic events.

Key data structures:

• Control: Compliance control with ID, title, severity, rationale, remediation. Controls can have Type: "inventory" for informational observations that do not affect compliance status (excluded from EvaluatedControlIDs).
• Finding: The canonical Finding struct is defined in internal/analysis/finding.go, with standardized fields including type (category), severity level, title, description, recommendation, component, and reference. The compliance.Finding type is now a type alias to analysis.Finding, ensuring consistency across all plugins. The Severity field uses the analysis.Severity type with values: "critical", "high", "medium", "low", "info"

Inventory Controls vs Info-Severity Findings#

The architecture distinguishes between inventory controls and info-severity findings:

• Inventory controls (Type: "inventory"): Informational observations that are excluded from compliance evaluation entirely. Their control IDs never appear in EvaluatedControlIDs and do not participate in the compliance map (no PASS/FAIL status). Example: FIREWALL-062 (DHCP Scope Inventory), FIREWALL-063 (Active Interface Summary). Inventory findings render in a separate "Configuration Notes" section.

• Info-severity findings (Severity: "info"): Participate in compliance evaluation normally — they can PASS or FAIL. Severity affects only presentation priority (summary counts, sort order), NOT compliance gating. The compliance flip in RunComplianceChecks applies to all severities equally. Example: FIREWALL-003 (Message of the Day) was reclassified from "low" to "info" in PR #510 to reflect its informational nature, but still participates in compliance evaluation.

See GOTCHAS.md §2.4 for the rationale behind this design decision.

Plugin Registry#

The PluginRegistry in internal/audit/plugin.go manages plugins:

• Thread-safe registration with sync.RWMutex
• Plugin validation on registration
• Dynamic .so plugin loading support via LoadDynamicPlugins(ctx, dir, explicitDir, logger) (LoadResult, error)
• Global registry singleton for convenience
• Injectable pluginLoader pluginLoaderFunc field for testing without real .so files
• newPluginRegistryWithLoader(loader) constructor for dependency injection in tests

Thread-Safety and Initialization#

The global PluginRegistry singleton uses sync.Once for initialization via GetGlobalRegistry(). This provides happens-before memory guarantees per the Go memory model: all writes within the initialization function are visible to every goroutine that subsequently calls GetGlobalRegistry() without additional synchronization.

Lifecycle contract: Plugins should be registered at startup (initialization phase) via RegisterGlobalPlugin() before concurrent access begins. While the internal sync.RWMutex makes concurrent reads and writes technically safe, the intended usage pattern is register-at-startup, read-during-operation.

Plugin Registry Consolidation#

NewPluginManager(logger, reg) accepts a *PluginRegistry parameter for shared or isolated plugin state. Pass a shared registry when multiple managers or subsystems must observe the same plugin set. Pass nil to allocate a fresh private registry (appropriate default for short-lived programmatic callers).

This consolidates plugin registration to a single path: the registry supplied to NewPluginManager is populated by InitializePlugins() and read by ListAvailablePlugins() / RunComplianceAudit().

The previous architectural pattern with two independent registries (global singleton via GetGlobalRegistry() / RegisterGlobalPlugin() / GetGlobalPlugin() / ListGlobalPlugins() vs. PluginManager's private instance) was resolved in TODO #143. The global registry functions are deprecated and scheduled for removal in v2.0. Production code must use NewPluginManager with an explicit *PluginRegistry.

Dynamic Plugin Loading#

The LoadDynamicPlugins method returns (LoadResult, error) to report both successful and failed plugin loads:

• LoadResult: Contains Loaded count and Failures []PluginLoadError slice
• LoadResult.Failed(): Method (not field) returning len(Failures) for convenience
• PluginLoadError: Per-plugin error type implementing the error interface, with Name (filename) and Err fields
• Aggregate errors: Multiple load failures are joined via errors.Join

explicitDir parameter: Controls error handling when the plugin directory does not exist:

• explicitDir=false: Silently skip missing directory (Debug log), used for default/optional paths
• explicitDir=true: Return error for missing directory (fail-fast), used when user explicitly specifies --plugin-dir

pluginLoaderFunc abstraction: The registry injects plugin loading logic via the pluginLoaderFunc type:

• defaultPluginLoader: Production loader opening .so files via plugin.Open
• newPluginRegistryWithLoader(loader): Test constructor accepting custom loaders for deterministic testing without real .so files
• Nil plugin guard: Loaders returning (nil, nil) are treated as failures, preventing panics

Two-Phase CLI Validation Pattern#

The audit command validation in cmd/audit.go follows a two-phase validation pattern to support dynamic plugins:

Phase 1: PreRunE (early static validation)

• Validates audit mode against known modes (blue, red)
• Enforces --plugins/--mode coupling (plugins only work with blue mode)
• Rejects multi-file + --output conflicts
• Validates shared output flags (format, wrap, section)
• Does NOT validate plugin names — plugin validation is deferred to phase 2

Phase 2: Post-initialization (runtime validation)

• ValidateModeConfig() in internal/audit/mode_controller.go validates SelectedPlugins against the live registry after InitializePlugins() populates it with both built-in and dynamic plugins
• Returns ErrPluginNotFound for unrecognized plugin names
• Called by GenerateReport() during the RunE phase via handleAuditMode()

Rationale: Plugin name validation requires the registry to be populated with both built-in and dynamically-loaded plugins. Since PreRunE executes before InitializePlugins() loads dynamic .so plugins from --plugin-dir, validating plugin names in PreRunE would reject valid dynamic plugin names before they are loaded. Deferring validation to post-initialization allows dynamic plugins to be discovered, registered, and validated against the live registry.

Shell completions: The ValidAuditPlugins() function uses registryPluginNames() to source plugin names from a temporary PluginManager initialized with built-in plugins, mirroring the ValidDeviceTypes registry-driven pattern. This ensures shell completions stay synchronized with the live registry. Dynamic plugins are not included in completions because shell completion runs before --plugin-dir is processed.

See docs/solutions/logic-errors/cli-prerun-validation-timing-dynamic-plugins.md for comprehensive implementation details and prevention strategies.

5.2 Built-in Audit Plugins#

opnDossier includes three built-in compliance plugins:

Firewall Plugin (internal/plugins/firewall/)#

The Firewall plugin provides security controls (FIREWALL-001 through FIREWALL-063):

Security Controls (standard compliance evaluation):

• FIREWALL-001: SSH Warning Banner Configuration (medium)
• FIREWALL-002: Auto Configuration Backup (medium)
• FIREWALL-003: Message of the Day (info)
• FIREWALL-004: Hostname Configuration (low)
• FIREWALL-005: DNS Server Configuration (medium)
• FIREWALL-006: IPv6 Disablement (medium)
• FIREWALL-007: DNS Rebind Check (low)
• FIREWALL-008: HTTPS Web Management (high)
• Additional controls through FIREWALL-061

Inventory Controls (Type: "inventory", excluded from compliance evaluation):

• FIREWALL-062: DHCP Scope Inventory — reports configured DHCP scopes and interfaces
• FIREWALL-063: Active Interface Summary — reports enabled interfaces and types

Uses a three-state check pattern (checkResult{Result, Known}) to prevent false positives when data is insufficient. All findings derive their Severity from control metadata via the controlSeverity() helper, ensuring consistency with control definitions.

SANS Plugin (internal/plugins/sans/)#

The SANS plugin implements SANS Firewall Checklist (SANS-FW-001 through SANS-FW-004):

• SANS-FW-001: Default Deny Policy (high)
• SANS-FW-002: Explicit Rule Configuration (medium)
• SANS-FW-003: Network Zone Separation (high)
• SANS-FW-004: Comprehensive Logging (medium)

All findings derive their Severity from control metadata via the controlSeverity() helper.

STIG Plugin (internal/plugins/stig/)#

The STIG plugin implements Security Technical Implementation Guide (V-206xxx series):

• V-206694: Default deny policy (high)
• V-206674: Specific packet filtering (high)
• V-206690: Disable unnecessary services (medium)
• V-206682: Comprehensive logging (medium)

Most mature implementation with sophisticated checks:

• Overly permissive rules (any/any, broad ranges)
• Unnecessary services (insecure SNMP, DNSSEC stripping)
• Logging configuration with four-state enum

All findings derive their Severity from control metadata via the controlSeverity() helper.

5.3 Compliance Checking Workflow#

The compliance check execution in internal/audit/plugin.go:

1. Validate device is non-nil
2. Deduplicate plugin names (case-insensitive)
3. For each selected plugin:
   • Call plugin.RunChecks(device) to get findings, wrapped in panic recovery
   • Panic Recovery: Each RunChecks() call is wrapped in defer recover() to isolate plugin failures. If a plugin panics, it is logged via structured logging and retained in the result with zero findings
   • Invariant: Every selected plugin always appears in the final results (PluginFindings, PluginInfo, Compliance, summary maps), even if it panics. This ensures downstream consumers can see all requested plugins were evaluated
   • Normalize findings: Call deriveSeverityFromControl() to populate missing Finding.Severity values from control metadata
   • Return errors if control references cannot be resolved
   • Store findings in both aggregate (Findings) and per-plugin (PluginFindings) maps
   • Track plugin metadata (name, version, deep-cloned controls)
   • Initialize compliance tracking (all evaluated controls default to compliant)
   • Mark controls as non-compliant based on findings: Inventory findings (Type: "inventory") are explicitly skipped during the compliance flip — their referenced controls are not in EvaluatedControlIDs and do not participate in pass/fail status
4. Calculate summary statistics with per-plugin granularity:
   • Aggregate findings by severity using countSeverities() helper
   • Per-plugin severity breakdown via computePerPluginSummary()
   • Compliance percentages per plugin
5. Return ComplianceResult with findings, per-plugin findings map, compliance map, summary, and plugin info

As of PR #391, the Finding and Severity types have been unified across the codebase. The canonical types are defined in internal/analysis/finding.go, with compliance.Finding and processor.Finding implemented as type aliases. This eliminates previous duplication across audit, compliance, and processor packages, ensuring consistent finding representation throughout opnDossier.

5.4 Plugin Manager#

The PluginManager in internal/audit/plugin_manager.go handles plugin lifecycle:

• SetPluginDir(dir, explicit): Configures dynamic plugin directory before initialization. The explicit flag controls error handling for missing directories (true = fail-fast for user-specified paths, false = silent skip for default paths).
• InitializePlugins(): Registers built-in plugins (STIG, SANS, Firewall) and loads dynamic .so plugins if SetPluginDir was called. Dynamic plugin load failures are non-fatal — callers must check GetLoadResult() after initialization to detect per-plugin failures.
• GetLoadResult(): Returns LoadResult from the most recent LoadDynamicPlugins call, containing counts of loaded/failed plugins and per-plugin failure details.
• RunComplianceAudit(): Executes compliance checks for selected plugins
• ListAvailablePlugins(): Returns metadata about registered plugins
• GetPluginStatistics(): Provides plugin usage statistics

5.5 Audit Execution Paths#

opnDossier provides two distinct CLI entry points for running security audits and compliance checks, both using the same underlying audit/compliance plugin architecture:

Dedicated Audit Command#

The audit command in cmd/audit.go provides a focused interface for security and compliance assessments:

Command Structure:

• cmd/audit.go: Command definition, flag variables, init(), PreRunE validation, runAudit() orchestration, generateAuditOutput() parsing/audit execution
• cmd/audit_output.go: Output emission logic (emitAuditResult()), path derivation (deriveAuditOutputPath()), tilde-based segment escaping (escapePathSegment())

Key Features:

• Multi-file audit support: Processes multiple configuration files concurrently with semaphore-based concurrency limiting
• Auto-naming for multi-file runs: Derives unique output paths using tilde-based escaping to prevent filename collisions (e.g., some~upath_config-audit.md)
• Glamour-rendered terminal output: Markdown reports are styled for terminal display when written to stdout; file output and non-markdown formats remain raw
• Plugin mode coupling: --plugins flag only works with --mode blue (enforced in PreRunE); red mode ignores compliance plugins
• Default plugin behavior: Bare --mode blue runs all available plugins when no --plugins specified
• --failures-only flag: Filters blue mode compliance tables to show only non-compliant controls. Only valid with blue mode and markdown format (enforced in PreRunE).

Validation:

• PreRunE validation: Audit mode, plugin names, multi-file + --output rejection, plugin-mode coupling, --failures-only restrictions
• validateOutputFlags(): Shared flag validation (format, wrap, section) delegated to cmd/shared_flags.go

Convert Command Audit Integration#

The convert command no longer supports audit mode. For security audits and compliance checking, use the dedicated audit command instead.

Shared Audit Handler#

The handleAuditMode function in cmd/audit_handler.go orchestrates audit report generation:

1. Plugin Manager Setup: Creates PluginManager and calls SetPluginDir(auditOpts.PluginDir, auditOpts.ExplicitPluginDir) if --plugin-dir flag was provided
2. Plugin Initialization: Calls InitializePlugins() to register built-in plugins and load dynamic .so plugins
3. Load Failure Surfacing: Checks GetLoadResult() after initialization and logs warnings for any failed dynamic plugin loads with filenames
4. Compliance Execution: Runs compliance checks via PluginManager.RunComplianceAudit(), producing an audit.Report
5. Result Mapping: Calls mapAuditReportToComplianceResults() to convert audit.Report → common.ComplianceResults, mapping:
   • Top-level findings from audit.Finding to common.ComplianceFinding
   • Per-plugin results from audit.ComplianceResult to common.PluginComplianceResult
   • Controls from compliance.Control to common.ComplianceControl
   • Summary statistics with compliant/non-compliant counts
6. Device Population: Assigns ComplianceResults to device.ComplianceChecks
7. Report Delegation: Calls generateWithProgrammaticGenerator() for format-agnostic report generation

The handler contains no format-specific rendering code. All markdown generation happens in internal/converter/builder/, and JSON/YAML serialization occurs automatically via struct tags on CommonDevice.ComplianceChecks.

Shared Flag Validation#

The validateOutputFlags() function in cmd/shared_flags.go validates format/wrap/section flag combinations shared between audit and convert commands:

• Mutual exclusivity: Rejects --no-wrap + --wrap combinations
• Format validation: Checks format against converter.DefaultRegistry.ValidFormatsWithAliases()
• Section compatibility warnings: Warns when --section is used with JSON/YAML (sections ignored)
• Wrap width validation: Warns for out-of-range values, errors for invalid values

Command-specific validation (audit mode, plugin names) remains in each command's PreRunE.

Multi-File Output Path Derivation#

The deriveAuditOutputPath() function in cmd/audit_output.go computes unique output filenames for multi-file audit runs:

• Tilde-based escaping: Uses ~~ for literal tildes, ~u for underscores, freeing _ as an unambiguous segment separator
• Collision prevention: Avoids double-underscore ambiguity (e.g., a_/b vs a/_b both producing a___b)
• Absolute path handling: Prefixes absolute paths with ~a marker
• Bare filenames: Simple names for files without directory components (e.g., config-audit.md)

CLI Integration#

The --plugin-dir flag (registered in cmd/shared_flags.go) specifies the directory containing dynamic .so compliance plugins. The flag value is wired through audit.Options.PluginDir and audit.Options.ExplicitPluginDir, with the latter set to true when the user explicitly provides the flag. This enables fail-fast behavior for user-specified directories while allowing default/optional paths to silently skip.

5.6 Mode Controller#

The ModeController in internal/audit/mode_controller.go orchestrates report generation with two supported modes:

• Blue mode: Defensive audit with compliance checks and security findings (default)
• Red mode: Attack surface analysis, exposed services, weak configurations

The Finding struct in mode_controller.go embeds analysis.Finding for common fields (Title, Severity, Description, Recommendation, Component, Tags) and adds audit-specific extensions (AttackSurface, ExploitNotes, Control).

5.7 Validation Patterns#

The framework implements several validation patterns:

1. Three-state checks: Returns {Result, Known} to distinguish non-compliant from "cannot determine"
2. Nil-safe checks: All helpers validate device is non-nil before accessing fields
3. Conservative approach: When uncertain, assumes compliance rather than false positives
4. Plugin validation: All plugins pass ValidateConfiguration() on registration
5. Standardized findings: Uniform Finding structure for consistent processing

6. Data Flow Architecture#

The complete data flow through opnDossier follows this pattern:

Processing Pipeline#

1. Input: Firewall XML configuration file (OPNsense or pfSense)
2. Factory: Auto-detects device type by peeking at root XML element (<opnsense> or <pfsense>). The CreateDevice method returns (*CommonDevice, []ConversionWarning, error) to propagate warnings through the pipeline.
3. XML Parser: Device-specific parsers (OPNsense, pfSense) stream XML tokens into schema DTOs with security protections via shared NewSecureXMLDecoder
4. Converter: Device-specific converters (OPNsense, pfSense) transform XML DTOs to CommonDevice model, accumulating non-fatal warnings for incomplete or problematic data
5. Analysis (optional):
   • Audit plugins analyze CommonDevice for compliance violations
   • Data enrichment computes statistics
6. Report Generation:
   • MarkdownBuilder generates sections from CommonDevice with dynamic platform names via DeviceType.DisplayName()
   • HybridGenerator produces output in selected format
7. Output: Human-readable documentation in markdown, JSON, YAML, HTML, or plain text
8. Warning Handling: CLI commands (convert, diff, display, validate) log conversion warnings via structured logging (ctxLogger.Warn), suppressed in quiet mode

7. Key Architectural Benefits#

1. Security-First Design: XXE prevention, XML bomb protection, streaming processing, offline operation
2. Platform Extensibility: Clean separation between XML DTOs and CommonDevice enables multi-platform support
3. Plugin Architecture: Extensible compliance checking via interface-based plugins with dynamic loading
4. Type Safety: Compile-time validation through programmatic generation, strong Go typing, and typed enum constants for firewall rules, NAT modes, DHCP configuration, network protocols, and VIP modes. This refactoring eliminated magic strings across 70 files, reducing runtime errors and improving code maintainability.
5. Memory Efficiency: Streaming parser and writer support large configurations without memory exhaustion
6. Separation of Concerns: Clear boundaries between parsing (schema), modeling (common), analysis (audit), and presentation (converter)
7. Conservative Validation: Three-state check pattern prevents false positives in compliance checking
8. Multi-Format Output: Single data model supports markdown, JSON, YAML, HTML, and plain text
9. Plugin Isolation: Panic recovery in compliance checking prevents individual plugin failures from crashing the audit process

References#

• Architecture Documentation
• Architecture Review
• XML Parser
• CommonDevice Model
• Report Builder
• Compliance Plugins
• Audit Command
• Audit Output Emission
• Shared Flag Validation