Dosu Logo
Home
Documents
Technical Deep-Dive: Architecture, Internals & Roadmap
Technical Deep-Dive: Architecture, Internals & Roadmap
Type
External
Status
Published
Created
Mar 8, 2026
Updated
Apr 3, 2026
Updated by
Dosu Bot

Project Overview#

AttributeValue
LanguageGo 1.26 (toolchain go1.26.0)
LicenseApache-2.0
Latest Releasev1.2.1 (2026-02-12)
Repository[EvilBit-Labs/opnDossier](https://github.com/EvilBit-Labs/opnDossier)
Open Issues50
Stars / Forks15 / 4
opnDossier is a CLI tool for converting OPNsense firewall configuration files (`config.xml`) into human-readable documentation, structured exports (JSON/YAML), and compliance audit reports. It is designed for offline-first operation in airgapped environments with zero external dependencies. --- # Codebase Metrics
MetricValue
Production code (`internal/`)\~28,200 lines
Test code (`internal/`)\~55,500 lines
CLI layer (`cmd/`)\~4,500 lines production + \~4,300 lines test
Internal packages24
Schema structs (XML DTOs)\~96 types across 14 files
CommonDevice model types\~87 types across 15 files
Golden test files27
CI workflows7 (benchmarks, ci, copilot-setup, docs, release, sbom, scorecard)
Test-to-production ratio\~1.9:1 (tests significantly outnumber production code)
--- # Architecture Overview ## Design Principles - **Offline-First**: Zero external dependencies, complete airgap compatibility - **Operator-Focused**: Built for network administrators and security professionals - **Framework-First**: Leverages established Go libraries (Cobra, Charm ecosystem) - **Security-First**: No telemetry, input validation, secure processing, XXE prevention - **Multi-Device Extensible**: Clean separation between XML DTOs and platform-agnostic domain model ## Technology Stack
ComponentTechnology
CLI Framework`spf13/cobra` v1.8.0
CLI Enhancement`charmbracelet/fang` (styled help, errors)
Configuration`spf13/viper` (YAML, env vars, flags)
Terminal Styling`charmbracelet/lipgloss`
Markdown Rendering`charmbracelet/glamour`
Markdown Generation`nao1215/markdown` (programmatic builder)
Logging`charmbracelet/log` (structured)
Data Formats`encoding/xml`, `encoding/json`, `gopkg.in/yaml.v3`
TestingGo `testing` • `sebdah/goldie/v2` (golden files)
Task Runner`just` (Justfile)
ReleaseGoReleaser v2 (multi-platform)
--- # CLI Commands
CommandPurpose
`convert [file ...]`Parse config.xml and export to markdown, JSON, or YAML
`display [file]`Render config as styled markdown in terminal
`validate [file ...]`Validate OPNsense configuration structure
`sanitize [file]`Redact sensitive data from config files
`diff `Compare two configuration files
`config init`Initialize opnDossier configuration file
`config show`Display current configuration
`config validate [file]`Validate opnDossier config file
`completion [shell]`Generate shell completion scripts
`man [dir]`Generate man pages
`version`Print version information
--- # Two-Stage Parsing Architecture The core data pipeline separates XML-specific concerns from domain logic, enabling future multi-device support. ```mermaid graph TD A["config.xml"] --> B["cfgparser/xml.go"] B --> C["schema.OpnSenseDocument
(XML DTOs)"] C --> D["ParserFactory
(auto-detect device type)"] D --> E["opnsense/converter.go"] E --> F["common.CommonDevice
(platform-agnostic)"] F --> G["prepareForExport()
(enrichment)"] G --> H["JSON / YAML / Markdown
Output"] ``` ## Stage 1: XML Parsing **Package**: `internal/cfgparser/` The streaming XML parser reads OPNsense `config.xml` into `schema.OpnSenseDocument` DTOs. Key features: - **Security protections**: XXE prevention, XML bomb protection - **Charset conversion**: UTF-8, US-ASCII, ISO-8859-1, Windows-1252 - **Streaming processing**: Memory-efficient handling of large configurations - **Switch-case routing**: Each top-level XML element dispatched by tag name in `xml.go` ## Stage 2: Domain Conversion **Package**: `internal/model/opnsense/` The converter transforms XML DTOs into the platform-agnostic `CommonDevice` model: - `converter.go` -- core conversion and system fields - `converter_network.go` -- bridges, PPPs, GIFs, GREs, LAGGs, VIPs, interface groups - `converter_security.go` -- certificates, CAs, packages - `converter_services.go` -- DHCP, DNS, VPN, routing, etc. ## ParserFactory **File**: `internal/model/factory.go` `ParserFactory.CreateDevice()` auto-detects the device type by inspecting the root XML element. The `--device-type` flag bypasses auto-detection. Currently only OPNsense is implemented; the interface is designed for extension. --- # Data Model Layer ## Three-Layer Architecture
LayerPackagePurpose
XML DTOs`internal/schema/`Mirrors OPNsense config.xml structure with `xml:""` tags. \~96 struct types.
Domain Model`internal/model/common/`Platform-agnostic `CommonDevice` with JSON/YAML tags. \~87 types. No XML tags.
Re-export Seam`internal/model/`\~93 type aliases bridging schema and common. Temporary; targeted for removal in v1.3.0.
## CommonDevice Structure The `CommonDevice` model organizes data into three categories: **Core Configuration** (concrete types, populated during parsing): - System, Interfaces, VLANs, Bridges, LAGGs, VirtualIPs, GIFs, GREs, PPPs - Firewall Rules, NAT (inbound + outbound), VPN (OpenVPN, WireGuard, IPsec) - Routing (gateways, static routes), Services (DHCP, DNS, NTP, SNMP) - Security (certificates, CAs, IDS/IPS), Users, Groups, Packages **Optional Features** (pointer types for non-universal features): - IDS/IPS (Suricata), Monit, Netflow/IPFIX, Traffic shaper, Captive portal, Cron, Kea DHCP **Enrichment Fields** (pointer types, populated by `prepareForExport()`): - `*Statistics`, `*Analysis`, `*SecurityAssessment`, `*PerformanceMetrics`, `*ComplianceChecks` ## XML Type Patterns
OPNsense PatternGo TypeExample
Presence-based boolean (`isset()` in PHP)`BoolFlag```, ``, ``, ``
Value-based boolean (`== "1"` in PHP)`string``1`, `1`
Presence with value access`*string``` in Source/Destination
Container + children`[]ChildType` in container struct`VLANs` container with `[]VLAN` slice
--- # Enrichment Pipeline **File**: `internal/converter/enrichment.go` The `prepareForExport()` function is the single gate for all JSON/YAML exports. It enriches a shallow copy of `CommonDevice` with computed analytics: ```mermaid graph TD A["CommonDevice
(from converter)"] --> B["prepareForExport()"] B --> C["computeStatistics()"] B --> D["computeAnalysis()"] B --> E["computeSecurityAssessment()"] B --> F["computePerformanceMetrics()"] C --> G["Enriched CommonDevice"] D --> G E --> G F --> G G -->|"if redact=true"| H["redactSensitiveFields()"] H --> I["Final Export"] G -->|"if redact=false"| I ``` **Key design constraints**: - **Immutability**: Creates shallow copies with selective deep-copying before redaction - **Circular dependency avoidance**: Cannot import `internal/processor` -- analysis logic is mirrored, not shared. This is a deliberate architectural trade-off (package independence over code deduplication) - **Redaction flow**: CLI flag -\> `Options.Redact` -\> `prepareForExport(data, redact)` -\> conditional redaction - **Statistics synchronization**: Adding fields to `common.Statistics` requires updating three places (struct definition, `computeStatistics()`, and `computeTotalConfigItems()`) --- # Compliance Plugin System ## Architecture ```mermaid graph TD A["CLI audit command"] --> B["PluginManager"] B --> C["PluginRegistry
(thread-safe, sync.RWMutex)"] C --> D["Firewall Plugin"] C --> E["SANS Plugin"] C --> F["STIG Plugin"] D --> G["RunChecks(*CommonDevice)"] E --> G F --> G G --> H["[]Finding"] H --> I["Compliance Report"] ``` ## Core Components
ComponentFilePurpose
`compliance.Plugin` interface`internal/compliance/interfaces.go`Standardized contract: Name, Version, Description, RunChecks, GetControls
`PluginRegistry``internal/audit/plugin.go`Thread-safe registry with duplicate detection
`PluginManager``internal/audit/plugin_manager.go`Plugin lifecycle: load, validate, execute, report
Firewall plugin`internal/plugins/firewall/`OPNsense-specific firewall checks
SANS plugin`internal/plugins/sans/`SANS security best practices (SANS-XXX controls)
STIG plugin`internal/plugins/stig/`STIG compliance controls (STIG-V-XXXXXX)
## Audit Report Modes
ModeAudienceFocusStatus
StandardOperationsNeutral configuration documentationImplemented
BlueBlue TeamDefensive analysis, findings, recommendationsPlanned (issue #281)
RedRed TeamAttack surface enumeration, pivot analysisPlanned (issue #281)
## Known Audit Issues (Tracked) - **#310**: `calculateSummary` severity counting is non-functional -- `Finding.Type` is always "compliance", not a severity level - **#309**: No panic recovery around plugin `RunChecks()` calls -- dynamic plugins can crash entire audit - **#311**: Dynamic plugin load failures are silently swallowed --- # Report Generation System Reports are generated programmatically using `builder.MarkdownBuilder` (backed by `nao1215/markdown`). There is **no template system** -- all markdown is constructed via Go method calls for type safety, performance, and testability. ## Builder Architecture
ComponentLocationPurpose
`MarkdownBuilder``internal/converter/builder/`Fluent API for markdown construction
`SectionWriter` interface`internal/converter/builder/writer.go`Streaming `io.Writer` support
`Generator` interface`internal/converter/`Format-agnostic generation contract
`HybridGenerator``internal/converter/hybrid_generator.go`Multi-format dispatcher (markdown, JSON, YAML)
Formatters`internal/converter/formatters/`Format-specific serialization logic
--- # Sanitizer & Redaction The sanitizer uses a rule-based engine for sensitive data redaction: - **Field-pattern rules**: Match by field name (e.g., `password`, `secret`, `otp_seed`) - **Value-detector rules**: Match by value content (e.g., IP addresses with `IsIP` detector) - **Deterministic mapping**: Fresh `RuleEngine` creates fresh `Mapper` -- first private IP always maps to `10.0.0.1`, first hostname to `host-001.example.com` - **`--redact`**** flag**: Opt-in redaction for `convert` and `display` commands --- # Configuration Diff Engine **Package**: `internal/diff/` The `diff` command compares two OPNsense configurations and outputs a structured change report. Uses comparison functions with nil-safe patterns (`both-nil` -\> nil, `one-nil` -\> added/removed) and `slices.Equal()` for slice fields. --- # Package Dependency Structure The internal packages follow a layered architecture with clear dependency direction: ```mermaid graph TD CMD["cmd/"] --> MODEL["model/"] CMD --> PROCESSOR["processor/"] CMD --> CONVERTER["converter/"] CMD --> CFGPARSER["cfgparser/"] CMD --> VALIDATOR["validator/"] CMD --> DISPLAY["display/"] CMD --> AUDIT["audit/"] CMD --> CONFIG["config/"] CMD --> SANITIZER["sanitizer/"] CMD --> DIFF["diff/"] CMD --> PROGRESS["progress/"] CMD --> EXPORT["export/"] PROCESSOR --> MODEL CONVERTER --> MODEL CONVERTER --> BUILDER["converter/builder/"] AUDIT --> COMPLIANCE["compliance/"] AUDIT --> PLUGINS["plugins/"] PLUGINS --> COMPLIANCE PLUGINS --> MODEL CFGPARSER --> SCHEMA["schema/"] MODEL --> SCHEMA MODEL --> COMMON["model/common/"] MODEL --> OPNSENSE["model/opnsense/"] OPNSENSE --> SCHEMA OPNSENSE --> COMMON ``` **Key constraint**: `converter/` and `processor/` both depend on `model/common/` but cannot depend on each other (circular dependency). Analysis logic is deliberately mirrored in both packages. --- # CI/CD Pipeline
WorkflowFilePurpose
CI`ci.yml`Lint (golangci-lint), format (gofumpt), tests, CodeQL, Grype
Benchmarks`benchmarks.yml`Performance benchmarks (non-blocking, `continue-on-error: true`)
Release`release.yml`GoReleaser: multi-platform builds, changelogs, signing
SBOM`sbom.yml`Software Bill of Materials generation
Scorecard`scorecard.yml`OSSF Scorecard security assessment
Docs`docs.yml`MkDocs Material documentation build
Copilot Setup`copilot-setup-steps.yml`GitHub Copilot workspace configuration
## Quality Gate `just ci-check` runs the full CI-equivalent locally: pre-commit hooks + `gofumpt` formatting check + `golangci-lint` + all tests. **Tasks are not complete until ****`ci-check`**** passes.** --- # Open Issues & Roadmap ## Priority: Critical (Architecture)
#TitleCategory
322Move audit report rendering from cmd to converter/builder layerrefactor
321Extract shared analysis package to eliminate enrichment mirrorrefactor
320Split `report.go` into focused filesrefactor
319Split `opnsense.go` validator into domain-specific filesrefactor
318Split `builder.go` into domain-specific filesrefactor
## Priority: High
#TitleCategory
325Add FormatRegistry to replace scattered format routingrefactor
324Extract Severity type to shared packagerefactor
323Split ReportBuilder interface (18 methods) into focused interfacesrefactor
310Severity breakdown missing in audit reportsbug
309Add panic recovery around plugin RunChecks()enhancement
286Replace O(n\^2) duplicate rule detection with O(n) hash-basedperformance
282Eliminate re-export layer (93 type aliases)refactor
280Unify Finding types across compliance, processor, auditrefactor
281Disable blue/red audit modes until full implementationenhancement
154Complete template system removal for v2.0refactor
157Create shared evilbitlabs-network-model Go modulearchitecture
## Public API Roadmap
MilestoneIssueScope
v1.3.0#301Move types to `pkg/`: `pkg/model/`, `pkg/schema/opnsense/`, `pkg/parser/`, `pkg/parser/opnsense/`. Eliminates re-export layer (\~93 aliases, \~110 files need import updates).
v2.0.0#302Parser registry: replace hardcoded switch in ParserFactory with `Register()` pattern. External device types via `init()` functions.
## Milestones
MilestoneProgressFocus
v1.3.018 closed / 27 openUX improvements, documentation, alternative output formats, public `pkg/` API
v1.4.00 closed / 3 openExtract common model libraries into reusable packages
v2.0.017 closed / 14 openMajor architectural changes: programmatic generation, API changes, parser registry
v2.1.00 closed / 4 openPlatform expansion: pfSense, Cisco ASA, Fortinet multi-vendor support
## Multi-Device Support Roadmap
DeviceIssueStatus
OPNsense--Implemented (current)
pfSense#197Planned (v1.3.0+, community-requested)
Cisco ASA#198Planned (post-v2.0.0, enterprise focus)
Fortinet FortiGate#199Planned (post-v2.0.0)
## v2.0 Feature Epics
#FeatureLabels
213Web UI with Local Server Modepremium, v2.0
211TUI: Interactive Terminal Interfaceblue-team, v2.0
212File System Watch Mode for Continuous Monitoringv2.0
209SARIF Export for CI/CD Security Integrationpremium, compliance
208SIEM Export Formats (CEF/LEEF/JSONL)security, v2.0
207Professional PDF Report Generationpremium, v2.0
205Custom Rule Engine for Organization Policiescompliance, v2.0
204PCI-DSS Requirement 1 Firewall Checkscompliance, v2.0
201Configuration Drift Detection & Baseline Managementv2.0
--- # Recent Development Activity Most recent closed issues (as of March 2026):
#TitleClosed
316Implement opt-in `--redact` flag for convert/display2026-02-26
296Implement firewall compliance check helpers2026-02-23
295Restore semantic validation for CommonDevice2026-02-23
294Add nil guards for nested WireGuard structs2026-02-21
287Prevent shared backing array mutations in normalize()2026-02-26
284Populate unpopulated CommonDevice fields (Bridges, PPPs, GIFs, etc.)2026-02-26
272Additional tags for sanitize command (Secrets & Topology)2026-02-28
271`otp_seed` not redacted by sanitize2026-02-20
266Compliance findings not properly aggregated by plugin2026-02-23
263Architecture review: tech debt across model, enrichment, display, logging2026-02-20
--- # Known Technical Debt ## Architectural Constraints - **Circular dependency**: `converter/enrichment.go` and `processor/` both need analysis logic but cannot import each other. Analysis is mirrored in both packages. Issue #321 targets extracting a shared `internal/analysis/` package. - **Re-export layer**: 93 type aliases in `internal/model/` re-export types from `internal/schema/` and `internal/model/common/`. Temporary seam for multi-device migration; removal planned in v1.3.0 (issue #282). - **ReportBuilder interface**: Currently 18 methods in a single interface. Issue #323 targets splitting into focused interfaces. - **Sequential plugin execution**: No concurrent plugin execution; potential bottleneck for large configurations with many compliance checks. ## Code Quality Items - **Global state**: Package-level variables in `cmd/` (required by Cobra's flag binding) and `audit/plugin.go` - **Context underutilization**: `context.Context` not fully leveraged in long-running operations - **Finding type duplication**: Three similar `Finding` types across `compliance`, `processor`, and `audit` packages (issue #280) - **Severity type scattered**: No shared severity enum; each package defines its own (issue #324) ## Schema Gaps - \~40+ string fields should be `BoolFlag` type - \~9 `struct{}` fields should be `BoolFlag` - `Source`/`Destination` missing `Address`, `Not`, and `Port` (Source) fields (issue #255) - Full inventory documented in `docs/development/xml-structure-research.md` --- # Developer Gotchas
Critical patterns every contributor must know **1. XML Presence Detection**: Use `*string` not `string` for optional XML elements. `` and absent elements both produce `""` with `string`. **2. Struct Shallow Copy**: `normalized := *cfg` shares slice backing arrays. Always deep-copy slices before mutation. **3. Deterministic Output**: ALL lists from maps must be sorted (`slices.Sorted(maps.Keys())`) before rendering, comparing, or serializing. **4. Statistics Sync**: Adding a field to `common.Statistics` requires updating 3 places (struct, `computeStatistics()`, `computeTotalConfigItems()`). **5. cfgparser/Schema Sync**: Renaming schema fields requires updating `cfgparser/xml.go` switch cases. **6. Dual Validator Sync**: `internal/processor/validate.go` and `internal/validator/opnsense.go` maintain parallel whitelists that must stay in sync. **7. Mutex Non-Reentrancy**: Go's `sync.RWMutex` is NOT reentrant. Use internal `*Unsafe()` helpers for methods called while holding a lock. **8. Plugin Findings**: `RunChecks()` may return nil (normalize to empty slice) and results may be cached (always `slices.Clone()` before mutating). **9. No Templates**: All report generation is programmatic via `builder.MarkdownBuilder`. The `internal/templates/` directory does not exist. **10. ****`//nolint:`**** Placement**: Must be on a SEPARATE LINE above the call (inline gets stripped by gofumpt).
--- ## Package Breakdown by Size
PackageProduction LOCTest LOCRatioResponsibility
`converter`5,0746,2271.23:1Multi-format export (MD/JSON/YAML), streaming I/O, enrichment
`model`4,1796,8901.65:1CommonDevice, OPNsense converter, factory, re-export layer
`processor`3,4684,5211.30:1Report generation, compliance audit, statistics
`diff`2,8543,4121.19:1Config diffing, security change analysis, formatters
`schema`2,4792,0880.84:1OPNsense XML struct definitions
`sanitizer`1,5391,7021.11:1Sensitive data redaction, field/value pattern matching
`validator`1,1941,4061.18:1Semantic validation (whitelists, constraints)
`config`1,0438730.84:1Viper-based config management
`display`9261,0401.12:1Terminal rendering with lipgloss, wrapping
`audit`9009451.05:1Plugin manager, compliance check orchestration
--- ## Document Metadata
FieldValue
Document Version1.0
Created Date2026-03-07
Generated FromCodebase analysis, [AGENTS.md](http://AGENTS.md), architecture docs, Dosu knowledge base, GitHub issues
Approval StatusDraft