Data Model Documentation Maintenance#

Data Model Documentation Maintenance refers to the systematic practices and automated tooling used in the opnDossier project to keep data model documentation synchronized with Go source code. The project uses a reflection-based documentation generator that automatically produces markdown reference documentation from Go struct definitions in internal/model/common/, ensuring that exported JSON/YAML field documentation remains accurate as the codebase evolves.

The documentation focuses on the CommonDevice export model—a platform-agnostic, normalized representation of firewall configurations designed for analysis and reporting—not the internal XML schema used for parsing OPNsense config files. This distinction is critical: field documentation is derived from JSON tags on CommonDevice structs, while validation rules and type constraints are extracted from struct tag annotations.

The regeneration workflow is triggered via just generate-docs, which compiles and executes the docgen tool to rebuild field reference tables. Manual documentation files (docs/data-model/examples/json-export.md and yaml-processing.md) provide jq/yq query patterns and integration examples that complement the auto-generated model reference.

Core Concepts#

The CommonDevice Export Model#

The CommonDevice struct serves as the root of the export model and organizes firewall configuration data into four logical tiers:

1. Device metadata - Platform type, version, theme
2. Core configuration - Always-present domains (system, interfaces, firewall rules, NAT)
3. Optional features - Pointer-typed fields for services that may not be configured (IDS, VPN, traffic shaping)
4. Enrichment data - Analytics, statistics, and security assessments added during export

The model contains only json: and yaml: tags, maintaining clean separation from the internal XML parsing layer.

Documentation File Structure#

The documentation lives in docs/data-model/ and consists of:

The model-reference.md file contains a timestamp and auto-generation warning to prevent manual edits.

Field Derivation from JSON Tags#

The docgen tool uses Go reflection to introspect struct definitions. For each exported field, it:

1. Extracts the JSON field name from struct tags (e.g., json:"hostname,omitempty" → hostname)
2. Formats the Go type name (handles pointers, slices, maps, nested structs)
3. Constructs JSON path notation with array indicators (e.g., users[].name)
4. Infers field descriptions from validation tags (validate:"required", validate:"oneof=...")

Example from internal/model/common/system.go:

// Hostname is the system hostname.
Hostname string `json:"hostname,omitempty" yaml:"hostname,omitempty"`

Generates:

Struct Tag Patterns#

CommonDevice structs follow consistent tagging conventions:

Core struct fields (always included in JSON):

System System `json:"system" yaml:"system,omitempty"`

Slices and optional primitives:

Interfaces []Interface `json:"interfaces,omitempty" yaml:"interfaces,omitempty"`
Version string `json:"version,omitempty" yaml:"version,omitempty"`

Pointer types (optional features):

IDS *IDSConfig `json:"ids,omitempty" yaml:"ids,omitempty"`

Boolean fields use clean Go bool types in CommonDevice (not XML presence-based BoolFlag):

Disabled bool `json:"disabled,omitempty" yaml:"disabled,omitempty"`

Regeneration Workflow#

Step-by-Step Process#

1. Modify Go structs in internal/model/common/
2. Run regeneration: Execute just generate-docs
3. Verify output: Check docs/data-model/model-reference.md for new fields
4. Update examples (if needed): Add jq/yq queries to manual docs

The justfile Recipe#

Lines 320-322 of justfile:

# Generate model reference documentation
[group('docs')]
generate-docs:
    @{{ mise_exec }} go run tools/docgen/main.go

This runs through mise's managed environment to ensure consistent Go tooling versions.

The docgen Tool#

tools/docgen/main.go is a standalone generator with:

• Build directive //go:build ignore for isolation from main builds
• Direct import of internal/schema package for compile-time type references
• Default output to docs/data-model/model-reference.md
• Customizable -output and -timestamp flags

jq/yq Example Patterns#

JSON Query Patterns (jq)#

docs/data-model/examples/json-export.md provides:

Basic field access:

jq '.system.hostname' config.json

Filtering arrays with booleans:

jq '.firewallRules[] | select(.disabled != true)' config.json

Aggregation and counting:

jq '[.firewallRules[] | .interfaces[]] | group_by(.) | map({interface: .[0], count: length})' config.json

Advanced patterns include:

• Configuration comparison with diff
• CSV/TSV export with @csv and @tsv operators
• Statistics generation with object construction
• VPN query examples (OpenVPN servers/clients, WireGuard servers/peers)
• Certificate and CA enumeration
• Enrichment data access (statistics, security assessment, analysis)

YAML Query Patterns (yq)#

docs/data-model/examples/yaml-processing.md covers:

Equivalent yq syntax:

yq '.system.hostname' config.yaml
yq '.firewallRules[] | select(.disabled != true)' config.yaml

Data transformation:

yq -o=json config.yaml > config.json # YAML to JSON
yq '.system' config.yaml > system.yaml # Subset extraction

Integration examples include:

• Complete Ansible playbooks
• Shell scripts with automated analysis
• Python integration with yaml.safe_load()
• VPN configuration queries (OpenVPN, WireGuard)
• Certificate management examples

Critical Boolean Gotcha#

The background context notes that CommonDevice uses Go bool types (not strings or BoolFlag):

# Works correctly with boolean fields
jq '.firewallRules[] | select(.disabled | not)' config.json

# NOT this (string comparison)
jq '.firewallRules[] | select(.disabled != "1")' config.json

This differs from the internal XML schema where disabled/log/floating/quick use presence-based BoolFlag types.

Adding New Fields to CommonDevice#

Workflow Checklist#

When extending the CommonDevice model:

1. Add field to appropriate struct in internal/model/common/

   • Include JSON/YAML tags
   • Add documentation comment
   • Add validation tags if applicable

2. Choose correct Go type:

   • Boolean flags → bool
   • Optional complex types → *StructType
   • Collections → []ElementType

3. Regenerate documentation: Run just generate-docs

4. Update query examples (if needed):

   • Add jq patterns to json-export.md
   • Add yq patterns to yaml-processing.md

5. Test round-trip: Verify the field exports correctly in JSON/YAML output

Example: Adding a New Field#

Add to internal/model/common/system.go:

// TimeZone is the configured system timezone.
TimeZone string `json:"timeZone,omitempty" yaml:"timeZone,omitempty" validate:"required"`

Run:

just generate-docs

Verify in model-reference.md:

Add query example to json-export.md:

# Get system timezone
jq '.system.timeZone' config.json

CommonDevice vs Internal XML Schema#

Key Distinctions#

Documentation Scope#

The data-model docs document CommonDevice, which is what users see in JSON/YAML output—not the internal OpnSenseDocument XML schema. Field tables must be derived from Go struct JSON tags in common/ files, not schema/ files.

Best Practices#

Maintenance Rules#

1. Never manually edit model-reference.md - Always regenerate via just generate-docs
2. Document type choices - Add comments explaining why a field uses specific Go types
3. Test round-trip transformations - Ensure XML → CommonDevice → JSON → verify works correctly
4. Keep examples current - Update jq/yq patterns when adding commonly-queried fields
5. Use validation tags - Add validate: tags to improve auto-generated descriptions

Struct Documentation Patterns#

Follow the comment patterns in common/ files:

// Address is the resolved effective address (e.g., "any", a CIDR, or hostname).
Address string `json:"address,omitempty" yaml:"address,omitempty"`

// Port is the port or port range specification.
Port string `json:"port,omitempty" yaml:"port,omitempty"`

// Negated indicates the endpoint match is inverted (NOT logic).
Negated bool `json:"negated,omitempty" yaml:"negated,omitempty"`

Include:

• Clear purpose statements
• Value format examples where helpful
• Valid value ranges or constraints

Avoiding Documentation Drift#

The automated regeneration workflow prevents documentation drift by:

• Generating field tables directly from current Go types
• Extracting validation rules from struct tags
• Timestamping generated output for audit trails
• Using compile-time type references (not string-based parsing)

Related Topics#

• Schema Parser Synchronization - Patterns for keeping XML DTOs, parser, and converter aligned
• XML Presence Detection - Understanding BoolFlag vs string for presence-based fields
• Model Re-Export Layer - Type alias layer masking internal schemas (scheduled for removal)
• opnDossier Architecture Overview - Four-layer data model architecture

Relevant Code Files#