File Write Safety Patterns#

File Write Safety Patterns are a collection of coding standards and best practices used in the opnDossier project to ensure data integrity, safe file operations, and consistent code quality across the codebase. These patterns encompass multiple aspects of Go programming including file I/O with explicit sync operations, nil-safe comparison functions, slice manipulation, and standalone tool architecture.

The patterns emerged from practical development experience and are formally documented in the project's AGENTS.md guidelines. They represent defensive programming techniques designed to prevent data loss during file operations, ensure consistent behavior when handling nil values, and maintain code clarity through standardized idioms.

These patterns are particularly important in opnDossier's context as a configuration analysis tool that processes and writes critical firewall configuration data, where data loss or corruption could have security implications.

File Write Safety with Sync() and Close()#

Overview#

The primary file write safety pattern ensures that data is reliably persisted to disk by calling file.Sync() before closing files and properly handling close errors for write operations.

The Sync-Before-Close Pattern#

The standard pattern combines three key elements:

Deferred Close with Error Logging: Use defer to ensure the file is closed, with explicit error handling via logger.Warn
Explicit Sync Before Return: Call file.Sync() before function return to flush data to disk
Error Propagation for Sync: Sync errors return wrapped errors since they indicate potential data loss

Example from cmd/diff.go:

outputFile, err = os.Create(diffOutputFile)
if err != nil {
    return fmt.Errorf("failed to create output file: %w", err)
}
defer func() {
    // Close error is important for write operations - data could be lost
    if cerr := outputFile.Close(); cerr != nil {
        // Log but don't override a previous error
        logger.Warn("failed to close output file", "error", cerr)
    }
}()

// ... write operations ...

// Sync to ensure all data is written to disk before returning success
if outputFile != nil {
    if err := outputFile.Sync(); err != nil {
        return fmt.Errorf("failed to sync output file: %w", err)
    }
}

Multiple File Operations#

When writing multiple output files, each follows the same pattern independently:

// Main output file
outputWriter, err = os.Create(actualOutputFile)
if err != nil {
    return fmt.Errorf("failed to create output file %s: %w", actualOutputFile, err)
}
defer func() {
    if cerr := outputWriter.Close(); cerr != nil {
        ctxLogger.Warn("failed to close output file", "error", cerr)
    }
}()

// ... perform write operations ...

if err := outputWriter.Sync(); err != nil {
    return fmt.Errorf("failed to sync output file: %w", err)
}

// Mapping file follows same pattern
mappingWriter, err := os.Create(mappingPath)
defer func() {
    if cerr := mappingWriter.Close(); cerr != nil {
        ctxLogger.Warn("failed to close mapping file", "error", cerr)
    }
}()

if err := mappingWriter.Sync(); err != nil {
    return fmt.Errorf("failed to sync mapping file: %w", err)
}

Atomic File Operations#

For critical files requiring atomic updates, the pattern uses write-sync-close-rename:

// Ensure cleanup on error
defer func() {
    if tempFile != nil {
        if closeErr := tempFile.Close(); closeErr != nil {
            if e.logger != nil {
                e.logger.Warn("Failed to close temporary file during cleanup",
                    "path", tempPath, "error", closeErr)
            }
        }
    }
    // Only remove temp file if we haven't successfully renamed it
    if _, statErr := os.Stat(tempPath); statErr == nil {
        if removeErr := os.Remove(tempPath); removeErr != nil {
            if e.logger != nil {
                e.logger.Warn("Failed to remove temporary file during cleanup",
                    "path", tempPath, "error", removeErr)
            }
        }
    }
}()

// Write content to temporary file
if _, err := tempFile.Write(content); err != nil {
    return fmt.Errorf("failed to write to temporary file: %w", err)
}

// Ensure content is flushed to disk
if err := tempFile.Sync(); err != nil {
    return fmt.Errorf("failed to sync temporary file: %w", err)
}

// Close the temporary file before renaming
if err := tempFile.Close(); err != nil {
    return fmt.Errorf("failed to close temporary file: %w", err)
}

tempFile = nil // Prevent cleanup in defer

// Atomically rename temporary file to target location
if err := os.Rename(tempPath, path); err != nil {
    return fmt.Errorf("failed to rename temporary file to target: %w", err)
}

Design Rationale#

The pattern addresses several failure modes:

Buffer Flush Failures: Operating system buffers may not be flushed without explicit Sync()
Close Errors: Close can fail on write operations, indicating data loss
Partial Writes: Atomic operations prevent readers from seeing incomplete data
Double-Close Prevention: Setting file handles to nil after successful close prevents double-close errors

Comparison Function Patterns#

Nil Handling#

Comparison functions must handle nil inputs at the start using a three-part check:

func (a *Analyzer) CompareSystem(old, newCfg *common.System) []Change {
    var changes []Change

    // Handle nil pointers gracefully
    if old == nil && newCfg == nil {
        return changes // Both nil → return empty changes
    }
    if old == nil {
        return []Change{{
            Type: ChangeAdded,
            Section: SectionSystem,
            Path: "system",
            Description: "System configuration section added",
        }}
    }
    if newCfg == nil {
        return []Change{{
            Type: ChangeRemoved,
            Section: SectionSystem,
            Path: "system",
            Description: "System configuration section removed",
        }}
    }
    // ... continue with actual comparison
}

The pattern ensures:

Both nil: Returns empty result (no changes)
One nil: Returns "added" or "removed" based on which is nil
Neither nil: Proceeds to actual comparison logic

Slice Equality Checks#

Use slices.Equal() instead of manual iteration:

func rulesEqual(a, b common.FirewallRule) bool {
    return a.Type == b.Type &&
        a.Description == b.Description &&
        a.Protocol == b.Protocol &&
        a.Disabled == b.Disabled &&
        a.Source == b.Source &&
        a.Destination == b.Destination &&
        slices.Equal(a.Interfaces, b.Interfaces)
}

For order-independent comparison, clone and sort before comparing:

func (p *CoreProcessor) rulesAreEquivalent(rule1, rule2 common.FirewallRule) bool {
    ifaces1 := slices.Clone(rule1.Interfaces)
    ifaces2 := slices.Clone(rule2.Interfaces)
    slices.Sort(ifaces1)
    slices.Sort(ifaces2)

    if rule1.Type != rule2.Type ||
        rule1.IPProtocol != rule2.IPProtocol ||
        !slices.Equal(ifaces1, ifaces2) {
        return false
    }
    // ... additional comparisons
}

Nil Presence Checks#

For pointer fields representing presence-based flags, use XOR logic:

func (s Source) Equal(other Source) bool {
    if (s.Any != nil) != (other.Any != nil) {
        return false
    }
    return s.Network == other.Network &&
        s.Address == other.Address &&
        s.Port == other.Port &&
        s.Not == other.Not
}

Map-Like Types and Get() Methods#

The codebase uses two different return signature patterns for Get() methods depending on context.

(value, bool) Pattern#

Simple map wrappers follow Go's standard map idiom:

type Interfaces struct {
    Items map[string]Interface `xml:",any"`
}

func (i *Interfaces) Get(key string) (Interface, bool) {
    if i.Items == nil {
        return Interface{}, false
    }
    iface, ok := i.Items[key]
    return iface, ok
}

This pattern is used for:

Direct map lookups where absence is normal
Schema/configuration types
Cases where no additional error context is needed

(value, error) Pattern#

Complex retrieval operations use error returns:

type PluginRegistry struct {
    plugins map[string]compliance.Plugin
    mutex sync.RWMutex
}

func (pr *PluginRegistry) GetPlugin(name string) (compliance.Plugin, error) {
    pr.mutex.RLock()
    defer pr.mutex.RUnlock()

    p, exists := pr.plugins[name]
    if !exists {
        return nil, compliance.ErrPluginNotFound
    }
    return p, nil
}

This pattern is used for:

Thread-safe operations requiring mutex locks
Cases needing specific error types for context
Searches that iterate rather than direct map access

Slice Pre-allocation Patterns#

Default Pattern: No Capacity Hints#

For small, variable-length slices, use make([]T, 0) without capacity:

func NewNetworkConfig() NetworkConfig {
    return NetworkConfig{
        VLANs: make([]VLANConfig, 0),
        Gateways: make([]Gateway, 0),
        Interfaces: Interfaces{
            Items: make(map[string]Interface),
        },
    }
}

func buildDHCPv6Items(dhcp common.DHCPScope) []string {
    items := make([]string, 0)
    if dhcp.Track6Interface != "" {
        items = append(items, "Track6 Interface: "+dhcp.Track6Interface)
    }
    // ... conditional appends
    return items
}

Capacity Hints: When to Use#

Add capacity hints only when:

The capacity value is reused elsewhere (e.g., in a loop)
The code is performance-critical (benchmarks)

// Benchmark setup with known sizes
doc := &common.CommonDevice{
    Interfaces: make([]common.Interface, 0, 50),
    FirewallRules: make([]common.FirewallRule, 0, 1000),
    Sysctl: make([]common.SysctlItem, 0, 200),
}

for i := range 50 {
    doc.Interfaces = append(doc.Interfaces, /* ... */)
}

for i := range 1000 {
    doc.FirewallRules = append(doc.FirewallRules, /* ... */)
}

Guidelines#

Per AGENTS.md:

Use make([]T, 0) without capacity hints for small, variable-length slices
Only add capacity hints when the capacity value is reused elsewhere or performance-critical
Avoid creating constants solely for capacity hints (adds maintenance burden)

Standalone Tools Pattern#

Build Isolation with //go ignore#

Standalone development tools are placed in tools/<name>/main.go with a build constraint:

// Package main generates model reference documentation from Go types.
//
//go:build ignore

package main

import (
    "flag"
    "fmt"
    "os"
    // ...
)

func main() {
    outputFile := flag.String("output", defaultOutputFile, "Output file path")
    flag.Parse()

    content := generateModelReference(*timestamp)

    if err := os.WriteFile(*outputFile, []byte(content), 0o644); err != nil {
        fmt.Fprintf(os.Stderr, "Error writing file: %v\n", err)
        os.Exit(1)
    }
}

Running Tools#

Tools are invoked via:

# Via justfile task
just generate-docs

# Or directly
go run tools/docgen/main.go

Design Principles#