Development Setup#

This guide covers setting up a development environment for contributing to libmagic-rs, including tools, workflows, and best practices.

Current Implementation Status#

Project Phase: Active Development with Solid Foundation

Completed Components ✅#

Core AST Structures: Complete with 29 comprehensive unit tests
Parser Components: Numbers, offsets, operators, values (50 unit tests)
CLI Framework: Basic command-line interface with clap
Code Quality: Zero-warnings policy with comprehensive linting
Serialization: Full serde support for all data structures
Memory Safety: Zero unsafe code with bounds checking

In Progress 🔄#

Complete Magic File Parser: Integration of parsing components
Rule Evaluation Engine: Offset resolution and type interpretation
Memory-Mapped I/O: Efficient file access with memmap2
Output Formatters: Text and JSON result formatting

Test Coverage#

Current test suite includes 79 passing unit tests:

# Run current test suite
cargo test
# Output: running 79 tests ... test result: ok. 79 passed; 0 failed

Test Categories:

AST structure tests (29 tests)
Parser component tests (50 tests)
Serialization round-trip tests
Edge case and boundary value tests
Error condition handling tests

Prerequisites#

Required Tools#

Rust 1.85+ with the 2021 edition
Git for version control
Cargo (included with Rust)

Recommended Tools#

# Enhanced test runner
cargo install cargo-nextest

# Auto-rebuild on file changes
cargo install cargo-watch

# Code coverage
cargo install cargo-llvm-cov

# Security auditing
cargo install cargo-audit

# Dependency analysis
cargo install cargo-tree

# Documentation tools
cargo install mdbook # For this documentation

Environment Setup#

1. Clone the Repository#

git clone https://github.com/EvilBit-Labs/libmagic-rs.git
cd libmagic-rs

2. Verify Setup#

# Check Rust version
rustc --version # Should be 1.85+

# Verify project builds
cargo check

# Run tests
cargo test

# Check linting passes
cargo clippy -- -D warnings

3. IDE Configuration#

VS Code#

Recommended extensions:

rust-analyzer: Rust language server
CodeLLDB: Debugging support
Better TOML: TOML syntax highlighting
Error Lens: Inline error display

Settings (.vscode/settings.json):

{
  "rust-analyzer.check.command": "clippy",
  "rust-analyzer.check.extraArgs": [
    "--",
    "-D",
    "warnings"
  ],
  "rust-analyzer.cargo.features": "all"
}

Other IDEs#

IntelliJ IDEA: Use the Rust plugin
Vim/Neovim: Configure with rust-analyzer LSP
Emacs: Use rustic-mode with lsp-mode

Development Workflow#

Daily Development#

# Start development session
cargo watch -x check -x test

# In another terminal, make changes and see results automatically

Code Quality Checks#

# Format code (required before commits)
cargo fmt

# Check for issues (must pass)
cargo clippy -- -D warnings

# Run all tests
cargo nextest run # or cargo test

# Check documentation
cargo doc --document-private-items

Testing Strategy#

# Run specific test modules
cargo test ast_structures
cargo test parser
cargo test evaluator

# Run tests with output
cargo test -- --nocapture

# Run ignored tests (if any)
cargo test -- --ignored

# Test documentation examples
cargo test --doc

Project Standards#

Code Style#

The project enforces strict code quality standards:

Linting Configuration#

See Cargo.toml for the complete linting setup. Key rules:

No unsafe code: unsafe_code = "forbid"
Zero warnings: warnings = "deny"
Comprehensive clippy: Pedantic, nursery, and security lints enabled
No unwrap/panic: unwrap_used = "deny", panic = "deny"

Formatting#

# Format all code (required)
cargo fmt

# Check formatting without changing files
cargo fmt -- --check

Documentation Standards#

Code Documentation#

All public APIs must have rustdoc comments:

/// Parses a magic file into an AST
///
/// This function reads a magic file from the given path and parses it into
/// a vector of `MagicRule` structures that can be used for file type detection.
///
/// # Arguments
///
/// * `path` - Path to the magic file to parse
///
/// # Returns
///
/// Returns `Ok(Vec<MagicRule>)` on success, or `Err(LibmagicError)` if parsing fails.
///
/// # Errors
///
/// This function will return an error if:
/// - The file cannot be read
/// - The magic file syntax is invalid
/// - Memory allocation fails
///
/// # Examples
///
/// ```rust,no_run
/// use libmagic_rs::parser::parse_magic_file;
///
/// let rules = parse_magic_file("magic.db")?;
/// println!("Loaded {} rules", rules.len());
/// # Ok::<(), Box<dyn std::error::Error>>(())
/// ```
pub fn parse_magic_file<P: AsRef<Path>>(path: P) -> Result<Vec<MagicRule>> {
    // Implementation
}

Module Documentation#

Each module should have comprehensive documentation:

//! Magic file parser module
//!
//! This module handles parsing of magic files into an Abstract Syntax Tree (AST)
//! that can be evaluated against file buffers for type identification.
//!
//! # Magic File Format
//!
//! Magic files use a simple DSL to describe file type detection rules:
//!
//! ```text
//! # ELF files
//! 0 string \x7fELF ELF
//! >4 byte 1 32-bit
//! >4 byte 2 64-bit
//! ```
//!
//! # Examples
//!
//! ```rust,no_run
//! use libmagic_rs::parser::parse_magic_file;
//!
//! let rules = parse_magic_file("magic.db")?;
//! # Ok::<(), Box<dyn std::error::Error>>(())
//! ```

Testing Standards#

Unit Tests#

Every module should have comprehensive unit tests:

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_basic_functionality() {
        // Test basic case
        let result = function_under_test();
        assert_eq!(result, expected_value);
    }

    #[test]
    fn test_error_conditions() {
        // Test error handling
        let result = function_that_should_fail();
        assert!(result.is_err());
    }

    #[test]
    fn test_edge_cases() {
        // Test boundary conditions
        // Empty inputs, maximum values, etc.
    }
}

Integration Tests#

Place integration tests in the tests/ directory:

// tests/integration_test.rs
use libmagic_rs::*;

#[test]
fn test_end_to_end_workflow() {
    // Test complete workflows
    let db = MagicDatabase::load_from_file("third_party/magic.mgc").unwrap();
    let result = db
        .evaluate_file("third_party/tests/elf64.testfile")
        .unwrap();
    assert_eq!(result.description, "ELF 64-bit LSB executable");
}

Error Handling#

Use the project's error types consistently:

use thiserror::Error;

#[derive(Debug, Error)]
pub enum ModuleError {
    #[error("Invalid input: {0}")]
    InvalidInput(String),

    #[error("Processing failed: {reason}")]
    ProcessingFailed { reason: String },

    #[error("IO error: {0}")]
    IoError(#[from] std::io::Error),
}

pub type Result<T> = std::result::Result<T, ModuleError>;

Contribution Workflow#

1. Issue Creation#

Before starting work:

Check existing issues and discussions
Create an issue describing the problem or feature
Wait for maintainer feedback on approach

2. Branch Creation#

# Create feature branch
git checkout -b feature/descriptive-name

# Or for bug fixes
git checkout -b fix/issue-description

3. Development Process#

# Make changes following the standards above
# Run checks frequently
cargo watch -x check -x test

# Before committing
cargo fmt
cargo clippy -- -D warnings
cargo test

4. Commit Guidelines#

Use conventional commit format:

# Feature commits
git commit -m "feat(parser): add support for indirect offsets"

# Bug fixes
git commit -m "fix(evaluator): handle buffer overflow in string reading"

# Documentation
git commit -m "docs(api): add examples for MagicRule creation"

# Tests
git commit -m "test(ast): add comprehensive serialization tests"

5. Pull Request Process#

Push branch: git push origin feature/descriptive-name
Create PR with:
- Clear description of changes
- Reference to related issues
- Test coverage information
- Breaking change notes (if any)
Sign off commits with git commit -s (DCO required)
Address feedback from code review
Ensure CI passes all checks

For full details on code review criteria, DCO requirements, and project governance, see CONTRIBUTING.md.

Debugging#

Logging#

The project uses log = "0.4" as its logging facade. Logging is used throughout the codebase to provide diagnostic information during development and troubleshooting.

Logging Conventions:

log::debug!() for diagnostic information (such as skipped rule evaluations in src/evaluator/engine/mod.rs)
log::warn!() for warnings (parse failures in src/parser/loader.rs, confidence validation issues in src/output/mod.rs)

Example usage:

use log::{debug, warn};

pub fn evaluate_rules(rules: &[MagicRule], buffer: &[u8]) -> Result<Vec<RuleMatch>> {
    for rule in rules {
        match evaluate_single_rule(rule, buffer) {
            Ok(data) => { /* ... */ }
            Err(e) => {
                // Expected evaluation errors -- skip gracefully
                debug!("Skipping rule '{}': {}", rule.message, e);
                continue;
            }
        }
    }
    // ...
}

Enable logging during development:

# See debug output during testing
RUST_LOG=debug cargo test

# See debug output when running the CLI
RUST_LOG=debug cargo run -- --use-builtin file.bin

# Filter to specific module
RUST_LOG=libmagic_rs::evaluator=debug cargo run

Debug-level logging is particularly useful for understanding rule evaluation behavior in src/evaluator/engine/mod.rs, where it shows which rules are skipped due to buffer overruns, invalid offsets, or type read errors.

Debugging Tests#

# Run single test with output
cargo test test_name -- --nocapture

# Debug with lldb/gdb
cargo test --no-run
lldb target/debug/deps/libmagic_rs-<hash>

Performance Profiling#

# Install profiling tools
cargo install cargo-flamegraph

# Profile specific benchmarks
cargo flamegraph --bench evaluation_bench

# Memory profiling with valgrind
cargo build
valgrind --tool=massif target/debug/rmagic large_file.bin

Continuous Integration#

The project uses GitHub Actions for CI. Local checks should match CI:

# Run the same checks as CI
cargo fmt -- --check
cargo clippy -- -D warnings
cargo test
cargo doc --document-private-items

Release Process#

For maintainers:

Version Bumping#

# Update version in Cargo.toml
# Update CHANGELOG.md
# Commit changes
git commit -m "chore: bump version to 0.3.0"
git tag v0.3.0
git push origin main --tags

Documentation Updates#

# Update documentation
mdbook build docs/
# Deploy to GitHub Pages (automated)

This development setup ensures high code quality, comprehensive testing, and smooth collaboration across the project.