SlateDB's garbage collector runs as a background task in the client process, periodically checking for obsolete files in the database storage.

The garbage collector has a configurable minimum age and interval for each file type (WAL SSTs, WAL fence SSTs, compacted SSTs, manifests, and compactions). Garbage collection for a file type can be disabled by setting its options to None. The collector runs every interval seconds and will delete files older than min_age that are not referenced by any active manifest or checkpoint.

Each garbage collection directory type supports a dry_run option. When enabled, the collector logs files that would be deleted without actually deleting them. This is useful for testing or verifying garbage collection behavior before enabling actual deletion.

Boundary files#

Before deleting old manifest or compactions metadata, SlateDB normally advances a durable boundary
file. Metadata writers check that boundary after creating a new version, which prevents a stale
writer from successfully publishing metadata that GC has already passed.

Boundary advancement can be disabled with
GarbageCollectorOptions::boundary_files_enabled. This supports object stores without conditional
overwrite (If-Match) while allowing GC to continue deleting eligible metadata. Metadata readers
and writers still check any existing boundary; on a new database, no boundary file is created while
all garbage collectors use this mode. Without a boundary, a SlateDB client or compactor can begin
updating a manifest or compactions file, stop making progress (for example, because its process or
host is suspended), then resume after min_age. GC may have deleted the ID it intended to create,
so create-if-absent can reuse that ID and report a stale update as successful even though newer
metadata has superseded it. Every garbage collector must use a compatible setting, and the manifest
and compactions min_age values must exceed the maximum lifetime of a stale process.

Filtering Deletion Candidates#

SlateDB supports custom filtering of garbage collection candidates through the GcFilter trait. This allows users to intercept files before deletion and approve or reject them based on custom logic.

The GcFilter::filter method receives a set of files that SlateDB has already determined are eligible for deletion (files older than min_age that are not referenced by any active manifest or checkpoint). The filter returns the subset of files that may be physically deleted. Any files not returned by the filter are retained.

A common use case for GcFilter is verifying that files have been replicated to another bucket before allowing deletion. For example, in a mirror-maker-like scenario for cross-region or cross-cloud replication, the filter can check that a file exists in the destination bucket before approving it for deletion from the source.

To configure a filter, use the with_gc_filter() method on GarbageCollectorBuilder:

let gc_filter = Arc::new(MyGcFilter::new());
let gc = GarbageCollectorBuilder::new(path, object_store)
    .with_gc_filter(gc_filter)
    .build()
    .await?;

The filter is optional. If no filter is configured, garbage collection proceeds normally based on age and reference checks alone.

Below is a diagram illustrating the high-level flow of garbage collection in SlateDB:

Default Garbage Collection Behavior#

By default, garbage collection is enabled for all managed directories (manifest, WAL, WAL fence, compacted SSTs, and compactions) using standard interval and minimum age settings.

WAL fence garbage collection runs in dry-run mode by default. This means it logs files that would be deleted without actually deleting them. This conservative default prevents accidental data loss while still providing visibility into what would be cleaned up.

To enable actual deletion for WAL fence GC, set dry_run: false with a high min_age to safely clean up old fences. Alternatively, to silence the dry-run logging entirely, set wal_fence_options: None.