RocksDB Integration#

Overview#

HugeGraph uses RocksDB as a local, embedded storage backend in two distinct contexts:

• hugegraph-server (hugegraph-rocksdb module): the primary OLTP/OLAP backend for standalone or non-distributed deployments, exposing RocksDB directly to graph queries.
• hugegraph-store (hg-store-rocksdb module): the underlying storage for each Raft partition in the distributed hg-store tier.

Entry points:

• Server store: RocksDBStore.java, RocksDBStdSessions.java
• Store-tier options: hugegraph-store/.../RocksDBOptions.java
• Server options: hugegraph-server/.../RocksDBOptions.java

Configuration#

Both modules share the same option key namespace (rocksdb.*) but have different defaults tuned for their workloads.

Data & WAL Paths (server)#

1

Durability#

• rocksdb.use_fsync (default false): when true, every store to stable storage issues an fsync. 2
• rocksdb.bytes_per_sync / rocksdb.wal_bytes_per_sync (default 0): incremental OS sync of SST/WAL files; 0 disables. 3
• rocksdb.atomic_flush (default false): atomically flush multiple column families to MANIFEST. Not needed when WAL is always enabled. 4
• Raft mode: when RAFT_MODE=true, each session disables WAL (setDisableWAL(true)) and sync (setSync(false)) because Raft log + snapshot handles recovery. 5

Performance Tuning#

6 7

Bulk-load mode (rocksdb.bulkload_mode=true): disables auto-compaction and removes all level-0/pending-compaction limits for maximum write throughput. 8

Block cache / Bloom filter: block cache defaults to 8 MB (rocksdb.block_cache_capacity). Bloom filter is off by default (bloom_filter_bits_per_key=-1); set to 10 for ~1% FPR. 9

initOptions() applies all of the above at open time. 10

Snapshot-Based Recovery (hugegraph-server)#

RocksDBStore exposes createSnapshot and resumeSnapshot at the store level; RocksDBStdSessions executes them:

• createSnapshot(snapshotPath): calls rocksdb.createCheckpoint(snapshotPath) on every open RocksDB instance in the store. 11 12
• resumeSnapshot(snapshotPath, deleteSnapshot): closes RocksDB, deletes the origin data directory, moves (or hard-links when deleteSnapshot=false) the snapshot directory to the original path, then calls reloadRocksDB(). 13
• hardLinkSnapshot: opens the snapshot read-only, creates a checkpoint into a _temp directory, and returns that path. Used to preserve the original snapshot while restoring. 14

Note: resumeSnapshot closes RocksDB before deleting the origin directory. Closing after copying would risk dirty data. 15

Snapshot-Based Recovery (hugegraph-store / Raft tier)#

In the distributed store, snapshots tie into the Raft state machine. The flow is:

1. PartitionStateMachine.onSnapshotSave() → SnapshotHandler.onSnapshotSave() → RocksDBSession.saveSnapshot() creates a RocksDB Checkpoint at _temp, then atomically renames to the final path. 16
2. Files are registered with the JRaft SnapshotWriter; CRC64 checksums are computed (full checksum for files ≤ 8 KB; head+tail 4 KB blocks for larger files). 16
3. A should_not_load marker prevents the local node from redundantly loading its own snapshot. 16
4. RocksDBSession.loadSnapshot(): under a write lock, hard-links snapshot files to _temp, calls verifySnapshot() (opens read-only, validates all column family handles), and only on success atomically moves the verified directory to the live database path. A corrupt snapshot therefore never replaces a healthy database. 16

Exception Handling#

In RocksDBStore.open(), specific RocksDBException messages are caught and handled gracefully 17:

General corruption (e.g., a RocksDBException that is none of the above) causes startup failure. Built-in repair (RocksDB.repairDB()) is not implemented in HugeGraph's integration; recovery is expected via snapshot restore.

Key Source Files#