This tutorial shows you how to use SlateDB on Azure Blob Storage (ABS). You would need an ABS account to complete the tutorial.
Setup#
Install the Azure CLI.
Create Storage account#
The following steps creates a storage account and list the keys. This section can be skipped if you already have a storage account created.
# Set storage account names
StorageAccountName=<ReplaceWithAccountName>
ContainerName=<ReplaceWithContainerName>
ResourceGroupName=<ReplaceWithResourceGroupName>
# Login
az login
# Create Resource Group in the default subscription.
az group create --name $ResourceGroupName --location westus
# Create Azure Storage account.
az storage account create --name $StorageAccountName --resource-group $ResourceGroupName --location westus --sku Standard_LRS
# Create a storage container
az storage container create --name $ContainerName --account-name $StorageAccountName
# Get the keys.
az storage account keys list --resource-group $ResourceGroupName --account-name $StorageAccountName
Create a project#
Let's start by creating a new Rust project:
cargo init slatedb-abs
cd slatedb-abs
Add dependencies#
Now add SlateDB and the required dependencies to your Cargo.toml:
cargo add slatedb tokio --features tokio/macros,tokio/rt-multi-thread
cargo add object-store --features object-store/azure
cargo add anyhow
Write some code#
This code demonstrates puts that wait for results to be durable, and then puts that do not wait.
import { Code } from '@astrojs/starlight/components';
import absExample from '/../examples/src/azure_blob_storage.rs?raw'
Check the blob contents#
az storage blob list --container-name $ContainerName --account-name $StorageAccountName --prefix "tmp/slatedb_azure_blob_storage/" --delimiter "/" --output table
compactions/
wal/
manifest/
SlateDB uses four top-level directories over time:
manifest: Contains the manifest files. Manifest files define the state of the DB, including the set of SSTs that are part of the DB.wal: Contains the write-ahead log files.compacted: Contains the compacted SST files (may not appear in short examples).compactions: Contains the compaction-state snapshots.
Let's check the wal folder.
az storage blob list --container-name $ContainerName --account-name $StorageAccountName --prefix "tmp/slatedb_azure_blob_storage/wal/" --delimiter "/" --output table
Name Blob Type Blob Tier Length Content Type Last Modified Snapshot
----------------------------------------------------------- ----------- ----------- -------- ------------------------ ------------------------- ----------
tmp/slatedb_azure_blob_storage/wal/00000000000000000001.sst BlockBlob Hot 64 application/octet-stream 2024-09-07T01:15:49+00:00
tmp/slatedb_azure_blob_storage/wal/00000000000000000002.sst BlockBlob Hot 138 application/octet-stream 2024-09-07T01:15:49+00:00
tmp/slatedb_azure_blob_storage/wal/00000000000000000003.sst BlockBlob Hot 23388 application/octet-stream 2024-09-07T01:15:49+00:00
tmp/slatedb_azure_blob_storage/wal/00000000000000000004.sst BlockBlob Hot 64 application/octet-stream 2024-09-07T01:15:50+00:00
Each of these SST files is a write-ahead log (WAL) file, and each WAL file can contain many RowEntry values. They get flushed based on the flush_interval config or when flush is called explicitly.