Namespace Layer Architecture#

LanceDB's namespace layer is the subsystem responsible for managing table metadata, discovery, and location resolution. It sits between the Connection API and the physical Lance datasets, and can operate in two modes: directory-listing (default) or manifest-enabled. Both ultimately delegate to LanceNamespaceDatabase, which wraps a LanceNamespace trait object from the external lance_namespace_impls crate.

Connection Routing#

The routing decision happens in ConnectBuilder::execute():

• db:// URIs → remote (RemoteDatabase)
• manifest_enabled: true → calls ListingDatabase::connect_manifest_enabled_namespace_database(), returns a LanceNamespaceDatabase directly
• default → calls ListingDatabase::connect_with_options(), returns a ListingDatabase (which still embeds a LanceNamespaceDatabase for child-namespace operations)

The manifest_enabled flag lives on ConnectRequest and is set via ConnectBuilder::manifest_enabled().

ConnectBuilder::execute()
  ├── db:// URI → RemoteDatabase
  ├── manifest_enabled=true → connect_manifest_enabled_namespace_database()
  │ └── LanceNamespaceDatabase ("dir" impl)
  └── default → connect_with_options()
                   └── ListingDatabase
                        └── [embedded] LanceNamespaceDatabase (for child namespaces)

For direct namespace connections (bypassing the URI-based path entirely), use connect_namespace(ns_impl, properties), which produces a ConnectNamespaceBuilder that calls LanceNamespaceDatabase::connect() directly.

Initialization Path#

Standard mode#

ListingDatabase::connect_namespace_database() is called during ListingDatabase construction to create the embedded namespace database. It:

1. Builds namespace client properties via build_namespace_client_properties(), setting root to the URI and prefixing storage options with storage. 1
2. Calls LanceNamespaceDatabase::connect("dir", ...), which instantiates a ConnectBuilder from lance_namespace_impls 2

Manifest-enabled mode#

connect_manifest_enabled_namespace_database() additionally injects two properties before connecting 3:

• manifest_enabled = "true" — makes the namespace the source of truth for table metadata
• dir_listing_to_manifest_migration_enabled = "true" — automatically migrates existing directory-listed tables into the manifest on first access

URI restrictions apply: schemes containing + (e.g. s3+ddb://), engine query parameters, and mirroredStore are rejected because they conflict with manifest-based commit management 4.

Delegation to lance_namespace_impls::ConnectBuilder#

Inside LanceNamespaceDatabase::connect_with_new_table_config() 2:

1. Creates lance_namespace_impls::ConnectBuilder::new(ns_impl) 5
2. Iterates over ns_properties and calls .property(key, value) for each
3. Optionally installs a Session via .session()
4. Installs a ReadFreshnessContextProvider for read consistency 6
5. Calls .connect().await to obtain the Arc<dyn LanceNamespace> trait object

The resulting LanceNamespaceDatabase stores the namespace client alongside storage options, read consistency interval, pushdown operation set, and new-table config 7.

LanceNamespaceDatabase – Core Structure#

LanceNamespaceDatabase implements the Database trait. Its fields:

Table operation dispatch:

• List: namespace.list_tables() 9
• Create: namespace.declare_table() → write data → NativeTable::create_from_namespace() 10
• Open: NativeTable::open_from_namespace() 11
• Drop: namespace.drop_table() 12
• Rename: namespace.rename_table() 13

clone_table is explicitly not supported on namespace connections.

Managed Versioning#

When describe_table or declare_table returns managed_versioning: Some(true), table creation installs ExternalManifestCommitHandler as the Lance commit handler 14. This routes all version writes through LanceNamespaceExternalManifestStore (from lance::io::commit::namespace_manifest), making the namespace server — not the object store — the version-of-record.

Activated by setting namespace_client_property("table_version_tracking_enabled", "true") on the connection 15.

Python API#

Python exposes two entry points in namespace.py:

• connect_namespace(ns_impl, properties, ...) → sync LanceNamespaceDBConnection
• connect_namespace_async(ns_impl, properties, ...) → AsyncLanceNamespaceDBConnection

For "dir" and "rest" impls, these call the Rust-native _connect_namespace binding; other impls fall back to constructing a Python LanceNamespace client and passing it to the Rust layer via _connect_namespace_client 16.

Key Source Files#