Yes, docling-serve has S3 support through dedicated batch and regular convert endpoints, depending on your configuration.
S3 Integration via Endpoints#
S3 sources can be used in two contexts:
Regular Convert Endpoint (POST /v1/convert/source)#
- S3 sources are supported only with the KFP engine (
DOCLING_SERVE_ENG_KIND=KFP). - S3 sources require
S3Target— presigned URL targets are not supported on this endpoint. - Use
ConvertSourcesRequestwithS3SourceRequestandS3Target.
Batch Endpoint (POST /v1/convert/source/batch)#
- S3 sources are available regardless of engine type.
- S3 sources require
S3Target— presigned URL targets are not supported. - Use
BatchConvertSourcesRequestwithS3SourceRequestandS3Target. - This endpoint is async-only and returns a
TaskStatusResponse.
When using S3 sources:
- The
S3SourceRequestrequires:endpoint,access_key,secret_key, andbucket. - Optional:
max_num_elementslimits the number of S3 objects to iterate for the source. If not specified, all objects matching the criteria will be processed. Must be >= 1 if provided. - S3-compatible systems like MinIO or IBM COS may work by pointing to a custom endpoint, though this is not explicitly documented.
- Set
DOCLING_SERVE_ARTIFACT_STORAGE_VERIFY_SSL=falsefor local HTTP or self-signed MinIO setups.
Python SDK Support#
The DoclingServiceClient provides a submit_batch() method for S3-based batch processing:
from docling.service_client import DoclingServiceClient
from docling.datamodel.service.requests import S3SourceRequest
from docling.datamodel.service.targets import S3Target
client = DoclingServiceClient(base_url="http://localhost:5000")
sources = [
S3SourceRequest(
endpoint="s3.amazonaws.com",
access_key="your-key",
secret_key="your-secret",
bucket="my-bucket",
key="path/to/document.pdf"
)
]
job = client.submit_batch(
sources=sources,
target=S3Target(
endpoint="s3.amazonaws.com",
access_key="your-key",
secret_key="your-secret",
bucket="converted-bucket"
),
output_formats=[OutputFormat.DOCLING_V2]
)
result = job.result(timeout=300)
Result Delivery Options#
PresignedUrlTarget#
The PresignedUrlTarget allows docling-serve to upload conversion results directly to S3 using presigned URLs:
- Available on regular convert endpoints (
/v1/convert/source,/v1/convert/file) and the batch endpoint with HTTP sources (not S3 sources). - Requires artifact storage to be enabled on the server (
DOCLING_SERVE_ARTIFACT_STORAGE_ENABLED=true). - The server returns artifact manifests with presigned URLs for each output format.
- You can download conversion results directly from S3 without fetching them through the server.
- More efficient for large documents or batch processing.
The result type depends on your target:
PresignedUrlTarget→PresignedUrlConvertResponsewith per-document artifact manifestsS3Target→PresignedUrlConvertDocumentResponsewith counts only (artifacts delivered remotely)
Workaround: Fetch and Upload#
If you need to process S3 documents outside the batch workflow, you can fetch the document yourself and pass it as a file upload:
file_bytes = s3_client.get_object(...)['Body'].read()
file_stream = BytesIO(file_bytes)
doc_stream = DocumentStream(stream=file_stream, name=filename, format=input_format)
result = doc_converter.convert(doc_stream)