Task File Lifecycle Management#

The MinerU API server (mineru/cli/fast_api.py) organizes all uploaded files and parse results under a per-task directory tree. Each task receives a UUID-based subdirectory under a configurable output root; the server periodically sweeps completed or failed tasks and deletes their directories once a configurable retention window expires.

Directory Structure#

Every task directory is created by create_task_output_dir():

<output_root>/
  <task-uuid>/
    uploads/ # raw uploaded files (saved by save_upload_files)
    <parse-output> # markdown, JSON, images written by the parse job

<output_root> defaults to ./output and is resolved by get_output_root(), which also creates the directory tree on first access.
Uploaded files land in <task-uuid>/uploads/ via save_upload_files(). Duplicate filenames within a request are de-conflicted by appending __upload_<N>.
Parse results (.md, _middle.json, _model.json, _content_list.json, images/) are written inside the task directory by the underlying parse job.

Environment Variables#

Three environment variables control lifecycle behavior :

Variable	Default	Min	Purpose
`MINERU_API_OUTPUT_ROOT`	`./output`	—	Root directory for all task output
`MINERU_API_TASK_RETENTION_SECONDS`	`86400` (24 h)	`0`	Seconds after task completion before the directory is deleted
`MINERU_API_TASK_CLEANUP_INTERVAL_SECONDS`	`300` (5 min)	`1`	How often the cleanup loop wakes to scan for expired tasks

Values are read at startup by get_task_retention_seconds() and get_task_cleanup_interval_seconds() via the shared get_int_env() helper, which silently falls back to the default on parse errors or values below the minimum.

Setting MINERU_API_TASK_RETENTION_SECONDS=0 disables the cleanup loop entirely — the AsyncTaskManager.start() method only launches the cleanup coroutine when retention is greater than zero.

Cleanup Loop#

AsyncTaskManager._cleanup_loop() is an asyncio background task that:

Sleeps for task_cleanup_interval_seconds.
Calls cleanup_expired_tasks(), which scans self.tasks and collects expired entries.
For each expired task: removes the task and its event from in-memory dicts, signals the event (unblocks any waiters), and calls cleanup_file(task.output_dir) to recursively delete the directory via shutil.rmtree.

Expiry is checked by _is_task_expired():

Only tasks in COMPLETED or FAILED state are eligible — active (pending, processing) tasks are never expired.
The clock starts at completed_at (UTC ISO timestamp). A task becomes eligible when (now − completed_at) ≥ task_retention_seconds.

Health Check Integration#

The GET /health endpoint exposes the current retention and cleanup interval , making it easy to verify the live configuration. is_healthy() returns false if the cleanup loop has stopped unexpectedly while retention is enabled, surfacing lifecycle failures in health monitoring.

Early Cleanup Paths#

Two additional paths delete task data immediately rather than waiting for the scheduled sweep:

Failed submission — if create_async_parse_task() raises (e.g., unsupported file type), the newly created task_output_dir is deleted inline .
Zip response files — temporary zip archives (tempfile.mkstemp) created for response_format_zip results are cleaned via FastAPI BackgroundTasks immediately after the response is sent .

Task File Lifecycle Management#

Directory Structure#

<output_root>/
  <task-uuid>/
    uploads/ # raw uploaded files (saved by save_upload_files)
    <parse-output> # markdown, JSON, images written by the parse job

<output_root> defaults to ./output and is resolved by get_output_root(), which also creates the directory tree on first access.

Uploaded files land in <task-uuid>/uploads/ via save_upload_files(). Duplicate filenames within a request are de-conflicted by appending __upload_<N>.

Parse results (.md, _middle.json, _model.json, _content_list.json, images/) are written inside the task directory by the underlying parse job.

Environment Variables#

Three environment variables control lifecycle behavior :

Variable	Default	Min	Purpose
`MINERU_API_OUTPUT_ROOT`	`./output`	—	Root directory for all task output
`MINERU_API_TASK_RETENTION_SECONDS`	`86400` (24 h)	`0`	Seconds after task completion before the directory is deleted
`MINERU_API_TASK_CLEANUP_INTERVAL_SECONDS`	`300` (5 min)	`1`	How often the cleanup loop wakes to scan for expired tasks

Setting MINERU_API_TASK_RETENTION_SECONDS=0 disables the cleanup loop entirely — the AsyncTaskManager.start() method only launches the cleanup coroutine when retention is greater than zero.

Cleanup Loop#

Sleeps for task_cleanup_interval_seconds.

Calls cleanup_expired_tasks(), which scans self.tasks and collects expired entries.

For each expired task: removes the task and its event from in-memory dicts, signals the event (unblocks any waiters), and calls cleanup_file(task.output_dir) to recursively delete the directory via shutil.rmtree.

Only tasks in COMPLETED or FAILED state are eligible — active (pending, processing) tasks are never expired.

The clock starts at completed_at (UTC ISO timestamp). A task becomes eligible when (now − completed_at) ≥ task_retention_seconds.

Early Cleanup Paths#

Two additional paths delete task data immediately rather than waiting for the scheduled sweep:

Failed submission — if create_async_parse_task() raises (e.g., unsupported file type), the newly created task_output_dir is deleted inline .

Zip response files — temporary zip archives (tempfile.mkstemp) created for response_format_zip results are cleaned via FastAPI BackgroundTasks immediately after the response is sent .