Logging and Instrumentation#

RAGFlow uses a multi-layer logging system built around a central utility in common/log_utils.py, covering rotating file handlers, environment-based log-level configuration, live admin-API controls, and per-step timing instrumentation in the chat pipeline and search backends.

Logger Initialization#

init_root_logger(logfile_basename) is the single entry point for logger setup. It:

• Configures a RotatingFileHandler writing to <project_root>/logs/<basename>.log with a 10 MB max size and 5 backup files.
• Attaches a StreamHandler so the same output goes to stdout/stderr (visible via docker logs).
• Calls logging.captureWarnings(True) to route Python warnings through the logging framework 1.
• Reads the LOG_LEVELS environment variable (comma-separated pkg=LEVEL pairs) and applies each to the named logger 2.
• Defaults peewee and pdfminer to WARNING and the root logger to INFO unless overridden 3.

init_root_logger is called once per service process at startup:

Container Log Volumes#

The ragflow-cpu / ragflow-gpu services in docker/docker-compose.yml mount ./ragflow-logs:/ragflow/logs, so file-based logs are always accessible on the host regardless of container lifecycle.

Environment-Based Log Levels#

Set LOG_LEVELS in docker/.env before starting containers. Examples:

# Change ragflow.es_conn to DEBUG
LOG_LEVELS=ragflow.es_conn=DEBUG

# Multiple packages
LOG_LEVELS=ragflow.es_conn=DEBUG,peewee=INFO

Valid level names: DEBUG, INFO (default), WARNING, ERROR 4.

Runtime Admin API Controls#

Log levels can be changed without restarting via the admin server's REST endpoints, which call get_log_levels() and set_log_level(pkg_name, level) from common/log_utils.py.

Both routes require admin authentication (@check_admin_auth). The in-memory pkg_levels dict is updated alongside the live logger, so a subsequent GET reflects the change immediately 7.

Chat Pipeline Timing Instrumentation#

api/db/services/dialog_service.py uses timeit.default_timer for microsecond-precision wall-clock measurements across the async chat pipeline 8. Timestamps are captured after each major stage:

All deltas are calculated in milliseconds and appended to the prompt field of every chat response under a ## Time elapsed: section 16, including token count and generation speed (tokens/s). This data is also forwarded to Langfuse if a tracer is configured 17.

Search Backend Timing (OceanBase)#

The OceanBase connector in rag/utils/ob_conn.py logs per-query latency using time.time() around each SQL search call 18:

OBConnection.search table <index>, search type: hybrid, elapsed time: 0.123 seconds, got count: 42

This is consistent across fusion, vector, fulltext, and filter search variants 19. The pattern originates in the base class _execute_search_sql which returns (rows, elapsed_time) and is logged at INFO level in the caller.

The Elasticsearch/OpenSearch path in rag/nlp/search.py uses logging.debug for result counts (e.g., Dealer.search TOTAL: ...) 20 but does not instrument per-query latency.

Helper Utilities#

• log_exception(e, *args) — logs the exception with logging.exception, then logs and re-raises with additional context (e.g., HTTP response .text) from each extra argument. Use it in request handlers to ensure full tracebacks appear in logs.
• pkg_levels dict — module-level dict that the admin API updates in-place; the Python logging framework reads the logger's level directly from the Logger object, so set_log_level must also call logger.setLevel() 21.