Database Session Management#

Dify uses Flask-SQLAlchemy with a context-local db.session proxy. Historically, service and core modules accessed this global session directly, which led to three classes of bugs:

1. DetachedInstanceError — ORM objects loaded in one session context become detached when accessed in another (e.g., across thread or task boundaries). 1
2. PendingRollbackError — Calling db.session.refresh() on a session with a pending rollback causes a cascade failure. 2
3. Transaction isolation violations — An explicit db.session.commit() inside a helper or decorator flushes all pending dirty state in the request-scoped session, not just the intended writes, making any upstream rollback impossible. 3 4

A large-scale refactoring effort (issue #37403) is resolving these by making session access explicit across ~153+ files: api/controllers/ (62), api/services/ (51), api/core/ (68), and api/tasks/ (42).

Pattern 1 — Explicit Session Injection#

Service methods that read or mutate ORM state now accept an explicit session parameter instead of importing db and using the global proxy. Controllers own the session lifecycle and pass it down.

Signature convention — see load_user in api/services/account_service.py:

# Service layer
def load_user(user_id: str, session: scoped_session | Session) -> None | Account:
    account = session.get(Account, user_id)
    ...

The module-level docstring captures the contract: "methods that read or mutate ORM state accept an explicit session so controllers, tasks, and tests can control transaction lifetime and avoid hidden Flask-scoped session usage inside service logic." 5

This pattern has been applied across ApiKeyAuthService, FeedbackService and MetadataService, AppService, DatasetService, TagService, and many others under issue #37403.

Type annotation: Methods use Session | scoped_session to support both concrete and proxy sessions without requiring a specific SQLAlchemy import at the call site.

Pattern 2 — Independent Session for Side-Effect Writes#

For audit logging, hit-count updates, or rate-limit recording that must commit immediately but must not affect the caller's transaction, use an independent session via sessionmaker:

with sessionmaker(bind=db.engine, expire_on_commit=False).begin() as session:
    session.add(record)
# auto-commits on context exit; isolated from db.session

This context manager auto-commits on exit and is completely decoupled from the request-scoped db.session.

Canonical examples in the codebase:

• Dataset query audit logging — _on_query in api/core/rag/retrieval/dataset_retrieval.py: writes DatasetQuery records in an independent session so a downstream workflow failure cannot roll back the audit trail.
• Rate limit logging — cloud_edition_billing_rate_limit_check in api/controllers/service_api/wraps.py: the decorator records a RateLimitLog in its own session before raising Forbidden, preventing uncommitted request-scoped state from leaking into the log write.
• End-user and built-in tool services — api/services/end_user_service.py lines 27, 57, 136 use this pattern for all reads/writes.

The problems this pattern fixes are documented in issues #37886 (premature commit in _on_query) and #37885 (premature commit in the rate-limit decorator).

Common Failure Modes and Fixes#

AccountService.get_account_by_email_with_case_fallback#

Previously created its own session_factory, detaching ORM instances from the caller's transaction and triggering DetachedInstanceError in auth flows. PR #37847 changed the signature to accept session: Session | scoped_session and removed the internal factory; 9+ call sites across controllers and CLI were updated.

Key Source Files#

Tracking issue: #37403 — make db.session pass from parameter is the umbrella for the ongoing refactoring across 150+ files.