SQLAlchemy ORM Migration#

Dify's ORM migration modernizes api/models/ to SQLAlchemy 2.0 standards — replacing legacy db.Column() definitions and untyped mapped_column() calls with Mapped[T] type annotations and MappedAsDataclass-backed models (TypeBase). The effort runs in two phases:

Column-definition migration — db.Column(...) → mapped_column(...), bare columns → Mapped[T] = mapped_column(...). This phase is complete across all model files.
Base-class migration — class Foo(Base) → class Foo(TypeBase), gaining Python dataclass semantics. Approximately 76 of ~101 model classes had been converted as of mid-2026, with the remainder concentrated in model.py and dataset.py.

Tracking issues: #22652 , #23579, and continuation #38564.

Key Abstractions (`api/models/base.py`)#

All foundational base classes and mixins live in api/models/base.py.

Symbol	Inherits from	Purpose
`Base`	`DeclarativeBase`	Legacy base — still used by unconverted models
`TypeBase`	`MappedAsDataclass, DeclarativeBase`	Target base for all models; generates a typed `__init__`
`DefaultFieldsMixin`	—	For `Base` models; provides `id`, `created_at`, `updated_at` via `default=callable`
`DefaultFieldsDCMixin`	`MappedAsDataclass`	For `TypeBase` models; same three fields but uses `insert_default`, `default_factory`, and `init=False`

TypeBase's inline docstring reads "after all finished, rename to Base" — it's a migration staging ground, not a permanent name .

Why two mixin variants? MappedAsDataclass models process mapped_column() parameters differently: default=callable is not valid; instead insert_default (DB INSERT) + default_factory (Python __init__) must be used. DefaultFieldsDCMixin was introduced in PR #34686 to unblock per-model Base → TypeBase conversions.

Migration Pattern#

Column annotations#

Before (no type annotation or legacy Column):

# Legacy
created_at = db.Column(db.DateTime, nullable=False, server_default=func.current_timestamp())
# Or: untyped mapped_column
description = mapped_column(LongText, nullable=True)

After (PR #37449) :

created_at: Mapped[datetime] = mapped_column(sa.DateTime, nullable=False, ...)
description: Mapped[Optional[str]] = mapped_column(LongText, nullable=True)

Standard type mapping :

Column Type	Python Type (non-null)	Nullable
`StringUUID` / `String` / `LongText` / `Text`	`str`	`str \| None`
`DateTime`	`datetime`	`datetime \| None`
`Integer`	`int`	`int \| None`
`Boolean`	`bool`	`bool \| None`
`JSONB` / `AdjustedJSON`	`dict`	`dict \| None`
`JSONModelColumn(T)`	`T`	`T \| None`

`TypeBase` (dataclass) conversion#

When converting a class from Base to TypeBase:

Change inheritance: class Foo(Base) → class Foo(TypeBase) (or class Foo(DefaultFieldsDCMixin, TypeBase))
Replace default=callable with insert_default=callable, default_factory=callable
Add init=False to auto-populated fields (id, created_at, updated_at)
Ensure all other fields have a default or default_factory or appear before fields that do (dataclass ordering rules)

See Account in account.py for a fully-converted example, including per-field init=False and default_factory declarations. Per-model TypeBase migrations include PR #34806 , PR #34807 , and PR #34808 .

CI Automation via AST-grep#

Autofix rules in .github/workflows/autofix.yml run on every push when API files change. They apply the following rewrite rules automatically via uvx ast-grep:

Rule	Pattern	Rewrite
`.filter()` → `.where()`	`db.session.query($X).filter($Y)`	`db.session.query($X).where($Y)`
`db.Column` → `mapped_column`	`$A = db.Column($$$B)`	`$A = mapped_column($$$B)`
Typed `db.Column`	`$A : $T = db.Column($$$B)`	`$A : $T = mapped_column($$$B)`
`Optional[T]` modernization	inline YAML rule matching `generic_type`	`$T \| None`
Forward-ref fix	quoted type `"T" \| None` (from above)	`Optional["T"]` (restored via `sed`)

The rules were first introduced in PR #26262 to prevent regression of migrated code. The workflow uses autofix-ci/action to commit the automated changes back to pull requests .

Progress tracking: api/cnt_base.sh counts (Base): vs (TypeBase): class declarations (excluding .venv and tests) and is invoked from the same workflow to surface migration progress on each PR.

Current State & Key Files#

Phase	Status
`db.Column()` → `mapped_column()`	✅ Complete — CI autofix prevents regressions
Bare `mapped_column()` → `Mapped[T]`	✅ Complete (81 fields addressed in PR #37449)
`Base` → `TypeBase`	🚧 ~76/101 classes — remaining in `model.py` (~15) and `dataset.py` (~8)

Key files:

File	Role
`api/models/base.py`	`Base`, `TypeBase`, `DefaultFieldsMixin`, `DefaultFieldsDCMixin`
`api/models/account.py`	Fully migrated example (`Account` class on `TypeBase`)
`api/models/model.py`	Largest remaining migration target
`api/models/dataset.py`	Second remaining migration target
`.github/workflows/autofix.yml`	AST-grep autofix + migration progress reporting
`api/cnt_base.sh`	Counts `Base` vs `TypeBase` class declarations

Ongoing work is tracked in #38564 (continuation of #23579).