GitHub Deployment Data Collection (GraphQL)#
DevLake collects GitHub deployment data via the github_graphql plugin using three subtasks: Collect Deployments, Extract Deployments, and Convert Deployments. The pipeline stores raw JSON into a raw table, extracts it into the _tool_github_deployments tool-layer table, then converts to the CICD domain layer.
Entry points:
- Collector:
deployment_collector.go - Extractor:
deployment_extractor.go - Converter:
deployment_convertor.go
The raw table name is github_graphql_deployment .
GraphQL Fields Requested#
The collector's GraphqlQueryDeploymentDeployment struct defines exactly what is requested from the GitHub GraphQL API :
| Field | Notes |
|---|---|
task | Usually "deploy"; value not always guaranteed |
id | GraphQL node ID |
databaseId | Numeric REST API ID |
commitOid | Commit SHA being deployed |
environment | Target environment name |
state | Deployment state (e.g., ACTIVE, SUCCESS, INACTIVE) |
description | Optional description string |
payload | Arbitrary JSON payload |
ref.id, ref.name, ref.prefix | Branch/tag reference |
latestStatus.id, .state, .updatedAt | Most recent deployment status |
repository.id, .name, .url | Owning repository |
createdAt, updatedAt | Timestamps |
commit.oid, .message, .author.name, .author.email, .committedDate | Commit metadata |
Deployments are fetched in pages of 100, ordered by CREATED_AT DESC . Incremental collection is supported: collection stops early if UpdatedAt is not newer than the since watermark .
The exact GraphQL query sent to the API is visible in the e2e raw data fixture.
Tool-Layer Storage (_tool_github_deployments)#
The GithubDeployment model stores the following columns:
| Column | Type | Source |
|---|---|---|
connection_id | uint64 (PK) | Task context |
id | varchar(255) (PK) | GraphQL node ID |
github_id | int | Repo ID |
display_title | string | First line of commit message |
url | <repo_url>/deployments/<environment> | Constructed |
database_id | uint | databaseId |
commit_oid | varchar(255) | commitOid |
description | varchar(255) | description |
environment | varchar(255) | environment |
state | varchar(255) | state |
latest_status_state | varchar(255) | latestStatus.state |
latest_status_update_date | *time.Time | latestStatus.updatedAt |
repository_id/name/url | varchar(255) | repository.* |
ref_name | varchar(255) | ref.name |
payload | text | payload |
created_at, updated_at | time.Time | createdAt, updatedAt |
Note the DisplayTitle is derived from the commit message, not from a GitHub deployment title field .
Domain-Layer Conversion#
The converter maps GithubDeployment → devops.CicdDeploymentCommit + devops.CICDDeployment :
- Result: derived from
State—SUCCESS/ACTIVE/INACTIVE→ success;ERROR/FAILURE→ failure - Status:
QUEUED/WAITING/PENDING/INPROGRESS→ in-progress; all others → done - Duration:
UpdatedDate - CreatedDatein seconds - Environment override: if the environment name matches a user-configured regex pattern, it is reclassified as
PRODUCTION StartedDate: set toCreatedDate(no separate started timestamp)
Notable Gaps: Missing Creator/Actor Identity#
The GraphQL query does not request the creator field on the Deployment object, and the GithubDeployment model has no column for the deploying user's identity (login, ID, or account node ID). As a result:
- There is no
creator_id,creator_login, or equivalent in the tool-layer model . - The domain-layer
CicdDeploymentCommitis produced without any triggering user linkage. - The only person-related data available is the commit author (name + email), fetched via
commit.author— this is the author of the deployed commit, not necessarily who triggered the deployment.
The GitHub GraphQL Deployment type does expose a creator field (a User object with login, id, etc.), but it is not included in GraphqlQueryDeploymentDeployment. Adding it would require extending that struct and the GithubDeployment model with a new migration.