Azure DevOps Build Data Extraction#

DevLake has two implementations of the Azure DevOps plugin — a Go plugin (azuredevops_go) and a Python plugin (azuredevops) — both of which collect Azure DevOps Build API responses and produce the same CICD domain records. The pipeline is: collect → extract/parse → convert into devops.CICDPipeline + devops.CiCDPipelineCommit domain rows.

Go Plugin (azuredevops_go)#

Step 1 – API shape (AzuredevopsApiBuild).
The Go struct AzuredevopsApiBuild in models/build.go is the raw JSON representation of the Build API response. It captures:

requestedBy and requestedFor are not declared in this struct and are not extracted. The raw JSON fields are silently dropped during unmarshal.

Step 2 – Tool layer record (AzuredevopsBuild).
ExtractApiBuilds in ci_cd_build_extractor.go unmarshals each raw row into AzuredevopsApiBuild, then writes a subset of fields to the tool table _tool_azuredevops_go_builds via AzuredevopsBuild. The Tags array is JSON-serialized to a string at this step 1; it is later used as input for environment/type regex enrichment.

Step 3 – Domain layer records.
ConvertBuilds in ci_cd_build_converter.go joins _tool_azuredevops_go_builds with the repo table to attach the URL, then creates two domain records per build 2:

• devops.CICDPipeline — carries Name, normalized Result/Status (plus OriginalResult/OriginalStatus), DurationSec, timestamps (CreatedDate, QueuedDate, StartedDate, FinishedDate), CicdScopeId, and regex-enriched Environment / Type derived from build.Name + ";" + build.Tags 3.
• devops.CiCDPipelineCommit — carries CommitSha (from SourceVersion), Branch (from SourceBranch), RepoId, and RepoUrl 4.

Python Plugin (azuredevops)#

Tool model.
The Python Build in models.py maps API fields via pydevlake's Field(source=...) path syntax. Notable field paths:

• name ← /definition/name
• queue_time ← /queueTime
• display_title ← /triggerInfo/ci.message
• url ← /_links/web/href

Like the Go model, neither requestedBy nor requestedFor appear in the Python Build model 5.

Conversion.
The Builds stream in streams/builds.py emits the same two domain types: devops.CICDPipeline 6 and devops.CiCDPipelineCommit 7. Status/result normalization is inline rather than rule-table driven. Type and Environment are set by matching deployment_pattern and production_pattern regexes from the scope config against the build name 8.

Jobs as CICDTask.
Azure DevOps timeline records (jobs) are collected as a sub-stream of Builds in streams/jobs.py and converted to devops.CICDTask, linked to the parent pipeline via the build ID. The Go plugin does the equivalent in ci_cd_timeline_records_converter.go 9.

Notable Gaps: requestedBy / requestedFor#

The Azure DevOps Build API returns two identity objects relevant to triggering:

• requestedBy — the identity that triggered the build.
• requestedFor — the identity on whose behalf the build runs (often the PR author).

Neither field is captured in either plugin implementation. They are absent from AzuredevopsApiBuild 10 and from the Python Build tool model 5. As a result, devops.CICDPipeline records produced by the plugin do not carry triggering-user information. Engineers adding identity mapping for builds would need to:

1. Add RequestedBy / RequestedFor identity struct fields to AzuredevopsApiBuild (Go) or Field(source=...) entries to Build (Python).
2. Persist them in the tool-layer table (_tool_azuredevops_go_builds or equivalent).
3. Map to the appropriate domain field in the converter (e.g., a TriggeredBy or account ID field if the domain model supports it).

Key File Index#