Azure DevOps Integration#

Overview#

The azuredevops_go plugin collects engineering data from Azure DevOps into DevLake's domain layer — covering pull requests, commits, CI/CD builds and pipeline jobs (timeline records), and user accounts.

Key locations:

File	Purpose
`azuredevops.go`	Plugin entry point, exports `PluginEntry`
`impl/impl.go`	Full plugin implementation (interfaces, init, subtask list)
`models/connection.go`	Connection + auth model
`models/repo.go`	Scope model (`AzuredevopsRepo`)
`tasks/task_data.go`	Task options and params (org/project/repo IDs)
`api/azuredevops/client.go`	Internal API client used during connection testing
`tasks/shared.go`	Shared helpers: pagination, error handlers, status/result rules

The primary scope unit is an AzuredevopsRepo, stored in _tool_azuredevops_go_repos. Connection records are stored in _tool_azuredevops_go_connections .

Authentication (PAT Tokens)#

The plugin authenticates using Personal Access Tokens (PATs) via HTTP Basic Auth. The SetupAuthentication method encodes :<token> (empty username, token as password) in Base64 and sets the Authorization: Basic <b64> header on every request — the standard Azure DevOps PAT format.

The token field uses GORM's encryption serializer (gorm:"serializer:encdec") for at-rest encryption . The Sanitize() method masks the token for logging/display .

AzuredevopsConn embeds the token struct and adds:

Organization — the Azure DevOps org name
Proxy — optional HTTP proxy
Endpoint hardcoded to https://dev.azure.com

Rate limiting is not configured (GetRateLimitPerHour returns 0) .

Repository Identifier Hierarchy#

All Azure DevOps API calls are scoped to a three-level hierarchy: Organization → Project → Repository.

These IDs flow through AzuredevopsOptions:

OrganizationId / ProjectId / RepositoryId

AzuredevopsParams uses the same triple as the deduplication key for raw data. The URL template in the PR collector shows the pattern directly:

{{ .Params.OrganizationId }}/{{ .Params.ProjectId }}/_apis/git/repositories/{{ .Params.RepositoryId }}/pullrequests

The internal client enforces the same structure:

GetProjects: {OrgId}/_apis/projects
GetRepositories: {OrgId}/{ProjectId}/_apis/git/repositories
GetServiceEndpoints / GetRemoteRepositories: {OrgId}/{ProjectId}/_apis/...

AzuredevopsOptions also carries RepositoryType and ExternalId for linking external (e.g., GitHub-backed) repositories .

HTTP Error Code Handling#

Azure DevOps has non-standard auth failure behavior that the plugin explicitly handles.

203 → 401 remapping. When a PAT is invalid, Azure DevOps issues an HTTP 302 redirect to a sign-in page, which ultimately resolves as 203 Non-Authoritative Information. The change203To401 function intercepts this and returns errors.Unauthorized — the same error raised for a direct 401. This function is wired as AfterResponse in collectors (e.g., the PR collector).

The same 203/401 pattern is checked directly in the internal API client for GetUserProfile and GetProjects .

Other error mappings:

HTTP Status	Context	Mapped To
401 or 302	`GetUserAccounts`	`errors.Unauthorized`
401 or 403	`GetRepositories`, `GetServiceEndpoints`	`errors.Unauthorized`
404	`GetRepositories`, `GetServiceEndpoints`	`errors.NotFound`
404	Build timeline collector	`api.ErrIgnoreAndContinue` — deleted builds are silently skipped

Task Pipeline#

The plugin registers 20 subtasks following a collect → extract → convert pattern . Each task file registers itself via init() / RegisterSubtaskMeta().

Entity	Collector	Extractor	Converter
Pull Requests	`collectApiPullRequests`	`extractApiPullRequests`	`convertApiPullRequests`
PR Commits	`collectApiPullRequestCommits`	`extractApiPullRequestCommits`	`convertApiPullRequestsCommits`
PR Labels	—	—	`convertPrLabels`
Commits	`collectApiCommits`*	`extractApiCommits`*	`convertApiCommits`
Builds	`collectApiBuilds`	`extractApiBuilds`	`convertApiBuilds` → `cicd_pipelines`
Timeline Records	`collectApiTimelineRecords`	`extractApiTimelineRecords`	`convertApiTimelineRecords` → `cicd_tasks`
Accounts	`collectAccounts`	`extractAccounts`	`convertAccounts`
Repo	—	—	`convertRepo` → `repos` + cicd scope

Commit collection/extraction is disabled by default.

Pagination uses two strategies :

Offset-based ($skip / $top): used for most endpoints
Cursor-based (X-Ms-Continuationtoken response header): used for remote repository listing

The API version is pinned to 7.1 and the max page size is 100 .

CI/CD status and result normalization rules are defined in shared.go and map Azure DevOps build states (e.g., succeeded, failed, inProgress) to DevLake's devops.ResultRule / devops.StatusRule domain types.