Azure Pipelines VSCode Extension

[ChristopherHX]

Azure Pipelines VSCode Extension

This is a minimal Azure Pipelines Extension, the first vscode Extension which can Validate and Expand Azure Pipeline YAML files locally without any REST service.

Features

Remote Template References

The azure-pipelines.repositories settings maps the external Repositories to local or remote folders.

Syntax [<owner>/]<repo>@<ref>=<uri> per line. <uri> can be formed like file:///<folder> (raw file paths are not supported (yet?)), vscode-vfs://github/<owner>/<repository> and vscode-vfs://azurerepos/<owner>/<project>/<repository>

Validate Azure Pipeline

> Validate Azure Pipeline

This command tries to evaluate your current open Azure Pipeline including templates and notifies you about the result.

Once this extension has been activated by any command, you can validate your pipeline via a statusbar button with the same name on all yaml or azure-pipelines documents

Expand Azure Pipeline

> Expand Azure Pipeline

This command tries to evaluate your current open Azure Pipeline including templates and show the result in a new document, which you can save or validate via the official api.

Azure Pipelines Debug Adapter

Sample Debugging configuration

{
    "type": "azure-pipelines-vscode-ext",
    "request": "launch",
    "name": "Test Pipeline (watch)",
    "program": "${workspaceFolder}/azure-pipeline.yml",
    "repositories": {},
    "parameters": {},
    "variables": {},
    "watch": true,
    "preview": true
}

Pros

• Make changes in multiple dependent template files and show a live preview on save
• Everything is done locally
• Whole template engine is open source
• You can run template files with the same template engine locally using the Runner.Client tool using the official Azure Pipelines Agent
• Fast feedback
• Less trial and error commits
• You can help by reporting bugs
• It's fully Open Source under the MIT license

Cons

• May contain different bugs than the Azure Pipelines Service
• You could self-host Azure Devops Server and commit your changes to your local system, may have license implications with more accurate results of the template engine
• May not have feature parity with Azure Pipelines
• Missing predefined Variables, feel free to add them manually as needed

Further Discussions about how to improve this extension are welcome

[ChristopherHX]

@bryanbcook

What do you think about the fake debug adapter of the vscode extension? See the gif of the first post for a preview.

It allows you to create multiple launch configs with different parameters and variables.
Custom entrypoint yaml file, doesn't change if you edit a different file.
The global variables / repos are overidden by variables/repos of the same name in launch config.
The global parameters are replaced if defined by launch config.

[bryanbcook]

It's interesting, though I don't know if I was a first-time user of the extension that I would think to use the Debugger as a way of providing a live preview of template expansion. In my mind, the Debugger is something I would use to "step through" the pipeline or to run my pipeline with a debugger attached.

I like the way the vscode-markdownlint extension does this. They keep track of a list of markdown violations as a collection of diagnostics and use a set of filesystem watchers for the workspace. When it detects files have been modified it triggers validation only for the open editor files.

For this extension, because the IFileProvider is responsible for the resolution of all templates, you could build a list of associated files per pipeline evaluation. You could then use the file system watcher to re-evaluate whenever these files are modified.

[ChristopherHX]

You are correct, debug adapter are not really meant for starting and stopping file watching..

I actually had the idea to extend the debug adapter to be able to stop on the error location and allow to manipulate context variables of subtemplates. Or step into a template and view the yaml preview of the not yet merged child template.
Building a real debug adapter needs a lot of support from the template engine, which doesn't exist yet.

How would you provide different sets of variables of the same workflow file? (debug configs makes this trivial)

• specify the filename in the config?
• use a complex json settings object to include all possible configurations?
• use a custom special file in the workspace like markdownlint? (e.g. yaml / json)

How would the extension know if it needs to watch the file currently opened? (debug configs are not automatically started, so impact on other files is minimal)
For markdownlint it's easy, markdown... Not all yaml files are pipelines.
Especially if the yaml syntax is violated / unique azure pipelines keys are missing.
Or only start if the user pressed a button to activate the file and it's dependencies?
Or remember if it was a valid pipeline file in some sort of settings and keep linting these files on any error?

If the extension requires azure-pipelines as file type, then it would depend on the official azure pipelines extension. I'm not shure if this extension can polyfill it (yaml formatter don't like that and are unavailable by default).

For this extension, because the IFileProvider is responsible for the resolution of all templates, you could build a list of associated files per pipeline evaluation. You could then use the file system watcher to re-evaluate whenever these files are modified.

Yes this can be used to improve performance, currently I only set up a watcher for all yml/yaml files.
If the extension always watches for yaml file changes this would really be needed to not overload the ide.

[bryanbcook]

Great questions. Here are my views...

Let's start with settings. Some of these settings might make sense at the user or workspace level, assuming that most users of the extension are going to be working in a single Azure DevOps Organization with well-known repositories.

settings.json:

{
   "azure-pipelines-vscode-ext.repositories": [
      "project/repo@ref:path"
   ]
}

Regarding parameters and variables per workflow/pipeline, that's an interesting problem. In my view, you shouldn't need parameters or variables to be able to validate a pipeline (parameters that don't have a default value can never be used in a pull request and generally should be avoided). However, because the extension provides template expansion and parameters can influence conditional insertion, there needs to be a way to opt-in to provide them if desired. As not all variables are available in templates only certain variables should be included during the evaluation. (This also suggests that there should be dummy values for static-variables if they're not provided...)

As these settings for the pipelines would only be applicable at the repository or workspace level, a dedicated file might make sense. The extension could have a setting scoped to the workspace that has the default location of the dedicated file and attempt to resolve values for the pipeline at evaluation time.

If the pipeline.json doesn't contain a definition for the selected pipeline, it should assume default values.

settings.json:

{
   "azure-pipelines-vscode-ext.pipelinesFile": ".vs/pipelines.json"
}

.vs/pipelines.json:

{
  "pipelines": {
     "/folder/path/to/pipeline.yml": {
         "parameters": {
             "myParameter": "value"
          },
         "variables": {
             "build.definitionname": "my-pipeline"
         }
     }
   }
}

Regarding different settings for Pull Requests vs Manual vs IndividualCI builds, the only thing that changes is the Build.BuildReason and a few other static variables like System.PullRequest.TargetBranch and System.PullRequest.SourceBranch. These could be accomplished with different commands > Expand Azure Pipeline (CI) and > Expand Azure Pipeline (Pull Request). The pull-request command could prompt for the source and target branches. The extension should assume that > Expand Azure Pipeline is a Manual build-reason and the Build.SourceBranch could be inferred from the current git repo or a default workspace-scoped setting.

settings.json:

{
   "azure-pipelines-vscode-ext.inferbranchnamefromgitrepo": false,
   "azure-pipelines-vscode-ext.defaultbranchname": "refs/heads/feature/myfeature"
}

Regarding which files to watch, these are gathered from the IFileProvider when you run the > Expand Azure Pipeline|> Validate Azure Pipeline command. This command would require that the active document is a pipeline/workflow. When evaluating a pipeline, the extension could create and maintain an "evaluationRun" that contains the pipeline, the impacted files identified by the file-provider, a collection of diagnostics obtained from any resulting error messages, plus any other necessary meta-data (eg reference to the preview window if there is one, pull-request branch names, etc). You could dispose the objects in the evaluationRun when the associated document for the workflow/pipeline is closed.

In the background, the onDidSaveTextDocument could be used as a trigger to locate the document in the list of impacted files in the evaluationRun and re-evaluate the associated pipeline(s).

Regarding file-system watchers, on closer inspection of the vscode-markdownlint extension, they are only looking at configuration files and re-evaluating documents if the config changes. It might be valuable to have the extension re-evaluate pipelines if the config contains a definition for a pipeline that is in the evaluationRun set.

[ChristopherHX]

I'm still experimenting with the extensions api, but it doesn't seem easy for a webextension to get access to the git branch...
A node extension could ask the vscode.git extension about it, but due to isolations (only an restricted api proposal allows such communications between extension hosts, but I'm notshure if that usecase would work it).

However, because the extension provides template expansion and parameters can influence conditional insertion, there needs to be a way to opt-in to provide them if desired

Conditionally included template files are another possible reason. e.g. a each loop of template file names constucting template includes.

steps:
- ${{ each t in split('file1.yml,file2.yml,file3.yml', ',') }}:
  - template: ${{ t }}

Azure Pipelines has powerful templating, easy to get wrong. (Only /pipeline.yml: File /file1.yml not found in repository is shown, because the file is really missing)

Regarding which files to watch, these are gathered from the IFileProvider when you run the > Expand Azure Pipeline|> Validate Azure Pipeline command. This command would require that the active document is a pipeline/workflow. When evaluating a pipeline, the extension could create and maintain an "evaluationRun" that contains the pipeline, the impacted files identified by the file-provider, a collection of diagnostics obtained from any resulting error messages, plus any other necessary meta-data (eg reference to the preview window if there is one, pull-request branch names, etc). You could dispose the objects in the evaluationRun when the associated document for the workflow/pipeline is closed.

Yes it makes sense to stop file watching if the preview pane is closed via the Expand Azure Pipeline command.

I was initially not able to understand how to detect if the preview has been closed, because of unclear documentation and old open questions on e.g stackoverflow. Currently I register an event handler to the tab changed event, but maybe during moving the preview pane to another tab group etc. might cause the close event to happen too early with a open event shortly after. The textdocument close event might be too late (or it isn't ?), sometimes takes 10s or more of seconds to get triggered.

[ChristopherHX]

For multiple parameter combination, I will shift towards Tasks.

Tasks are meant to be used for linter tasks like template validation. You can configure multiple tasks to run parallel in the background, each has it's own log.

Soon the vscode extension will provide diagnostic error messages to vscode pointing to the error location.

There are still some errors with missing file locations, those won't be mapped yet.

In the best case all errors are merged and you will see what callers you had broke by the current change.

Another useful detail for errors is the exact path to the error, e.g. a nested template might be called multiple times...