Building an OutSystems pipeline with Azure DevOps Multistage Pipeline - OutSystems/outsystems-pipeline GitHub Wiki

This guide provides a step-by-step description of how to implement a recommended continuous delivery pipeline for OutSystems applications by leveraging the LifeTime Deployment API along with Azure DevOps.

Please confirm that you have the following prerequisites before following the steps described in this guide:

• Make sure you have the necessary rights to configure pipelines in your Azure DevOps project.

• Make sure that LifeTime is able to make calls to Azure DevOps Services REST API.

• At least one Azure Pipelines agent with:

  • Python 3 installed

  • Access to PyPI.org

  • HTTPS connectivity with LifeTime

  • HTTPS connectivity with the front-end servers that are going to run test suites

Note:

If you are using self-hosted Windows OS agents, make sure that you also install the pip package installer.
Also make sure to activate the option "Add Python to PATH" in the installation wizard.

In Azure, variable groups are a convenient way to make sets of reusable variables available across multiple pipelines.
In this guide, we'll create and use a single variable group that will provide all infrastructure context to run the pipelines.

From the Azure Pipelines Dashboard, open the "Library" tab to see a list of existing variable groups for your project and choose "+ Variable group".

You need to create a LifeTime service account and generate an authentication token to authenticate all requests to the LifeTimeAPI.

From the variable group previously created, select "+ Add" and provide the following configuration values:

• Name: LifeTime.ServiceAccountToken
• Value: <your LifeTime authentication token>

Click on the "lock" icon at the end of the row to encrypt and securely store the value. (The values of secret variables are stored securely on the server and cannot be viewed by users after they are saved. After being encrypted the values can only be replaced.)

For a smooth run of the pipeline, you'll need to provide some context about your OutSystems factory global values to be used by the pipelines.

From the variable group previously created, select "+ Add" and provide the following configuration values:

Name	Value	Example value
Agent.VMImage	<Microsoft-hosted agent image>	ubuntu-latest
LifeTime.Hostname	<Hostname of LifeTime environment>	lifetime.example.com
LifeTime.APIVersion	<Version of LifeTime Deployment API to use>	2
Environment.Development.Label	<Name of Development environment as defined on LifeTime>	Development
Environment.Regression.Label	<Name of Regression environment as defined on LifeTime>	Regression
Environment.Acceptance.Label	<Name of Acceptance environment as defined on LifeTime>	Acceptance
Environment.PreProduction.Label	<Name of Pre-Production environment as defined on LifeTime>	Pre-Production
Environment.Production.Label	<Name of Production environment as defined on LifeTime>	Production
Environment.Regression.Key	<Id of the Azure DevOps Environment for Regression>	reg-env
Environment.Acceptance.Key	<Id of the Azure DevOps Environment for Acceptance>	acc-env
Environment.PreProduction.Key	<Id of the Azure DevOps Environment for Pre-Production>	pre-env
Environment.Production.Key	<Id of the Azure DevOps Environment for Production>	prd-env
Artifacts.Folder	<Folder that will store all pipeline outputs>	Artifacts
Manifest.File	<Name specification for the compressed pipeline manifest output>	trigger_manifest.json
Manifest.Folder	<Folder that will store the manifest outputs>	trigger_manifest
OSPackage.Version	<Outsystems Python package version>	0.8.0

You may additionally add the following configuration values depending on your integrations; choose the ones that apply to you:

Name	Value	Example value
CICDProbe.EnvironmentURL	<URL of environment where CI/CD probe is deployed>	https://regression-env.example.com/
BDDFramework.EnvironmentURL	<URL of environment where BDD tests will run automatically>	https://regression-env.example.com/
AIMentorStudio.ActivationCode	<OutSystems License Activation Code>	AAA.AAA.AAA.AAA.AAA.AAA.AAA.AAA
AIMentorStudio.APIKey	<API key to authenticate request>
AIMentorStudio.Hostname	<Hostname of AI Mentor Studio>	aimentorstudio.outsystems.com
AIMentorStudio.Folder	<Name specification for the AI Mentor Studio outputs>	techdebt_data
AIMentorStudio.Thresholds.SecurityFindingsCount	<Microsoft-hosted agent image>	10
AIMentorStudio.Thresholds.TechDebtLevel	<Hostname of LifeTime environment>	Medium
Python.Version	<Python version to run scripts>	3.10

Our current approach is leveraging the Azure DevOps Environments capabilities to handle pipeline resources such as approvals, deployment window time, and exclusive locks.

For each OutSystems environment in your delivery pipeline (except Development), create a matching Azure Environment and ensure that the Environment name matches the value in the corresponding Environment.*.Key variable.

Each Azure Environment is associated with a pipeline stage, which references it in the YAML pipeline definition through a Deployment Job, thus allowing the pipeline to leverage the capabilities of the Azure Environment.

The easiest way to do this is by providing a YAML file containing the multistage pipeline definition. The template YAML files for the OutSystems continuous delivery pipeline are provided here. As the given template is built on Azure DevOps templates, which define the base functions for the pipeline to run, make sure you also copy all the provided file structure.

It is highly advisable to store your template YAML files using a version control system such as Git, as this ensures that you are able to keep track of any changes made to your pipeline definition going forward. Additionally, any other supporting artifacts that you may need to run your pipeline can also be stored in a single location alongside the YAML files and synced to the pipeline workspace folder on every run.

Note:

You may want to edit the YAML template to match the name of the previously created Variable Group.
If you're using self-hosted agents you may also want to edit the YAML template to define the pool name.

From the Azure Pipeline Dashboard, navigate to the Pipelines tab, under the Pipelines page, click New pipeline, and select the version control system (Source) where you've previously stored the YAML file.

Once you have configured the version control system, choose Existing Azure Pipelines YAML file, select the template YAML file, and click Continue.

Review Pipeline YAML and click Save from the dropdown.

Click the Edit button, select Triggers from the More actions submenu, and rename the pipeline to "<YourProduct>-Pipeline".

Properties Services allows you to get and set Site Properties, Timers, REST References, and SOAP Reverences from the pipelines.

To install the Properties Services, download the Properties Services and publish it on every environment that you'd like to have this functionality using Service Center. Alternatively, you can install the component directly from the Service Studio interface.

Properties Management provides the APIs to externally use the Properties Services component.

To install the Properties Management, download the Properties Management and publish it on the LifeTime environment using Service Center. Alternatively, you can install the component directly from the Service Studio interface.

To retrieve environment-specific information that is required when running the continuous delivery pipeline, the CI/CD Probe Forge component must be installed on the Regression environment of your deployment pipeline.

To install the CI/CD probe, download the CI/CD Probe matching your Platform Server version and publish it on the Regression environment using Service Center. Alternatively, you can install the component directly from the Service Studio interface.

The Trigger Pipeline LifeTime plugin available on the OutSystems Forge automatically detects when new versions (tags) are available for a configured subset of LifeTime applications and triggers an associated Azure DevOps pipeline by leveraging the Azure DevOps REST API.

To install the Trigger Pipeline LifeTime plugin, download the latest version of the Trigger Pipeline and publish it to your LifeTime environment using Service Center. Alternatively, you can install the component directly from the Service Studio interface.

Configure the Trigger plugin module in your Lifetime Service Center using the following steps.

Access the integrations tab and configure the following REST APIs:

• LifeTimeAPI: https://your-lifetime-example.com/lifetimeapi/rest/v2
• PropertiesAPI: https://your-lifetime-example.com/PropertiesAPI/rest/v1

Access the Site Properties tab and configure the following properties:

• Feature_UseTriggerManifest: True
• ServiceAccountToken: <your LifeTime authentication token>

Note:

If you wish to tag changes on the fly when triggering the pipeline, the Service Account must have at least Change & Deploy permissions on the target environment.

After the plugin is successfully published and configured in the LifeTime environment, select Configure Triggers from the plugin landing page in LifeTime and configure the following parameters:

• Source Environment: <Select your OutSystems source environment.>
• Pipeline Server Type: Azure DevOps
• Pipeline Server Address: <Your Azure Pipeline instance address, including organization and project. For example, https://dev.azure.com/{organization}/{project}.>
• Pipeline Server Credentials: <Credentials of an Azure user account, including the username and a personal access token with sufficient permissions for running pipeline jobs. We recommended creating a dedicated user or service account for this purpose.>

Pipeline triggers can be configured by providing the following data:

• Pipeline: <Unique name that identifies the pipeline in Azure DevOps>
• Pipeline Type: <Select YAML for Multistage Pipeline>
• Source Environment: <Select your OutSystems source environment.>
• Environment Labels (optional): <Define the label assigned to each OutSystems environment along the deployment path>
• Applications: <List of LifeTime applications that will trigger the CI/CD pipeline, identifying which ones are Test applications>

After the Trigger Pipeline plugin is properly configured, the dashboard screen will show the list of pipeline triggers, along with the current versions in the Development of the LifeTime applications defined for each pipeline scope.

Once there are new application versions available for triggering a pipeline, a button is shown that allows running the pipeline on-demand without the need to log in to Azure.