Versions Compared

Old Version 3

changes.mady.by.user Amy Smith

Saved on

compared with

New Version 4

changes.mady.by.user Amy Smith

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This document explains how to setup an Azure pipeline with an agent to execute Liquibase Enterprise operations. An Azure DevOps agent can be shared among multiple projects, so one agent can serve several teams. You can learn more about Azure agents here: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser.

Instructions

Table of Contents

Prerequisites

Before following the steps in this document, setup your databases and create the Liquibase project.

...

Code Block
git init --initial-branch=main
git remote add origin git@ssh.dev.azure.com:v3/asmith0101/Liquibase/ABC_DDB
git push -u origin --all
git add .
git commit -m "Initial commit"
git push -u origin main

Step 1: Setup an Azure Agent Pool

...

Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops

An Azure Agent Pool can be created under Project Settings. Agent pools are shared across an organization.

...

Step 2: Create an Azure Agent

Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops

After setting up an Agent Pool you can create a New agent. Instructions will be provided for the type of agent (Windows , macOS, or Linux) that you wish to use.

...

Step 2: Setup an Azure Key Vault with the database credentials

Instructions: Note there are also Detailed Instructions for each of the types. Make sure to use a supported version of Windows or Linux. Hardware, Software, and Database Requirements

...

Step 3: Setup authentication from the Agent to the Repositories

This example uses SSH keys for git authentication. You can find information on using SSH keys with Azure at https://learn.microsoft.com/en-us/azure/devops/pipelinesrepos/releasegit/keyuse-vaultssh-inkeys-ownto-projectauthenticate?view=azure-devops&tabs=portal

The Azure Key Vault is a secure method for storing database credentials. It is recommended that you add Key Vault secrets for all databases in your pipeline including the DMC database.

In this example, Secrets added in the Key Vault will be referenced by the pipeline using Variable Groups.

...

Step 3: Setup an Azure Library Variable Group

Instructions: . Copy the SSH private key your ~/.ssh folder.

If you wish to use HTTPS for git authentication you will need to setup Git Credential Manager. Instructions can be found at https://learn.microsoft.com/en-us/azure/devops/pipelinesrepos/library/git/set-up-credential-managers?view=azure-devops and https://learn.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml

Use an Azure Library to group credentials from your Key Vault into a Variable Group. This grouping makes it easy to associate credentials to Azure Pipelines and Releases.

...

Step 1: Install Liquibase Enterprise on an Agent

The agent can be a VM on Azure or other cloud platform. Instructions to setup an Azure VM can be found at https://learn.microsoft.com/en-us/azure/devops/pipelines/process/environments-virtual-machines?view=azure-devops. Make sure to use a supported version of Windows or Linux. Hardware, Software, and Database Requirements

Follow these instructions to install Liquibase Enterprise:

Make sure that git is installed on the agent as it will be needed by the Liquibase Enterprise Deploy Packager. SCM Requirements and Features

Step 2: Install GitLab Runner on the Linux Server

https://docs.gitlab.com/runner/install/linux-repository.html

Login to the Linux VM and run the following commands.

Code Block
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash

sudo yum install gitlab-runner

This example uses SSH keys for git authentication. You can find information on using SSH keys with GitLab here: https://docs.gitlab.com/ee/user/ssh.html . Copy the SSH private key to /home/gitlab-runner/.ssh.

Step 3: Register the Runner

https://docs.gitlab.com/runner/register/index.html

  1. Login to GitLab.

  2. Navigate to the SQL repo.

  3. Go to Settings > CI/CD

  4. Expand the Runners section.
    Copy the registration token.

  5. Start a terminal session on the GitLab runner. Run the “gitlab-runner register” command.
    You will be prompted for the required information. We recommend creating a tag for the runner so that GitLab CI/CD jobs for Liquibase can be associated with this runner. The executor for the runner should be “shell”.

    Code Block
    sudo gitlab-runner register

Runtime platform     arch=amd64 os=linux pid=11051 revision=5316d4ac version=14.6.0
Running in system-mode.

Enter the GitLab instance URL (for example, https://gitlab.com/):
https://gitlab.com

Enter the registration token:
<the token that was copied above>

Enter a description for the runner:
[ip-172-30-3-12.ec2.internal]: myrunner.mydomain.com

Enter tags for the runner (comma-separated):
liquibase

Registering runner... succeeded                     runner=q8McuSyX

Enter an executor: docker, parallels, virtualbox, docker+machine, kubernetes, custom, shell, ssh, docker-ssh+machine, docker-ssh:
shell

Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Step 4: Create an Access Token

  • The Access Token (PAT) is needed to download artifacts.

  • To create a personal access token:

    • Login to GitLab.

    • On the top-right corner, select your avatar.

    • Select Edit Profile

    • On the left sidebar, select Access Tokens.

    • Enter a name.

    • Leave expiry date blank (never expires).

    • Select access token scope: api

    • Select create personal access token.

    • Save token. It cannot be retrieved later.

  • If you would prefer to use a Project Access Token, the instructions are here: https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html .

...

.

Step 4: Install Liquibase Enterprise on the Agent

Follow these instructions to install Liquibase Enterprise:

Make sure that git is installed on the agent as it will be needed by the Liquibase Enterprise Deploy Packager. SCM Requirements and Features

Step 5: Setup an Azure Key Vault with the database credentials

Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/release/key-vault-in-own-project?view=azure-devops&tabs=portal

The Azure Key Vault is a secure method for storing database credentials. It is recommended that you add Key Vault secrets for all databases in your pipeline including the DMC database.

In this example, Secrets added in the Key Vault will be referenced by the pipeline using Variable Groups.

...

Step 6: Setup an Azure Library Variable Group

Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/library/?view=azure-devops and https://learn.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml

Use an Azure Library to group credentials from your Key Vault into a Variable Group. This grouping makes it easy to associate credentials to Azure Pipelines and Releases.

...

Step 7: Configure the azure_pipelines.yml file

Deploy Packager jobs will be triggered automatically by commits to the packaging branch of the SQL repo. Forecast and Deploy jobs will be triggered manually from the GitLab web interface. The Logging job will always run to generate a .zip of files to be used for debugging errors.

...