Liquibase Enterprise can be integrated with Azure DevOps to deploy your database code. Visit the Azure DevOps website to learn more about https://learn.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops.
This document explains how to setup an Azure pipeline with an agent to execute Liquibase Enterprise operations. An Azure DevOps agent can be shared among multiple projects, so one agent can serve several teams. You can learn more about Azure agents here: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser.
Instructions
Prerequisites
Before following the steps in this document, setup your databases and create the Liquibase project.
Liquibase Enterprise requires two repositories which are referred to as Repos in Azure DevOps. Azure Pipelines can be built from code stored in Azure Repos, Bitbucket, GitHub, and other Git or Subversion sources. For this example we will be using Azure Repos.
In Azure Repos create the following two projects:
The Liquibase Project repository.
The SQL code repository.
This example also uses 2 optional repos:
A centralized Rules repository.
A centralized CI/CD repository.
Push the Liquibase project configuration files to the project repository in Azure Repos.
git init --initial-branch=main git remote add origin git@ssh.dev.azure.com:v3/asmith0101/Liquibase/ABC_DDB git push -u origin --all git add . git commit -m "Initial commit" git push -u origin main
Step 1: Setup an Azure Agent Pool
Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops
An Azure Agent Pool can be created under Project Settings. Agent pools are shared across an organization.
Step 2: Create an Azure Agent
Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops
After setting up an Agent Pool you can create a New agent. Instructions will be provided for the type of agent (Windows or Linux) that you wish to use. Note there are also Detailed Instructions for each of the types. Make sure to use a supported version of Windows or Linux. Hardware, Software, and Database Requirements
Step 3: Setup authentication from the Agent to the Repositories
This example uses SSH keys for git authentication. You can find information on using SSH keys with Azure at https://learn.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops.
If you wish to use HTTPS for git authentication you will need to setup Git Credential Manager. Instructions can be found at https://learn.microsoft.com/en-us/azure/devops/repos/git/set-up-credential-managers?view=azure-devops.
Step 4: Install Liquibase Enterprise on the Agent
Follow these instructions to install Liquibase Enterprise:
Windows: Installing Liquibase Enterprise Clients on Windows Systems
Linux: Theme: Using the CLI and the Composite Repository to Install Liquibase Enterprise on Linux Systems
Make sure that git is installed on the agent as it will be needed by the Liquibase Enterprise Deploy Packager. SCM Requirements and Features
Step 5: Setup an Azure Key Vault with the database credentials
Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/release/key-vault-in-own-project?view=azure-devops&tabs=portal
The Azure Key Vault is a secure method for storing database credentials. It is recommended that you add Key Vault secrets for all databases in your pipeline including the DMC database.
In this example, Secrets added in the Key Vault will be referenced by the pipeline using Variable Groups.
Step 6: Setup an Azure Library Variable Group
Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/library/?view=azure-devops and https://learn.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml
Use an Azure Library to group credentials from your Key Vault into a Variable Group. This grouping makes it easy to associate credentials to Azure Pipelines and Releases.
Step 7: Create a Pipeline to run Packager
Create an Azure Pipeline to run the Deploy Packager (Build) job. Azure Pipelines are created from an azure_pipelines.yml file.
azure_pipelines.yml file
trigger: none name: $(Application.Name)-$(Build.BuildId) variables: - template: ./variables.yml - group: Liquibase_Variables pool: name: $(Agent.PoolName) workspace: clean: all resources: repositories: - repository: DDB_REPOSITORY name: ABC_DDB type: git connection: AzureRepos source: ABC_DDB ref: 'refs/heads/main' - repository: SQL_REPOSITORY name: ABC_SQL type: git connection: AzureRepos source: ABC_SQL ref: 'refs/heads/current' - repository: CICD_REPOSITORY name: Liquibase/DB_CICD_PIPELINES type: git connection: AzureRepos source: Liquibase/DB_CICD_PIPELINES ref: 'refs/heads/main' - repository: CENTRALIZED_RULES_REPOSITORY name: Liquibase/CENTRALIZED_RULES type: git connection: AzureRepos source: Liquibase/CENTRALIZED_RULES ref: 'refs/heads/main' steps: - checkout: DDB_REPOSITORY persistCredentials: true clean: true - checkout: SQL_REPOSITORY clean: true fetchDepth: 100 persistCredentials: true - checkout: CENTRALIZED_RULES_REPOSITORY clean: true persistCredentials: true - script: | whoami hammer show version echo $(System.DefaultWorkingDirectory) printenv displayName: 'Run Prechecks' # Ensure the PATH includes the necessary executables, eg. hammer and sqlplus/sqlcmd/clpplus/psql # Also copy the Rules from the Centralized rules folder to the DDB folder - powershell: | $Env:Path += ";C:\Users\Administrator\DaticalDB\repl" $Env:Path += ";C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\" cd $(Git.Repository) xcopy /s/e/y ..\CENTRALIZED_RULES\Rules\. .\Rules\* condition: eq( variables['Agent.OS'], 'Windows_NT' ) displayName: 'Update PATH (Windows) and set Rules' - script: | export PATH="$PATH:/opt/datical/DaticalDB/repl" export PATH="$PATH:/opt/mssql-tools/bin" cd $(Git.Repository) cp -R ../CENTRALIZED_RULES/Rules . condition: eq( variables['Agent.OS'], 'Linux' ) displayName: 'Update PATH (Linux) and set Rules' - script: | cd $(Git.Repository) hammer groovy deployPackager.groovy pipeline=current commitPrefix="[skip ci]" scm=true labels=$(Build.BuildId),current if [ $? -ne 0 ]; then exit 1; fi displayName: 'Run Liquibase packager' env: DDB_USER: $(Liquibase-abc-ref1-User) DDB_PASS: $(Liquibase-abc-ref1-Pass) DDB_DMCDB_USER: $(Liquibase-abc-dmc-User) DDB_DMCDB_PASS: $(Liquibase-abc-dmc-Pass) # Compress files into .zip - task: ArchiveFiles@2 inputs: rootFolderOrFile: '$(System.DefaultWorkingDirectory)/$(Git.Repository)' includeRootFolder: true archiveType: 'zip' archiveFile: '$(System.DefaultWorkingDirectory)/artifacts/$(Application.Name)-$(Build.BuildId).zip' - upload: $(System.DefaultWorkingDirectory)/artifacts/$(Application.Name)-$(Build.BuildId).zip artifact: drop - task: ArtifactoryGenericUpload@2 inputs: artifactoryService: 'ArtifactoryCS' specSource: 'taskConfiguration' fileSpec: | { "files": [ { "pattern": "artifacts/$(Application.Name)-$(Build.BuildId).zip", "target": "$(Project.Name)/$(Application.Name)/" } ] } replaceSpecVars: true specVars: '$(Application.Name),$(Build.BuildId)' collectBuildInfo: true buildName: '$(Build.DefinitionName)' buildNumber: '$(Build.BuildNumber)' module: '$(Project.Name)' failNoOp: true displayName: 'Upload zip to artifactory' - task: ArtifactoryPublishBuildInfo@1 inputs: artifactoryService: 'ArtifactoryCS' buildName: '$(Build.DefinitionName)' buildNumber: '$(Build.BuildNumber)' displayName: 'Publish Build Info to artifactory' - task: ArtifactoryBuildPromotion@1 inputs: artifactoryService: 'ArtifactoryCS' buildName: '$(Build.DefinitionName)' buildNumber: '$(Build.BuildNumber)' targetRepo: '$(Project.Name)' status: 'Released' sourceRepo: '$(Project.Name)' includeDependencies: false copy: false dryRun: false displayName: 'Promote Build to artifactory' - task: AzureKeyVault@2 inputs: azureSubscription: 'CustomerSuccessPayAsGo' KeyVaultName: 'cs-key-vault1' SecretsFilter: '*' RunAsPreJob: true
variables.yml file
variables: Project.Name: ABC Application.Name: ABC-App Git.Repository: ABC_DDB Git.Repository2: ABC_SQL Agent.PoolName: 'CS' JFrog.EndPoint: 'JFrog-DevOps' JFrog.Repository: 'abc' App.Version: $(Build.BuildId) Source.Branch: 'current''
Step 8: Create a Release to run Forecast and Deploy Jobs
Use an Azure Release to run the Deploy Packager (Build) job. Azure Pipelines are created from an azure_pipelines.yml file.
Step 9: Test your pipeline.
Commit a change to the packaging branch in your SQL repo. GitLab should run the pipeline Packaging job using your runner.