Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Liquibase Enterprise can be integrated with Azure DevOps to deploy your database code. Visit the Azure DevOps website to learn more about https://learn.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops.

This document explains how to setup an Azure pipeline with an agent to execute Liquibase Enterprise operations. An Azure DevOps agent can be shared among multiple projects, so one agent can serve several teams. You can learn more about Azure agents here: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser.

Instructions

Prerequisites

Before following the steps in this document, setup your databases and create the Liquibase project.

Liquibase Enterprise requires two repositories which are referred to as Repos in Azure DevOps. Azure Pipelines can be built from code stored in Azure Repos, Bitbucket, GitHub, and other Git or Subversion sources. For this example we will be using Azure Repos.

In Azure Repos create the following two projects:

  • The Liquibase Project repository.

  • The SQL code repository.

This example also uses 2 optional repos:

  • A centralized Rules repository.

  • A centralized CI/CD repository.

Push the Liquibase project configuration files to the project repository in Azure Repos.

git init --initial-branch=main
git remote add origin git@ssh.dev.azure.com:v3/asmith0101/Liquibase/ABC_DDB
git push -u origin --all
git add .
git commit -m "Initial commit"
git push -u origin main

Step 1: Setup an Azure Agent Pool

Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops

An Azure Agent Pool can be created under Project Settings. Agent pools are shared across an organization.

Step 2: Create an Azure Agent

Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops

After setting up an Agent Pool you can create a New agent. Instructions will be provided for the type of agent (Windows or Linux) that you wish to use. Note there are also Detailed Instructions for each of the types. Make sure to use a supported version of Windows or Linux. Hardware, Software, and Database Requirements

Step 3: Setup authentication from the Agent to the Repositories

This example uses SSH keys for git authentication. You can find information on using SSH keys with Azure at https://learn.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops.

If you wish to use HTTPS for git authentication you will need to setup Git Credential Manager. Instructions can be found at https://learn.microsoft.com/en-us/azure/devops/repos/git/set-up-credential-managers?view=azure-devops.

Step 4: Install Liquibase Enterprise on the Agent

Follow these instructions to install Liquibase Enterprise:

Make sure that git is installed on the agent as it will be needed by the Liquibase Enterprise Deploy Packager. SCM Requirements and Features

Step 5: Setup an Azure Key Vault with the database credentials

Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/release/key-vault-in-own-project?view=azure-devops&tabs=portal

The Azure Key Vault is a secure method for storing database credentials. It is recommended that you add Key Vault secrets for all databases in your pipeline including the DMC database.

In this example, Secrets added in the Key Vault will be referenced by the pipeline using Variable Groups.

Step 6: Setup an Azure Library Variable Group

Instructions: https://learn.microsoft.com/en-us/azure/devops/pipelines/library/?view=azure-devops and https://learn.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml

Use an Azure Library to group credentials from your Key Vault into a Variable Group. This grouping makes it easy to associate credentials to Azure Pipelines and Releases.

Step 7: Create a Pipeline to run Packager

Create an Azure Pipeline to run the Deploy Packager (Build) job. Azure Pipelines are created from an azure_pipelines.yml file.

azure_pipelines.yml file

trigger: none

name: $(Application.Name)-$(Build.BuildId)

variables:
- template: ./variables.yml 
- group: Liquibase_Variables

pool:
   name: $(Agent.PoolName)

workspace:
      clean: all

resources:         
  repositories:

  - repository: DDB_REPOSITORY
    name: ABC_DDB
    type: git
    connection: AzureRepos
    source: ABC_DDB
    ref: 'refs/heads/main'

  - repository: SQL_REPOSITORY
    name: ABC_SQL
    type: git
    connection: AzureRepos
    source: ABC_SQL
    ref: 'refs/heads/current'

  - repository: CICD_REPOSITORY
    name: Liquibase/DB_CICD_PIPELINES
    type: git
    connection: AzureRepos
    source: Liquibase/DB_CICD_PIPELINES
    ref: 'refs/heads/main'
 
  - repository: CENTRALIZED_RULES_REPOSITORY
    name: Liquibase/CENTRALIZED_RULES
    type: git
    connection: AzureRepos
    source: Liquibase/CENTRALIZED_RULES
    ref: 'refs/heads/main'

steps:
- checkout: DDB_REPOSITORY
  persistCredentials: true
  clean: true
- checkout: SQL_REPOSITORY
  clean: true
  fetchDepth: 100
  persistCredentials: true
- checkout: CENTRALIZED_RULES_REPOSITORY
  clean: true
  persistCredentials: true
  
- script: |
    whoami
    hammer show version
    echo $(System.DefaultWorkingDirectory)
    printenv
  displayName: 'Run Prechecks'
 
# Ensure the PATH includes the necessary executables, eg. hammer and sqlplus/sqlcmd/clpplus/psql
# Also copy the Rules from the Centralized rules folder to the DDB folder
- powershell: |
    $Env:Path += ";C:\Users\Administrator\DaticalDB\repl"
    $Env:Path += ";C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\"
    cd $(Git.Repository)
    xcopy /s/e/y ..\CENTRALIZED_RULES\Rules\. .\Rules\*
  condition: eq( variables['Agent.OS'], 'Windows_NT' )
  displayName: 'Update PATH (Windows) and set Rules'
  
- script: |
    export PATH="$PATH:/opt/datical/DaticalDB/repl"
    export PATH="$PATH:/opt/mssql-tools/bin"
    cd $(Git.Repository)
    cp -R ../CENTRALIZED_RULES/Rules .
  condition: eq( variables['Agent.OS'], 'Linux' )
  displayName: 'Update PATH (Linux) and set Rules'
  
- script: |
    cd $(Git.Repository) 
    hammer groovy deployPackager.groovy pipeline=current commitPrefix="[skip ci]" scm=true labels=$(Build.BuildId),current
    if [ $? -ne 0 ]; then exit 1; fi
  displayName: 'Run Liquibase packager'
  env:
    DDB_USER: $(Liquibase-abc-ref1-User)
    DDB_PASS: $(Liquibase-abc-ref1-Pass)
    DDB_DMCDB_USER: $(Liquibase-abc-dmc-User)
    DDB_DMCDB_PASS: $(Liquibase-abc-dmc-Pass)

# Compress files into .zip
- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(System.DefaultWorkingDirectory)/$(Git.Repository)'
    includeRootFolder: true
    archiveType: 'zip'
    archiveFile: '$(System.DefaultWorkingDirectory)/artifacts/$(Application.Name)-$(Build.BuildId).zip'

- upload: $(System.DefaultWorkingDirectory)/artifacts/$(Application.Name)-$(Build.BuildId).zip
  artifact: drop

- task: ArtifactoryGenericUpload@2
  inputs:
    artifactoryService: 'ArtifactoryCS'
    specSource: 'taskConfiguration'
    fileSpec: |
      {
        "files": [
          {
            "pattern": "artifacts/$(Application.Name)-$(Build.BuildId).zip",
            "target": "$(Project.Name)/$(Application.Name)/"
          }
        ]
      }
    replaceSpecVars: true
    specVars: '$(Application.Name),$(Build.BuildId)'
    collectBuildInfo: true
    buildName: '$(Build.DefinitionName)'
    buildNumber: '$(Build.BuildNumber)'
    module: '$(Project.Name)'
    failNoOp: true
  displayName: 'Upload zip to artifactory'

- task: ArtifactoryPublishBuildInfo@1
  inputs:
    artifactoryService: 'ArtifactoryCS'
    buildName: '$(Build.DefinitionName)'
    buildNumber: '$(Build.BuildNumber)'
  displayName: 'Publish Build Info to artifactory'

- task: ArtifactoryBuildPromotion@1
  inputs:
    artifactoryService: 'ArtifactoryCS'
    buildName: '$(Build.DefinitionName)'
    buildNumber: '$(Build.BuildNumber)'
    targetRepo: '$(Project.Name)'
    status: 'Released'
    sourceRepo: '$(Project.Name)'
    includeDependencies: false
    copy: false
    dryRun: false
  displayName: 'Promote Build to artifactory'

- task: AzureKeyVault@2
  inputs:
    azureSubscription: 'CustomerSuccessPayAsGo'
    KeyVaultName: 'cs-key-vault1'
    SecretsFilter: '*'
    RunAsPreJob: true

variables.yml file

variables:
   Project.Name: ABC
   Application.Name: ABC-App
   Git.Repository: ABC_DDB
   Git.Repository2: ABC_SQL
   Agent.PoolName: 'CS'
   JFrog.EndPoint: 'JFrog-DevOps'
   JFrog.Repository: 'abc'
   App.Version: $(Build.BuildId)
   Source.Branch: 'current''

Step 8: Create a Release to run Forecast and Deploy Jobs

Use an Azure Release to run the Deploy Packager (Build) job. Azure Pipelines are created from an azure_pipelines.yml file.

Step 9: Test your pipeline.

Commit a change to the packaging branch in your SQL repo. GitLab should run the pipeline Packaging job using your runner.

  • No labels