Introduction to Multiple Liquibase Enterprise Pipelines

What are multiple Datical Pipelines in a project?

• Multiple pipelines allow you to deploy changes to different database environments
• You will need a REF database for each pipeline

Datical DB allows you to create multiple pipelines in a single Datical project. This allows you to connect to different database steps. Your pipelines will allow you to create multiple deployment paths to deliver your changes to PROD database.

Multiple pipelines in a single Datical project

Notice that in pipeline1, you can deploy your database changes to REF1, DEV1 and QA1 database environments. In pipeline2, you can deploy your database changes to REF2, DEV2 and QA2. However, from both pipelines you can deploy code to UAT and PROD databases. 

It is important to note that you will need a REF database for each pipeline. Click here to read more about REF databases.

What are common use cases for using multiple Datical pipelines?

• Developers working on concurrent releases
• Development process requires changes to be merged from one pipeline to another in order for changes to be deployed to production
• Hotfix pipeline

There are several reasons for using multiple Datical pipelines. Let's visit a few common use cases:

1. Developers working on concurrent releases

Your team could be developing code for multiple releases.

A common example is that you have monthly scheduled releases and your project team have carved out features for each month. Developers then use dedicated SCM branches for each release. For example, you have a branch called "June2018" for all changes going into June 2018 release. And another branch called "July2018" for changes going into July 2018 release. 

Once changes are deployed to production, you merge your release branches to your master branch.

Multiple pipelines in a single Datical project

2. Development process requires changes to be merged from one branch to another in order for changes to be deployed to production

Your development team has adopted a process for working on dedicated branches. This allows you to develop features independently.

For example, your branches are called "line1" and "line2". The idea is that any code checked into "line1" branch is eligible for deployment to DEV1 and QA1 environments. Similarly, any code checked into "line2" branch is eligible to be deployed to DEV2 and QA2 environments. Hence, you can develop and test features independently on different environments.

Then, your project team or business decides which features to include in the release. For that, they choose which branch to merge to "release" branch. Any code merged into "release" would be eligible to deploy to PROD environment.

Multiple pipelines in a single Datical project

As a result of this approach, Datical is automatically able to detect changes packaged in "line1" branch that merged with "release" branch. 

3. Hotfix

Often, there is a need to patch changes made to production. In addition to requiring a shorter pipeline to PROD, you also need a separate SCM branch to maintain all hotfix changes.

Multiple pipelines in a single Datical project

How to setup multiple pipelines?

In order to setup multiple pipelines in Datical you would follow these steps:

1. Create a new pipeline in Datical project
2. Populate DB steps in the pipeline
3. Configure "deployPackager.properties" file to map pipelines to SCM branches

Let's take a look at each step in details:

1. Create a new pipeline in Datical project

In Datical DB GUI, click on the "Manage" button

Click on the Add button

Provide the new pipeline name and a description

Click OK. You should now have two pipelines in your list:

Click OK. Now switch to the newly created pipeline in Datical GUI:

2. Populate DB steps in the pipeline

Click "Add Steps" to setup connection to a new database, e.g., REF2 database. 

Click "Next" and proceed with providing connection information for your first DB step in the pipeline.

For connecting to an existing database step that is already used in another pipeline, e.g., UAT or PROD, use the "Select an existing deployment step" option at the top:

3. Configure "deployPackager.properties" file to map pipelines to SCM branches

Configure your "deployPacakager.properties" file to map your pipeline to branches. See the sample file below:

• Pipeline "release1" is mapped to branch "June2018". See line 12
• Pipeline "release2" is mapped to branch "July2018". See line 15

databaseBackupRestoreLocation=DATICAL_DATA_PUMP_DIR
databaseBackupMode=on_demand
 
ddbScmSystem=git
ddbScmBranch=master
ddbScmPath=.

sqlScmSystem=git
sqlScmPath=../EFS_SQL
sqlScmSQLBaseDir=sql_code
 
release1.sqlScmBranch=June2018
release1.sqlScmLastImportID=d7e7bac
 
release2.sqlScmBranch=July2018
release2.sqlScmLastImportID=f3896e0
validateRollback = none
disableTraceFile = false

How does Datical handle branch merges when changes are already packaged in the source branch?

See link in Datical Documentation space → SQL Code Branching & Merging (Advanced)