What are multiple Datical Pipelines in a project?
- Multiple pipelines allow you to deploy changes to different database environments
- You will need a REF database for each pipeline
Datical DB allows you to create multiple pipelines in a single Datical project. This allows you to connect to different database steps. Your pipelines will allow you to create multiple deployment paths to deliver your changes to PROD database.
Multiple pipelines in a single Datical project
| Pipeline Name | |||||
|---|---|---|---|---|---|
| Pipeline1 | REF1 | DEV1 | QA1 | UAT | PROD |
| Pipeline2 | REF2 | DEV2 | QA2 | UAT | PROD |
Notice that in pipeline1, you can deploy your database changes to REF1, DEV1 and QA1 database environments. In pipeline2, you can deploy your database changes to REF2, DEV2 and QA2. However, from both pipelines you can deploy code to UAT and PROD databases.
It is important to note that you will need a REF database for each pipeline. Click here to read more about REF databases.
What are common use cases for using multiple Datical pipelines?
- Developers working on concurrent releases
- Development process requires changes to be merged from one pipeline to another in order for changes to be deployed to production
- Hotfix pipeline
There are several reasons for using multiple Datical pipelines. Let's visit a few common use cases:
1. Developers working on concurrent releases
Your team could be developing code for multiple releases.
A common example is that you have monthly scheduled releases and your project team have carved out features for each month. Developers then use dedicated SCM branches for each release. For example, you have a branch called "June2018" for all changes going into June 2018 release. And another branch called "July2018" for changes going into July 2018 release.
Once changes are deployed to production, you merge your release branches to your master branch.
Multiple pipelines in a single Datical project
| Pipeline Name | Branch Name | |||||
|---|---|---|---|---|---|---|
Pipeline1 | June2018 | REF1 | DEV1 | QA1 | UAT | PROD |
| Pipeline2 | July2018 | REF2 | DEV2 | QA2 | UAT | PROD |
2. Development process requires changes to be merged from one branch to another in order for changes to be deployed to production
Your development team has adopted a process for working on dedicated branches. This allows you to develop features independently.
For example, your branches are called "line1" and "line2". The idea is that any code checked into "line1" branch is eligible for deployment to DEV1 and QA1 environments. Similarly, any code checked into "line2" branch is eligible to be deployed to DEV2 and QA2 environments. Hence, you can develop and test features independently on different environments.
Then, your project team or business decides which features to include in the release. For that, they choose which branch to merge to "release" branch. Any code merged into "release" would be eligible to deploy to PROD environment.
Multiple pipelines in a single Datical project
| Pipeline Name | Branch Name | |||
|---|---|---|---|---|
| Pipeline1 | line1 | REF1 | DEV1 | QA1 |
| Pipeline2 | line2 | REF2 | DEV2 | QA2 |
| Pipeline3 | release | REF3 | UAT | PROD |
As a result of this approach, Datical is automatically able to detect changes packaged in "line1" branch that merged with "release" branch.
3. Hotfix
Often, there is a need to patch changes made to production. In addition to requiring a shorter pipeline to PROD, you also need a separate SCM branch to maintain all hotfix changes.
Multiple pipelines in a single Datical project
| Pipeline Name | Branch Name | |||||
|---|---|---|---|---|---|---|
| Pipeline1 | release | REF1 | DEV1 | QA1 | UAT | PROD |
| Hotfix | hotfix | REF2 | UAT | PROD |
How to setup multiple pipelines?
In order to setup multiple pipelines in Datical you would follow these steps:
- Create a new pipeline in Datical project
- Populate DB steps in the pipeline
- Configure "deployPackager.properties" file to map pipelines to SCM branches
Let's take a look at each step in details:
1. Create a new pipeline in Datical project
In Datical DB GUI, click on the "Manage" button
Click on the Add button
Provide the new pipeline name and a description
Click OK. You should now have two pipelines in your list:
Click OK. Now switch to the newly created pipeline in Datical GUI:
2. Populate DB steps in the pipeline
Click "Add Steps" to setup connection to a new database, e.g., REF2 database.
Click "Next" and proceed with providing connection information for your first DB step in the pipeline.
For connecting to an existing database step that is already used in another pipeline, e.g., UAT or PROD, use the "Select an existing deployment step" option at the top:
3. Configure "deployPackager.properties" file to map pipelines to SCM branches
Configure your "deployPacakager.properties" file to map your pipeline to branches. See the sample file below:
- Pipeline "release1" is mapped to branch "June2018". See line 12
- Pipeline "release2" is mapped to branch "July2018". See line 15
databaseBackupRestoreLocation=DATICAL_DATA_PUMP_DIR databaseBackupMode=on_demand ddbScmSystem=git ddbScmBranch=master ddbScmPath=. sqlScmSystem=git sqlScmPath=../EFS_SQL sqlScmSQLBaseDir=sql_code release1.sqlScmBranch=June2018 release1.sqlScmLastImportID=d7e7bac release2.sqlScmBranch=July2018 release2.sqlScmLastImportID=f3896e0 validateRollback = none disableTraceFile = false
How does Datical handle branch merges when changes are already packaged in the source branch?
See link in Datical Documentation space → /wiki/spaces/DDD/pages/22511822