Introduction

To support development of database application across concurrent releases or to support Continuous Delivery lifecycles, application teams often manage the source code of their applications across several branches of the same source code control repository. This allows individual developers and teams to better control when their changes and additions are released and when they accept changes from other developers or teams that are working on a different feature or release in the same application. To incorporate changes from another source, an application team will merge the source code files from the branch in which they were created to the branch in which the new changes are needed.

To support these branching practices the Deployment Packager can detect when SQL scripts are merged into the SQL code repository, locate existing Change Sets associated with the merged scripts and modify the Change Sets' labels and contexts settings so they can be deployed in a new pipeline or environment.

Associating ChangeSets with SQL Script Merges

In order to identify the change sets that were previously generated by a merged script, the Deployment Packager will now assign a 'scriptChecksum' attribute to each Change Set it introduces when the Packager is run in SCM Mode. The value of the attribute is a checksum that is based on the contents of the script that generated them.  This checksum safeguards against unintended script modification or duplicate name use by ensuring that the script that was merged matches the script that was used to generate the Change Sets.

Merge Approaches

The Deployment Packager can be configured to manage source code merges in two ways: Concurrent Pipeline Merging & Feature Merging.

Concurrent Pipeline Merging

When an application development group is split into multiple teams working on separate planned releases, they often perform source code merges across branches that are tied to distinct pipelines established for each release. A pipeline is a chain of environments in which software is developed, tested, staged and released. After a team's release occurs, they will merge the branch for their pipeline into the branches associated with other teams' planned releases. This allows the other teams to better plan and execute their own development as changes are made to production.

The diagram below illustrates the flow of database changes for a group that is split into three teams, each working in its own pipeline on a separate release. Each pipeline terminates in the production environment. When development is complete and the new features and fixes from a pipeline are released to the production database, the branch in which those changes were developed and tested are merged into the branches associated with future releases.

When the Deployment Packager detects that a SQL script has been merged into the source code control branch associated with a pipeline, it will search for Change Sets with matching script names and scriptChecksum values. When matches are found, the label for the new pipeline is added to those Change Sets which makes them available for Deployment on the new pipeline.

To configure your project to take advantage of Concurrent Pipeline Merge mode, configure pipeline-specific branch properties in the deployPackager.properties file as described in /wiki/spaces/DDOC59/pages/795739037.  Once configured, the detection and handling of merged scripts should occur automatically when the Deploy Packager is run in SCM mode.

DbDef Label Requirements for Pipeline Merges

Each DbDef must have the pipeline name as the first label in order for pipeline merges to succeed. 

Feature Based Merging

Feature merges must merge existing changesets only. Do not attempt to package new changes during a merge. 

Feature Based Merge mode is designed to support development activities where source code control branches are associated with specific environments within a pipeline. In this mode the merge activity represents the opening of a deployment gate between environments in the lifecycle. Consider the following automated development process:

• A development team using a Feature Based Merge strategy has three environments: DEV, QA, & PROD.
• Each environment maps to a GIT branch:
  • DEV -> IN_WORK
  • UAT -> READY_FOR_TEST
  • RELEASE -> RELEASE_APPROVED
• Each environment/branch combo has a Jenkins job associated with it that builds and deploys the application from the branch tied to that environment.
• When a developer merges the files that provide a feature into the 'READY_FOR_TEST' branch of the source code repository, that developer is indicating that the feature is dev complete and should be deployed to the UAT environment.
• Jenkins detects the merge into the READY_FOR_TEST branch and starts the job that builds from that branch and deploys to the QA environment.
• A similar merge into the RELEASE_APPROVED source code control branch triggers the build and deploy to the RELEASE environment for that feature.

Unlike Concurrent Pipeline Merge mode where a group of features are merged after a release, the merges in Feature Based Merge mode precede the release of functionality to the next environment in the pipeline.

When the Deployment Packager detects that a SQL script has been merged into the source code control branch associated with a specific environment, it will search for Change Sets with a label that matches the script names and compare scriptChecksum values in order to validate that the scripts have not been modified since they were originally Packaged. When matches are found, the context for the new database definition is added to those Change Sets. The new context makes them available for Deployment in the new environments.

Follow the steps below to configure your Datical DB project to take advantage of Feature Based Merge mode.

1. Configure environment specific branch properties in the deployPackager.properties file. For this merge mode, your property names should take the format of <dbdefName>.branch. See Using the required deployPackager.properties file.
2. Add a unique identifying default context to each database definition in your project. This default context is required to take advantage of Feature Based Merge mode.
3. Configure the 'enableBackupRestore' property in the deployPackager.properties file for your Datical DB project. The value of this property is a comma separated list of environments that are safe to backup and restore. NOTE: Running the Packager's backup and restore routines is not recommended for environments beyond development and test.

Run the Deployment Packager and target the dbdef to which you wish to merge and deploy changes. Detection and handling of merged scripts should occur automatically when the Deploy Packager is run in SCM mode.