Use the Required deployPackager.properties File

Liquibase Enterprise was formerly known as Datical DB.

Use the Required deployPackager.properties File

This guide applies to Liquibase Enterprise version 8.2 and earlier. Liquibase Enterprise 8.3 and later automatically adds a deployPackager.properties file when a new project is created. This works in the CLI, GUI, or a project_creator groovy script.

A project must have a deployPackager.properties file in order for SQL scripts to be packaged for it. 

deployPackager.properties Configuration

Before using the Deployment Packager, a deployPackager.properties will need to be created in the top level of your Datical project directory. This property file contains a variety of configuration information used by the Deployment Packager to interact with SCM solutions and to perform database backups and restorations.

Note: there is a sample deployPackager.properties file provided in DaticalDB/repl/scripts.

Properties Prefixes for SQL Code and Datical Project Files

The property prefix indicates the type of files it affects and the location of those files.

  • sqlScm prefix - SQL code files (in predetermined subdirectories)

  • ddbScm prefix - Datical project files (in a predetermined hierarchy)

When properties for the project are not specified (ddbScm prefix), the values typically default to the properties for the SQL code (sqlScm prefix) 

Pipeline and DbDef Branch Settings

In order for the Deployment Packager to process the appropriate scripts for a specific dbdef or pipeline, the SCM branch name and the ID for the last import processed need to be tracked separately in the deployPackager.properties files. These properties should take the following form:

  • Branch name:

    • {PipelineName}.sqlScmBranch

    • {DbDefName}.sqlScmBranch for the branch name

  • Last 

    • {PipelineName}.sqlScmLastImportID

    • {DbDefName}.sqlScmLastImportID.

Refer to the properties in the bottom section of the file below for an example of the Deployment Pipeline specific properties.

Property List for deployPackager.properties

The following properties can be set in the deployPackager.properties file: (note that these properties are NOT in alphabetical order)

Name

Applies to

Description

Name

Applies to

Description

databaseBackupRestoreLocation

Backup

Location to use for packager-driven backups of the database. During processing, Deployment Packager makes backups of the database that are normally stored on the database server itself.

Depending on your DBMS, the location may need to be specified as directory or a specific file.

  • Oracle - the location will be the database directory object name.

  • MSSQL, and DB2 - the directory is on the server host, so the path expression is OS-specific. ('/' on Linux, '\' on Windows.)

  • Postgres - valid path on the server running hammer packaging

  • DB2 - include a trailing slash

  • Windows hosts - Double the backslashes in the path.  The backslash ( \ )  is an escape character in Java properties files, so you must write '\\' to stand for '\'. 

    • Example: C:\\Program Files\\Microsoft SQL Server\\MSSQL14.MyInstance\\DaticalBackups

db2BackupDir.<referenceDBName>=0

Backup

DB2 Only: Set automatically after the first execution of Deployment Packager. The warning from the first execution of packager on a given database can be ignored.

ddlExcludeList

Compares

SQL code types to exclude from a compare, typically to improve performance. When packaging DDL scripts, Deployment Packager compares the before and after state of the reference database. Use this property to exclude object types from the comparison. Values can be: 

  • none (the default)

  • all (stored logic)

  • Comma-separated list of these values: procedure, package, packagebody, function, trigger, view, and non_stored_logic. Excluding non-ddl objects can improve performance, especially with large databases.

    • The preferred style is to separate by comma only. However, spaces and tabs are trimmed if they are encountered.

    • The values list may be enclosed in double quotes or not.

Example ddlExcludeList property setting
ddlExcludeList="package,packagebody,procedure,function,trigger,view" ddlExcludeList=package,packagebody,procedure,function,trigger,view

ddbScmBranch

Project files

The SCM branch to use. If not specified, Deployment Packager uses the value from sqlScmBranch.

ddbScmDoCheckout

Project files

If this is not present or is set to true, Deployment Packager ensures that the local repository is up to date before packaging. To skip this operation, set to false.

  • Git - equivalent to git pull

ddbScmLastImportID

Project files

The SCM system ID for the most recently processed SQL code check in. It is automatically updated when the Deployment Packager completes successfully.

Note: Do not leave blank. Do not use the most recent check-in. For first time use, use an ID from an earlier check-in.

  • Git - first 8 digits of the hash

ddbScmPath

Project files

Path to the local copy of the SCM repository. If this is not present, Deployment Packager will use the value from sqlScmPath.

ddbScmSystem

Project files

SCM type – one of git

May be different than the sqlScmSystem. If not specified, Deployment Packager uses the value from sqlScmSystem.

enableBackupRestore

Backup

When the Packager is run in 'dbdef=' mode, the backup and restore routines can be skipped for certain sensitive environments. The value of this property is a comma separated list of the environments in which the Packager should perform backup and restore tasks. NOTE: Backup and restore tasks are NOT recommended for environments beyond development & test.

ephemeralCleanUpBeforeStart

Ephemeral database

(MSSQL, Postgres, and Oracle)

[Optional] Allows Liquibase to drop any existing Ephemeral database or PDB if found during an Ephemeral Packaging Operation. Existing databases can occur from failures or cancelations from previous Ephemeral Packaging jobs.

Property Validation: true/false

Available in Liquibase 8.9 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

ephemeralCopyName

Ephemeral database (MSSQL and Postgres)

[Optional] Defines a name for a DB copy (has impact on SQL Server Ephemeral copy only).

Default name for the ephemeral copy will be source database name with _eph postfix. For example, if source name is ABC_DEV, the copied database name will be ABC_DEV_eph.

Property Validation: Copy name should be up to 124 characters long, contain no spaces [ ], slashes [/] and semicolons [;] and cannot begin with a number.

Available in Liquibase 8.8 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

ephemeralCopyWithData

Ephemeral database (MSSQL and Postgres)

[Optional] By default Liquibase will copy databases without data. (*) If you would like to copy both the structure and data, use this boolean property.

(*) There is one exception in that the data from the DATABASECHANGELOG table is included in the database copy.

Property Validation: true/false

Available in Liquibase 8.8 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

ephemeralSkipFinalDeploy

Ephemeral database (MSSQL and Postgres)

[Optional] The default value for this setting is false. Any changes being packaged are deployed to the first database in the pipeline during the Deploy Packager operation.

Setting this value to true allows changes to be packaged and changesets to be created and stored in the changelog.xml, but the deployment to the first database in the pipeline does not occur.

Property Validation: true/false

Available in Liquibase 8.8 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleBackupNoCrossSchemaFk



When set to true Cross Schema Foreign Keys will not be included in the backup and older backups containing Foreign Keys will not apply those items during a restore.

The backup/restore method will not be affected by this property if this property is absent in the deployPackager.properties file or if the value is false.

Available in Liquibase 8.5 and later.

oracleBackupNoGrants



When set to true Grants will not be included in the backup and older backups containing Grants will not apply those items during a restore.

The backup/restore method will not be affected by this property if this property is absent in the deployPackager.properties file or if the value is false.

Available in Liquibase 8.5 and later.

oracleBackupNoDropSchema



When set to true, the Schema(s) will not be dropped. Instead all objects (such as tables, constraints, views, materialized views, functions, procedures, sequences, packages, triggers, synonyms, and user types) within the Schema(s) will be dropped instead.

The backup/restore method will not be affected by this property if this property is absent in the deployPackager.properties file or if the value is false.

Available in Liquibase 8.5 and later.

oracleEphemeralCopyName

Ephemeral database (Oracle)

[Optional] Defines a name for an Oracle copy PDB in the event the default name is not desired.

Default name for the ephemeral copy will be source PDB name with _eph postfix. For example, if source name is bucket_01, the copied bucket name will be bucket_01_eph.

This property should be used in conjunction with a custom connection type. Either oracleEphemeralCopyServiceName, oracleEphemeralCopyTnsName, or oracleEphemeralCopySid should be supplied when using this property.

Property Validation: Copy name should be up to 30 characters long, contain no spaces [ ], slashes [/] and semicolons [;] and cannot begin with a number.

Available in Liquibase 8.6 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemeralCopyWithData

Ephemeral database (Oracle)

[Optional] By default Liquibase will copy databases without data. (*) If you would like to copy both the structure and data, use this boolean property.

(*) There is one exception in that the data from the DATABASECHANGELOG table is included in the database copy.

Property Validation: true/false

Available in Liquibase 8.6 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemeralDatabaseLinkName

Ephemeral database (Oracle)

[Optional] Defines a name for a database link which is used with PackageMethod BackupRestoreOracleEphemeralExternalPdb.

Default link name will be liquibase_db_link.

Property Validation: Database link name should up to 128 characters long, contain no spaces [ ], slashes [/] and semicolons [;] and cannot begin with a number.

Available in Liquibase 8.6 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemeralCopyServiceName

Ephemeral database (Oracle)

[Optional] This impacts Oracle Ephemeral packaging only.
Overrides the Oracle copy dbDef SERVICE_NAME if the source dbDef uses SERVICE_NAME as a connection type. If the property is not specified, _eph postfix will be added to source SERVICE_NAME before a first dot

Example: datical.dev.com -> datical_eph.dev.com

If SERVICE_NAME does not contain dots, they will be added at the end

Example: datical_service_name -> datical_service_name_eph

Available in Liquibase 8.9 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemearalCopySid

Ephemeral database (Oracle)

[Optional] This impacts Oracle Ephemeral packaging only.

Overrides Oracle copy dbDef SID if the source dbDef uses SID as the connection type. If the property is not specified, the _eph postfix will be added to the source SID

Example: e.g., datical_siddatical_sid_eph

Available in Liquibase 8.9 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemeralCopyTnsName

Ephemeral database (Oracle)

[Optional] This impacts Oracle Ephemeral packaging only.

Overrides the Oracle copy dbDef TNS_NAME if the source dbDef uses TNS_NAME as connection type. If the property is not specified _eph postfix will be added to source TNS_NAME

datical_tns -> datical_tns_eph

Available in Liquibase 8.9 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemeralFileNameConvertPrefix

Ephemeral database (Oracle)

[Optional] Defines a custom file path prefix when copying the datafiles to an ephemeral database. Liquibase will determine the file path prefix used by the existing datafiles and replace it with the value specified.

Example: oracleEphemeralFileNameConvertPrefix=/opt/oracle/oradata/ORCLCDB/prefixFNC

Available in Liquibase 8.10 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemeralFileNameConvert

Ephemeral database (Oracle)

[Optional] Defines a custom file path mapping to be used when copying datafiles to an ephemeral database. Liquibase will replace the specified file path (first value supplied) with the requested file path (second value supplied.) 

Example: oracleEphemeralFileNameConvert='/opt/oracle/oradata/ORCLCDB/bucket_01','/opt/oracle/oradata/ORCLCDB/custom_FNC/bucket_01_eph'

Note: This property takes precedence over the oracleEphemeralFileNameConvertPrefix. If both properties are specified, oracleEphemeralFileNameConvert will be used.

Available in Liquibase 8.10 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemeralSkipFinalDeploy

Ephemeral database (Oracle)

[Optional] The default value for this setting is false. Any changes being packaged are deployed to the first database in the pipeline during the Deploy Packager operation.

Setting this value to true allows changes to be packaged and changesets to be created and stored in the changelog.xml, but the deployment to the first database in the pipeline does not occur.

Property Validation: true/false

Available in Liquibase 8.6 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

oracleEphemeralSourceName

Ephemeral database (Oracle)

[Optional] Defines a source name for an Oracle source PDB in the event Liquibase has not been granted the necessary permissions to query for this name.

Necessary permissions are included below. If these grants are not present, oracleEphemeralSourceName will be required.

  • Multi-schema projects: GRANT SELECT ANY DICTIONARY TO DATICAL_ROLE;

  • Single schema projects Liquibase requires either:

    • GRANT SELECT ANY DICTIONARY TO DATICAL_SCHEMA_OWNER_ROLE;

    • OR permission to query v$pdbs and dba_data_files

  • See Roles and Permissions for Liquibase Enterprise on Oracle Database for further information on permissions.

Property Validation: Source name should up to 30 characters long, contain no spaces [ ], slashes [/] and semicolons [;] and cannot begin with a number.

Available in Liquibase 8.6 and later. This property can be set or overwritten at the dbDef level by prefixing the property with the dbDef name with the format <dbDef>.<propertyName>=<propertyValue>.

scmBranchHandling

SCM

Specifies how to handle branches in a multiple-branch project, one of two methods:

  • scmswitch – use SCM-specific commands to switch branches in sqlScmPath

  • directory – use a specified directory.

    • When run against a pipeline, a pipeline-specific sqlScmPath is checked:

      <pipeline>.sqlScmPath=<path>
    • When run against a dbdef, a dbdef-specific sqlScmPath is checked:

      <dbdef>.sqlScmPath=<path>
    • If a pipeline-specific or dbdef-specific path is not provided, sqlScmPath is used. If sqlScmPath is also not specified, an error is returned.

See "Pipeline and DbDef Branch Settings" below.

sqlScmBranch

SQL code files

The repository branch to be used for SQL code. Format varies by SCM:

  • Git - branch name only

sqlScmDoCheckout

SQL code files

If this is not present or is set to true, Deployment Packager ensures that the local repository is up to date before packaging. To skip this operation, set to false.  (Note that when setting this property to false that the DDB repo status will still be checked to make sure there aren't any uncommitted changes, but a pull/update/skip will not be performed.)

  • Git - equivalent to git pull

sqlScmLastImportID

SQL code files

The SCM system ID for the most recently processed SQL code check in. It is automatically updated when the Deployment Packager completes successfully.

Note: Do not leave blank. Do not use the most recent check-in. For first time use, use an ID from an earlier check-in.

  • Git - first 8 digits of the hash

sqlScmPath

SQL code files

Path to the directory for SQL code in the local copy of the SCM repository. Can be an absolute path or path relative to the project directory. 

When packager determines the location of SQL code, it builds the location from sqlSCMPath and sqlScmSQLBaseDir

On Windows, backslashes must be doubled. Example:

sqlScmPath=c:\\users\\username\\scmworkdir\\projectname



sqlScmSQLBaseDir

SQL code files

Top level directory for SQL code stored in the local copy of the repository. This is a name only, not a path. If set to '.' uses the value of sqlScmPath. This directory contains:

  • Set of fixed-name sub-directories needed by Deployment Packager (ddl,data_dml,sql, and so on)

  • Specially configured directories (See Flexible Folder Names.)

  • Combination of fixed-name and flexible-name directories 

If not specified, a default directory name sql_code is used. 

When packager determines the location of SQL code, it builds the location from sqlSCMPath and sqlScmSQLBaseDir

sqlScmSystem

SQL code files

SCM type – one of git

commitMsgLabelRegexp

SQL code files

By default, the Deploy Packager will scan the commit messages in your SQL code repo for values in square brackets.  These values will be added as labels to the change sets that are produced by the files in that commit.  For example:

Commit Message: Adding column to hold area code information for [JIRA-1234] Change Set Label: JIRA-1234


If there is another pattern you would like to use for the designation of change set labels you can specify the regular expression for that pattern using this property.  For example:

commitMsgLabelRegexp=JIRA (\\d{6}) Commit Message: Adding column to hold area code information for JIRA 4321 which requires phone number separation Change Set Label: 4321

NOTE: If you provide a different pattern for matching desired label values, the values in square brackets will no longer be included as labels on change sets.

databaseBackupRestoreMethod

Backup

Class name to use for custom backup and restore. The class is implemented in a .groovy script. 

extensionsPath

Backup

Path to directory containing .groovy files that implement a custom backup and restore method. If not specified, the following directory is used: 
<datical-install>/repl/scripts/extensions.

databaseBackupMode

Backup

Specifies how to manage reference database backups that are required for deployments.

  • always - (Default) Back up the database for every change deployment. The schema to be backed up may be limited by using the schemaName property in metatdata.properties. This is the most commonly used mode. 

  • on_demand - Back up the database only when a deployPackager.groovy command line option is used: createDatabaseBackup=trueThis mode always backs up all managed schema and the tracking schema.  This mode is intended as a special case for use with large databases that take a very long time to back up. 

oracleBackupErrorWhitelist

Backup (Oracle Only)

A comma separated list of Oracle error codes to be ignored when evaluating expdp import/restore output for errors.  (This property is only available with versions 7.18 and higher.) During an Oracle backup that uses local expdp / impdp clients OR the DATAPUMP API on the server, error messages are compared to the white-list:

  • If any errors are found that do not exist in the white-list backup fails

  • If the only errors found do exist in the white-list, backup completes successfully

Example:

oracleBackupErrorWhitelist=ORA-39173,ORA-39181

oracleRestoreErrorWhitelist

Restore (Oracle Only)

A comma separated list of Oracle error codes to be ignored when evaluating impdp import/restore output for errors.  (This property is only available with versions 7.3 and higher.) During an Oracle restore that uses local expdp / impdp clients OR the DATAPUMP API on the server, error messages are compared to the white-list:

  • If any errors are found that do not exist in the white-list restore fails

  • If the only errors found do exist in the white-list, restore completes successfully

Example:

oracleRestoreErrorWhitelist=ORA-31684,ORA-39111,ORA-39151

parallel

Backup

Oracle only. Maximum of parallel threads for expdp and impdp to use during backup and restore. 

archive

Processing

Archive SQL scripts in the archive directory after processing. This is done for SQL code in the ddl, sql_direct, data_dml directories. Archiving is done by default. 

Set to false to override the default behavior.

If not set or if set to true, archiving is done as usual. 

The archive property has been deprecated.  Instead it is preferable to set the newer rerunnable property in the metadata.properties of each relevant folder.

validateRollback

SQL code files

Copyright © Liquibase 2012-2022 - Proprietary and Confidential