Versions Compared

Old Version 18

changes.mady.by.user Former user

Saved on

compared with

New Version 19

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

...

The Deployment Packager can be run from the CLI or the GUI. You provide a configuration file for the project that specifies the options to use during packaging. See Using the required deployPackager.properties file. 

Packaging Sources

You can package SQL scripts from the following sources.  

  • Source Code Management (SCM) Repository: Place SQL scripts in a defined location in a source code repository. Configure a properties file with information about source code management repositories where SQL scripts and the Datical DB Project reside. During processing, the Deployment Packager reads the properties file and gets the SQL scripts from the SCM. 
  • Manifest File: Define a manifest file to identify the SQL scripts to be processed and provide metadata about the scripts.  The manifest file is passed to the Deployment Packager.

Workflow

Deployment Packager uses the following workflow for changes checked into an SCM. When packaging with a manifest file, the manifest file specifies the files to package (steps 1 and 2 below). 

  1. Prereq:
    1. Get Project - Check out Datical project from the SCM (DDB SCM)
    2. Get Scripts - Check out SQL scripts from the SCM (SQL SCM)
  2. Determine list of scripts to package from SQL SCM - This is determined by what changes have been introduced to SQL SCM since last successful packaging job (If this is the first packaging job then all scripts will be packaged)
  3. Check Scripts for Rules Compliance - Run validation routines, which are a combination of Datical-provided rules and user-defined rules.  Rules ensure that changes are valid and compliant with organizational standards and best practices.
  4. Back Up REF Database - If the changes fail, the database is restored to its original state from this backup. 
  5. Validate - Check the files against built-in Datical rules and rules provided for the project.  End process if validation fails. (Note: rules validation can be set to fail according to different criteria). 
  6. Process Files
    • For scripts that use the convert method (If using Fixed Folders these are scripts packaged in /ddl folder only.  If using Flexible Folders these are scripts that use packageMethod CONVERT.)
      • Snapshot1 the existing schema of the REF database.
      • Convert transient files (like DDL) to an XML object model (changelog) (basic: take DDL, apply to DB using native tools, snapshot again, compare snapshots. The compare produces the object model.)
      • Snapshot2 the schema to record the changes produced by processing the files. 
      • Save a copy of the changelog as it was updated by the changes. 
    • For scripts that use all other methods (If using Fixed folders these are scripts packaged in all other folders such as /ddl_direct, /data_dml, /function, /procedure, /sql_direct, /ssis, etc.  If using Flexible Folders these are scripts that use packageMethod STOREDLOGIC, SSIS, SQLFILE, DIRECT, DDL_DIRECT, or DATA_DML)
      • Prepare non-transient files to be run. (changeset includes a deployPackagerNative call–some analysis of SQL and sanitization)
    • IF there were any CONVERT/DDL scripts, restore the REF database from backup in preparation for Forecast.
  7. Forecast Changes - Simulate applying the changes. [ Apply changes represented in the object model (changelog) to an in-memory copy of Snapshot1 ].   Produce a forecast report. 
    • Proceed if forecasting succeeds. 
    • End process and restore the REF database from backup if an error is encountered during the forecast.
  8. Deploy Changes to the REF Database.
  9. (optional) If there are rollbacks, run the rollbacks for the scripts that were packaged, if validateRollback is enabled.
    • Redeploy Changes to the REF Database.
  10. Move non-rerunnable SQL Files to the archive.
  11. Commit Changes to SCM repos.
  12. Update the packagerReport.html to show details of the Deployment Packager results.


The following diagram illustrates diagrams illustrate the workflow:


Image Added



CONVERT/ddl folder versus DIRECT/sql_direct folder


Info

If you use version 7.15 and later, use the DDL_DIRECT package method/ddl_direct folder instead of DIRECT package method/sql_direct folder.

The ddl_direct package method allows you to have more control over the processing order of scripts using the DIRECT package method so that scripts with DDL statements run before scripts with data changes that rely on those DDL changes.

You can use ddl_direct as follows:

  • A ddl_direct fixed folder name

  • A ddl_direct package method set for any other folder. You can set it using the metadata.properties file in any flexible folder where you want to use it: packageMethod=ddl_direct

An example of the new processing order for fixed folders: ddl, ddl_direct, view, function, procedure, package, packageBody, trigger, sql, sql_direct, data_dml.

An example of the new processing order when using flexible folders (where packageMethod is defined in metadata.properties): convert, ddl_direct, storedlogic, sqlfile, direct, data_dml.

If you use version 7.14 or earlier, refer to the following instructions.


...

If the order of the statements in the script is critical or if the script contains statements that would not be represented in a change set generated by diffing the before and after states of the database, you could instead package those scripts with the DIRECT method (which is used by default in the sql_direct folder). Scripts packaged with the DIRECT method in the sql_direct folder (or other folders that you have set to use the DIRECT packaging method) will run the actual script, thereby preserving the statements and the order of the statements.

Note that the possible drawback with using the DIRECT method/sql_direct folder with some configurations is losing some of the simulation/forecasting/modeling/profiling abilities that may happen with the CONVERT method/ddl folder. For Oracle you could use Datical's optional SQL Parser feature to add forecasting to the DIRECT method/sql_direct folder (or other folders that you have set to use the DIRECT packaging method).

...