How To: Recover when Archive Files have been Modified

Once a file has been moved to the archive folder it should no longer be modified. In the event that a file is updated in the archive folder there are a couple of methods for fixing.

Archive files have been modified error

The error you will see in the logs when an archive file has been modified is as below.

Datical DB was able to locate existing Change Sets for the script 'orders_table_05.sql'
but the checksums that were associated with those Change Sets when the script was originally packaged do not match the checksum generated from the current script.

Current Checksum Value:  81ebbca24c7e47f8bb40d1fce88b2dd7
________________________________________________________________________________
| Change Set ID                    | Author   | scriptChecksum Value            |
|===============================================================================|
| 20210428150912989_orders_table_05| Amy Smith| d30eebb4d0ac21f7eafdcb2567ec5edc|

ERROR: Datical DB was able to locate existing Change Sets for the script 'orders_table_05.sql'
but the checksums that were associated with those Change Sets when the script was
originally packaged do not match the checksum generated from the current script.
This typically indicates that the script has changed since it was originally
packaged OR you are attempting to manage different scripts with identical names.
If this was an expected change, you can rename the current file and check it 
into the appropriate folder outside of the archive to re-Package it.

The error is similar to the error you will get for trying to package a file with the same name as a previously packaged file. To distinguish that this error is because a archive file was modified, check the section of the log for PACKAGING ORDER FOR COMMIT FILES. You will see a file in the archive folder that is attempting to be packaged. For example:

Method 1: Update the sqlScmLastImportID for the branch in deployPackager.properties

By using this method, you are essentially skipping over any SQL commits that occurred between the last time Packager successfully packaged and when the archive folder was modified.

On the packaging branch, find the commitID where the archive file was modified.
1. This commitID should be present in the log in the section PACKAGING ORDER FOR COMMIT FILES.
2. 1. If there are changes after the above commitID dependent on changes between the last time Packager successfully packaged, just use the last commitID on the branch.
In the ddb repo, update the deployPackager.properties file <branch>.sqlScmLastImportID to use the commitID from step 1.
1. The existing <branch>.sqlScmLastImportID in the deployPackager.properties is the last time Packager ran successfully. This should also match the most recent “Datical automatic check-in” message if you are looking at the git commit log.
Run a Packaging job.
You will need to make small changes (eg. add a space) to any SQL files that were modified since the last time Packager ran successfully and the commitID you used in step 2 in order for Packager to pick up the changes and process them.
1. Commit these small changes to source control.
Run a Packaging job again to pick up the changes from all the SQL files modified in step 4.

Note: This method is not recommended for projects with multiple pipelines that are merged into one branch from another, eg. develop → qa → stress → prod. In this case, each pipeline branch would need to have the sqlScmLastImportID updated after each merge from the lower branch until the problem was resolved across all pipelines.

Method 2: Update the Packager branch to be based on commit prior to archive update

In this method, you are resetting the Packaging branch to an earlier state in git.

On the packaging branch, find the commitID prior to when the archive file was modified.
1. This commit containing the modified archive file should be present in the log in the section PACKAGING ORDER FOR COMMIT FILES.
2. Look in SCM commit history or use ‘git log’ to find the commit prior to the archive file commit.
Rename the existing packaging branch to eg. <branch>-archive. Instructions:
1. git checkout <branch> git branch -m <branch>-archive git push origin -u <branch>-archive git push origin --delete <branch>
Create a new git branch from the commitID found in step 1. Use the original branch name. Instructions:
1. git checkout <branch>-archive git checkout -b <branch> <commit_id_prior_to_commit_where_archive_file_was_modified> git push origin -u <branch>
2. You should not need to update the deployPackager.properties <branch>.sqlScmLastImportID as you do in Method 1. That commitID should still be present in the branch you’ve just created.
Run a Packaging job.
Any SQL file changes made after the commitID will no longer be in the branch. They will needed to be added back. The <branch>-archive branch should be available for reference.
Commit back to source control after adding back the changes and run Packaging job again.