Export/Import problem

I am trying to export a project from our onprem Octopus and then import it to Octopus Cloud. When I try to import I get the following message:

Unexpected duplicate action template ID’s: ActionTemplates-44, ActionTemplates-9. Reference path: Project ‘CALM’ => Release ‘1.1.343.1-hotfix’ => Deployment Action ‘Run Migrations’ => Action Template ‘Run Migrations - FluentMigrator v2’

I have had this sort of thing before and I was able to resolve it by deleting the Release. However when I go into the Project > Releases then click on this release I get the following message:

The resource ‘74dd892c-3253-4ccf-9a35-40b4a106e591’ was not found.

with no option to delete it or anything.

Please advise how I can get around this so I can export/import the project.

Hi @darren.woodford,

Thanks for reaching out!

Would you be able to send over the export that you’ve created of the project that has the issue importing, please?
I’ve created a secure upload link so that you can send it over instead of posting it here on the forums.

Here’s the link for upload: Secure Upload
Could you also PM me the password for the export, please?

Please let me know if you have any questions or concerns.

Kind Regards,
Adam

Hi Darren,

Thanks for sending over the export!

I’m going to bring up the issue regarding duplicate Action Templates internally to see if I can get more details on why this occurs.
The workaround you’ve suggested of deleting the release does appear to work. However, in this case, you’re clearly blocked by the error The resource ‘74dd892c-3253-4ccf-9a35-40b4a106e591’ was not found.

To resolve this, would you be able to run a quick query against the database or have your DBA perform the same instead, if necessary?

We strongly recommend backing up your database and retrieving a copy of your master key before running any queries against the database.

In this case, it’s simply a select query, but depending on the results, we may have to modify something to get that error fixed!

We don’t recommend manually manipulating the database unless absolutely necessary as it may have unintended consequences.

It looks like the error is in regards to a package ID.
Thanks to the export you provided, I was able to locate this:
image

Can you run this query against the database, please, and let me know your results?

SELECT * FROM dbo.deploymentsettings
WHERE JSON like ‘%74dd892c-3253-4ccf-9a35-40b4a106e591%’

We’re hopefully looking for a result of: deploymentsettings-Projects-101-D4C63A85E7894C5D8C20D9297FEA1A43

From there, we’ll look to fix the issue with the deployment settings for this project to allow you to remove the releases that are stopping the import on the other side.

Please let me know if you have any questions or issues with the above!

Kind Regards,
Adam

I’ve sent over the results by return email. Please let me know if you haven’t received it and I will post it here.

Not exactly what I get:

Id: deploymentsettings-Projects-101

ProjectId: Projects-101

JSON: Spaces-1 {

“VersioningStrategy”: {

“DonorPackage”: {

“DeploymentActionId”: “997039a6-f176-4f1b-b5ff-d1deba24395e”,

“PackageReferenceId”: “74dd892c-3253-4ccf-9a35-40b4a106e591”

},

“Template”: null

},

“ConnectivityPolicy”: {

“SkipMachineBehavior”: “None”,

“TargetRoles”: [],

“AllowDeploymentsToNoTargets”: false,

“ExcludeUnhealthyTargets”: false

},

“ReleaseNotesTemplate”: null,

“DefaultToSkipIfAlreadyInstalled”: false,

“DeploymentChangesTemplate”: null,

“DefaultGuidedFailureMode”: “EnvironmentDefault”

}

DataVersion: 0x000000000008F196

Thanks, Darren!

That is more or less what we expected to see; it looks like there’s an issue with how the version is decided in those deployment settings, the link to the package deployment step that’s used for determining the versioning strategy appears to be corrupt or missing.

I’ve constructed a query that you can run to reset the deployment settings for that project, allowing you to remove the releases you’re hitting errors on the import with.

Before I give you the query, I’d like to reiterate that we don’t usually recommend customers edit their database directly, as it can have unintended consequences. I would strongly recommend you take a database backup and a copy of your master key before running this query.

Here’s the query to reset those deployment settings, after running this you will have to manually re-configure the deployment settings for that project if you wish to continue using it in the same manner:

UPDATE dbo.deploymentsettings
SET JSON = ‘{“VersioningStrategy”:{“DonorPackage”:null,“Template”:"#{Octopus.Version.LastMajor}.#{Octopus.Version.LastMinor}.#{Octopus.Version.NextPatch}"},“ReleaseNotesTemplate”:null,“DefaultToSkipIfAlreadyInstalled”:false,“ConnectivityPolicy”:{“SkipMachineBehavior”:“None”,“TargetRoles”:[],“AllowDeploymentsToNoTargets”:false,“ExcludeUnhealthyTargets”:false},“DeploymentChangesTemplate”:null,“DefaultGuidedFailureMode”:“EnvironmentDefault”}’
WHERE JSON like ‘%74dd892c-3253-4ccf-9a35-40b4a106e591%’

Once you run this, please check to see if you can delete the release you had that error on last time.

I’m currently conversing with our developers on the duplicate ActionTemplate error that you’re experiencing and will update you as soon as I have more information.
I’m actually curious as to whether the import will still throw those errors once the deployment settings have been fixed; they’re likely entirely unrelated, but please let me know if exporting a new file from this project after running that query allows you to import successfully.

I hope this helps! Let me know if you have any questions on the above or if you’re unsure about anything.

Kind Regards,
Adam

Thanks for that SQL Adam. Don’t worry I have backed up the database (twice) and master key. It has not fixed the duplicate ActionTemplate error but it has allowed me to delete the first problematic release. However that just gets me to the next problem release which I then have to delete and go through the export/import process again until I get the next one. So far I have done this about 10 times for this project and I have multiple projects with this issue so it’s getting a bit tiresome. Because the import stops as soon as it hits the first problem I don’t know there are any more problem releases until I delete the current one and go through the export/import process again.

Is there a way I can determine all of the problematic releases please? Also can I suggest that you tweak the import routine to show ALL errors and not just the first one, and provide an option when exporting/importing to exclude releases as we really don’t care about these. Perhaps I could delete all of the release.json files out of the ZIP file. Would that work?

Hi Darren,

Thanks for the update, and I’m glad to hear that the query at least resolved the issue with the ‘resource was not found’ error.

I’ve reached out to our developers to ask the question regarding Import/Export and the ActionTemplates issue; there may be a delay in receiving a response as our developers are based in Australia; however, I will update you as soon as I have any feedback.

We don’t recommend that you manually alter the exports. Still, when it comes to manually pruning the releases from the export, you would also have to remove any reference to the release from the manifest.json that comes packaged with the export.

Apologies I couldn’t give you a more immediate solution, but I will provide you with an update as soon as I have one.

Kind Regards,

1 Like

Will this JSON work for any project or is it project specific? I have several projects with this same issue. Obviously I would have to change the WHERE clause for each to find the correct resource GUID but the rest of it looks fairly generic.

Hi Darren,

Thanks for keeping in touch! I’m just jumping in here for Adam, as he’s currently offline as part of our UK-based team.

I had a read through this thread and the provided UPDATE SQL script, and it looks like the only project-identifying piece of it is the WHERE JSON like ‘%...%’ as you mentioned. So it should be safe to run the same UPDATE script, only updating the WHERE JSON like '%... line to point to the project in which is experiencing this same issue. That is to say, another project experiencing the same issue of a corrupt PackageReferenceId entry in its versioning strategy, as it’s updating the “DonorPackage” snippet to null.

Though just to emphasise again, we’d urge database backups before touching things directly in the database in this way, just in case a roll back is needed.

Best regards,

Kenny

1 Like

Any news from our antipodean developer friends?

Hi Darren!

It does look like the developers agree that the issue with the duplicate action templates is a bug.
Unfortunately, I don’t yet have a time frame for you on when this will be fixed.

I’m currently trying to establish if there is a workaround for this or something that we can do to get your projects migrated while we await a fix for the issue.

I appreciate your patience on this and apologies that I don’t have a definitive resolution for you as of yet.

Kind Regards,
Adam

For anyone else following this thread I have written a python script that deletes all of the releases from an exported project. All releases are deleted from the manifest.json file in the zip file and any files with a pattern of “Release*.json” are also deleted. This saves me having to go through the cycle of deleting one then exporting and importing again to get the next error. This works for us because it is some time since we used Octopus and the previous releases aren’t important to us. For the avoidance of doubt this script will create a new zip file without any releases in it. This solves the problem of duplicate steps (or any other problems in a release) but not the problem of “duplicate action template ID’s” in the project. You will need to resolve that first before exporting your project and running this script.

PLEASE MAKE SUITABLE BACKUPS BEFORE USING THIS SCRIPT. IT IS SUPPLIED WITHOUT ANY WARRANTIES IMPLIED OR OTHERWISE. USE IS COMPLETELY AT YOUR OWN RISK. PLEASE REVIEW THE CODE YOURSELF OR BY A PYTHON EXPERT BEFORE USING ANY THIRD PARTY CODE. IT IS UP TO YOU TO DETERMINE THE SUITABILITY TO YOUR OWN SITUATION. IF THIS SCRIPT DESTORYS YOUR COMPUTER OR KILLS YOUR CAT, IT IS YOUR OWN FAULT AND NOT MINE! CAVEAT EMPTOR, CAVEAT EMPTOR, CAVEAT EMPTOR, ETC ETC ETC.

Right, now that is out of the way… To use the attached script place it into the same folder as your export zip file. Change the zip filename on line 3 then save and run the script. Note you will need Python installed on your machine. Your existing zip file is unaffected and a new zip file with a " fixed" suffix is created.

Known limitations

  • I have tested this script with single project exports but not multiple projects.
  • When importing you will get a warning message due to the modification of the zip file.
  • When importing you may get warning messages due to extra json files that are no longer linked to a release.

remove-releases.py (1.1 KB)

1 Like

Hi Darren!

Thank you for sharing!

This is fantastic, I was working on a solution to import the project without the need to remove releases but your situation of not requiring the releases is also a common one.

If I do get to the bottom of a work-around before the developers are able to fix the issue then I will also share it here, for visibility of others.

Thanks again, Darren!

Kind Regards,
Adam

1 Like