Importing development work into existing production server

We are currently wrapping up a bunch of development work that we would like to move into an existing production Octopus Server. Throughout the development phase we were regularly exporting json objects and saving in source control. Now we are getting ready to do some sort of mass import into the existing production server but I’m having difficulty find the suggested approach. I did find some tooling that supports importing projects and releases but we have to also include Script Modules, Custom Step Templates, Project Runbooks, Project Variables and Variable Templates, Tenant Tag Sets, and Variable Set. When possible, I’d like to utilize the json code we have in source control as that is the source of truth.

Any help would be greatly appreciated,
Dave Schmitz

So the bad news first. We are currently working on the configuration as code functionality. With it you can save your project in Git repo instead of the database.

We don’t have a set date when this will be production-ready. We are thinking sometime in November, but there are a lot of kinks to iron out.

The good news, we do have some tooling available to help out.

Option #1 - The migrator command-line tool.

With this you can export everything for a project, the process, the step templates, script modules, accounts, etc. as JSON files to be saved in source control. It will encrypt all sensitive items to be imported as well.

The downside is this tool is very heavy-handed. It will do it’s very best to ensure what has been exported matches what is imported. Which is a great thing, until you get to lifecycles and environments and variables.

Please read the linked documentation to see the pros and cons to that tool.

Option #2 - The space cloner script

This script does not export JSON, rather it reads from the test server’s API and clones the process over to the production server using its API. Because it is using the API, it won’t have access to your sensitive variables or accounts. You’d have to rekey them on the production server.

It was designed from the ground up to support the use case of wanting to clone a process from one server to another. You can exclude specific environments or include all of them. You can elect to overwrite existing variables or leave them as is. Please see the how it works page for more details.

I hope that helps!

Thank you for response @Bob_Walker.

I agree the migrator option might work but is very heavy-handed and might run into issues when using against an existing production server that does not match the development server we are using. I could see conflicts being a issue.

I did start reading through the documentation regarding the space cloner and feel this could be suited to our needs. One thing that did catch my eye was it shows support for 2020.1.x and 2020.2.x but did not see support for 2020.3.x, the latest which we are using. Internally has this tool still been used against the latest release line?

Ah, apologize for the confusion. It was developed using an instance running 2020.1 and later 2020.2. I run regular syncs for my internal Octopus instances, which are always running the latest version of Octopus I can get my hands on.

Thank you for the clarification. I have since downloaded and used the Space Cloner tool as it appeared to fit my need better. I’m currently in the process of reviewing the cloned projects between the two different servers. It did appear that the Space Cloner did not clone the project script module dependencies but it was very quick to go back and add afterwards.

Thanks for letting me know. I’ll take a look to see if I missed anything. Did it clone the script modules themselves? It just didn’t clone the project/script module relationship correctly?

I had manually created the script modules so it was just the project/script module relationship. I also just came across the project variable scopes being lost as well. For now, I am manually re-adding the scopes.

Could you share an example script you ran calling the cloner?

Sure.
.\CloneSpaceProject.ps1
-SourceOctopusUrl $sourceUrl -SourceOctopusApiKey $sourceKey
-SourceSpaceName $sourceSpace -DestinationOctopusUrl $destinationUrl
-DestinationOctopusApiKey $destinationKey -DestinationSpaceName $destinationSpace
-ProjectsToClone $projectstoClone -EnvironmentsToExclude "all"
-WorkersToExclude “all” -TargetsToExclude "all"
-TenantsToExclude “all” -OverwriteExistingVariables "false"
-AddAdditionalVariableValuesOnExistingVariableSets “false” -OverwriteExistingCustomStepTemplates "false"
-OverwriteExistingLifecyclesPhases “false” -CloneProjectRunbooks "true"
-CloneTeamUserRoleScoping “false” -CloneProjectChannelRules "false"
-CloneProjectVersioningReleaseCreationSettings “false”`

Was the missing variable scoping related to environments?

Just so you are aware. I did have to do one small code change to the OctopusDataAdapter.ps1 script to work with my original server since it was installed to a named virtual directory and I found the name in both the url and occasionally the end point name. Here is the method that was changed.

function Get-OctopusUrl
{
param (
$EndPoint,
$SpaceId,
$OctopusUrl
)

if ($EndPoint -match "/api")
{
    if (!$EndPoint.StartsWith("/api"))
    {
        $EndPoint = $EndPoint.Substring($EndPoint.IndexOf("/api"))
    }

    return "$OctopusUrl/$endPoint"
}

if ([string]::IsNullOrWhiteSpace($SpaceId))
{
    return "$OctopusUrl/api/$EndPoint"
}

return "$OctopusUrl/api/$spaceId/$EndPoint"

}

No the missing variable scoping was processes related to Deployment Process or a Project Runbook

Thanks for letting me know. v1.2.1 of the space cloner should fix those issues: https://github.com/OctopusDeployLabs/SpaceCloner/releases/tag/v1.2.1

Awesome…thank you very much Bob for addressing these.

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.