Hi there,
We are planning on migration of projects from one server to another but lots of data like project groups, lifecycles, environments, machine names are being changed to different naming conventions. Is there a way just to migrate projects and its process steps without pulling all these dependencies?
Is it actually possible to just migrate project and its process without pulling any of its dependencies like feeds, project groups, lifecycles etc…?
I have tried to edit the Json and stripped off most of the data which we are not migrating and imported but gives me errors… as below
Finding importer 'project’
Validating the import
Export file successfully loaded
The data is not a valid ProjectWithDependencies
Exit code: -1
Hi!
Thanks for getting in touch! Unfortunately in this case you’re right - the migrator tool is designed to bring a project and all of its dependencies across to another Octopus Server. It does this based on name-matching. You cannot avoid bringing the dependencies across, and I don’t recommend hand editing the exported files.
Is this a one-time migration for these projects? If so, I can think of two options:
- Rename the dependencies on the source Octopus Server to match the names on the new Octopus Server. The migrator matches based on names, and would see these as the same Lifecycles/ProjectGroups and “just work”.
- Bring the projects along with all their dependencies, ending up with “duplicates”. After that you could go through and connect the project to the desired Project Group and Lifecycle etc, then delete the unwanted Project Groups and Lifecycles.
If this is not a one-time migration we have a new feature we are planning called “Remote Release Promotions” you may like to investigate: https://octopus.com/blog/remote-release-promotions-rfc
Hope that helps!
Mike
I have decided on migrating projects in to a test server, make necessary changes there and migrate project wise after name matching.
Another problem I just ran in to today is that, I was using an project export to export and project and got struck at this step.
Handshaking with Octopus server: http://octoxxxxxxxx
Handshake successful. Octopus version: 3.4.14; API version: 3.0.0
Authenticated as: xxxxxxxxxxxxxxxxxxxxxxxxxxx
Finding exporter ‘project’
Beginning the export
Finding project: xxxxxxxxxx
Finding project group for project
Finding variable set for project
Finding deployment process for project
Error from Octopus server (HTTP 200): Unable to process response from server: Error reading string. Unexpected token: StartObject. Path ‘Steps[3].Actions[0]
us.Action.WindowsService.CustomAccountPassword’]’, line 138, position 69… Response content: {
“Id”: “deploymentprocess-Projects-144”,
“ProjectId”: “Projects-144”,
“Steps”: [
{
Exit code: -7
steps before getting this error are:
- I have migrated all the data toe test server,
- made changes to this one project like project groups, machine names, environment names, new lifecycle… etc…
- ran an export of this project when its ready to be migrated and expected to get the jsonfile of this export but threw an error as above.,
Please help/advise.
Thank you
Hi!
Thanks for keeping in touch! It looks like you’re using octo.exe export - is that correct? I would recommend using Octopus.Migrator.exe partial-export instead: learn more
Please let me know how that goes. 
Hope that helps!
Mike
Hi Michael.
Octopus.migrator seems like not part of the octo command line. I want to be able to import/export from any machine hence I am using octo import/export. I think octopus.migrator.exe has to be performed on the server and its pain to get the access and do work on production servers.
Hi!
Thanks for keeping in touch! That’s right, we have two separate command-line tools for achieving similar things, but octo.exe via the HTTP API, and Octopus.Migrator.exe via direct SQL Database access.
In order for me to help you constructively, can you answer these questions?
- Is this a one-time migration, or are you planning to repeat the migration periodically?
- Which versions of Octopus Server are you migrating between? The source and target servers must be on the exact same version. I can see the target is
3.4.14 but I can’t see the source version.
- Which version of
octo.exe are you using? You should tend to use the latest available version.
- Are you willing and able to upgrade your Octopus Servers to the latest version? Octopus 3.4 is quite old now, and if we need to patch something to help you out, it will be patched on the latest version.
Hope that helps!
Mike
Is this a one-time migration, or are you planning to repeat the migration periodically?
This is a periodic migration- few projects at a time
Which versions of Octopus Server are you migrating between? The source and target servers must be on the exact same version. I can see the target is 3.4.14 but I can’t see the source version.
Target version is also same as source version
Which version of octo.exe are you using? You should tend to use the latest available version.
dowloaded the latest available octo version
Are you willing and able to upgrade your Octopus Servers to the latest version? Octopus 3.4 is quite old now, and if we need to patch something to help you out, it will be patched on the latest version.
I doubt it, it may require big effort and we are currently in the process of migration and unsure about what the team thinks.
I was able to migrate one test project with with octo.exe, but i get this below error. Although there is an error project is imported fine (I guess). I am just unsure if I missed anything or not. Any clue why would this error pop up?
C:>octo import --server=XXXXXXX --apiKey=API-Vxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --type=project --filePath=C:\TEMP\Octopus\xxxxxxxxxx.json
Octopus Deploy Command Line Tool, version 4.15.7
Handshaking with Octopus server: http://xxxxxxx
Handshake successful. Octopus version: 3.4.14; API version: 3.0.0
Authenticated as: XXXXXXXXXXXXX
Finding importer 'project’
Validating the import
Export file successfully loaded
Checking that lifecycle Harmony exists
Found lifecycle 'Harmony’
Checking that all environments exist
Checking that all machines exist
Checking that all NuGet Feeds exist
Checking that all Action Templates exist
Checking that all Library Variable Sets exist
Checking that the Project Group exist
Checking that all channel lifecycles exist
No validation errors found. Project is ready to import.
Beginning the import
Beginning import of project 'xxxxxxxxxxxxxxx’
Importing Project
Project already exist, project will be updated with new settings
Importing the channels for the project
Channel ‘Default’ already exists, channel will be updated with new settings
Importing the Projects Deployment Process
Updating ID and version of Action Template
Updating IDs of Environments
Updating IDs of Channels
Updating ID and version of Action Template
Updating IDs of Environments
Updating IDs of Channels
Updating ID and version of Action Template
Updating IDs of Environments
Updating IDs of Channels
Updating IDs of Environments
Updating IDs of Channels
Updating ID and version of Action Template
Updating IDs of Environments
Updating IDs of Channels
Updating IDs of Environments
Updating IDs of Channels
Updating IDs of Environments
Updating IDs of Channels
Updating ID and version of Action Template
Updating IDs of Environments
Updating IDs of Channels
Importing the Projects Variable Set
Updating the Environment IDs of the Variables scope
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
at System.Collections.Generic.Dictionary2.get_Item(TKey key) at Octopus.Cli.Importers.ProjectImporter.UpdateVariables(VariableSetResource variableSet, IDictionary2 environments, IDictionary2 machines, IDictionary2
at Octopus.Cli.Importers.ProjectImporter.d__17.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at Octopus.Cli.Importers.ProjectImporter.d__13.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Octopus.Cli.Commands.ImportCommand.d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at Octopus.Cli.Commands.ApiCommand.d__31.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Octopus.Cli.Program.Run(String[] args)
Exit code: -3
Hi!
Thanks for keeping in touch! There were some issues with octo.exe and some versions of Octopus Server 3.4.x. The import you’ve run will be partially complete - octo.exe import works its way through the exported items, and pushes them via the HTTP API into the target server, then stops if it encounters an error. In this case your project will be partially successful.
From what you said, this sounds like you’ll be doing a “staged one-time migration” of these projects, one project at a time. I would strongly recommend a couple of things because it’s the way you’ll get the most successful migration:
- Upgrade your source and target servers to the latest version of Octopus. We have continually worked to fix any bugs in the migration processes, which has stabilised in later versions of Octopus.
- Go to the effort to use
Octopus.Migrator.exe partial-export instead of octo.exe export. The Octopus.Migrator.exe was built for this specific purpose, has support for doing a “dry run” import so you don’t get this kind of “partial success”, and allows for some advanced deduplication capabilities.
- Make sure you take a backup of the target server before each import just in case something does go wrong.
Hope that helps!
Mike
Hi Mike,
Thanks for the response. I will need to check with the team about octopus version upgrades.
Can we use Octopus.Migrator.exe partial-export from any machine or this has to be performed from the server? I want to be able to perform this migration of projects from an external machine, hence I am using the import/export.
Hi!
Thanks for keeping in touch! Running Octopus.Migrator.exe from the Octopus Server machine(s) is certainly the easiest way to make it work. You would do the export on the source Octopus Server machine, then run the import on the target Octopus Server machine.
The Octopus.Migrator.exe requires access to the SQL Server. You will also need to ensure the user account running Octopus.Migrator.exe has the right roles on each SQL Server.
This may not be a desirable situation if you were planning to run these migrations indefinitely, however it seems reasonable given this will end up being a staged, time-limited, one-time migration.
Hope that helps!
Mike