I have a query regarding ‘Transfer a package’ step in Octopus.
What am I trying to do:
I have an Artifactory which stores large size packages in the form of NuGet.
Octopus takes an external NuGet feed from Artifactory.
Octopus uses ‘Transfer a package’ step to transfer NuGet package from Artifactory to multiple deployment targets. Note: Step is configured to copy package onto Octopus Server node, apply delta compression and then transfer to the deployment targets.
When above step is run, it first takes a copy of NuGet package onto Octopus Server Node and then transfers it to the deployment targets.
Question: Is it possible to replace Octopus SERVER Node with Octopus WORKER Node in the above scenario?
I tried to have a worker node in the ‘default worker pool’ and also switched off the Built-in worker ( Configuration ➜ Features page) but still the above step copies package into Octopus Server Node -> PackageCache folder instead of using Worker Node.
Why I want to do this: I need to run this step for more than 500 deployment targets in a deployment and at-least 10 deployment targets in-parallel at one time (or more in future). Trying to take-off the load from Octopus Server node.
Workers currently only really come in to play when running scripts, Azure, AWS, and Terraform steps. For these steps, the Execution plan in the process step will include an option to run on the worker or worker pool.
Due to this, the management of packages and orchestration of the entire deployment is still completely handled by the Octopus server.
One possible way to achieve what you’re aiming for would be to amend your process slightly. You could alter the ‘Transfer a package’ step to push the package only to your Worker machines. Then have the next step ‘Run a script’ that performs a simple file copy on this package and set it to run on the Worker on behalf of the deployment targets.
I haven’t fully tested out this idea, so there may be issues with it, and it would be adding complexity to the process.
I had a big confusion around management of packages and orchestration of the entire deployment but it is pretty much clear now.
Even I had this idea which you suggested but have a doubt. Step ‘Run a script’ to perform a simple file copy will run on the Worker on behalf of deployment targets. Will that Step require using username/password or ssh (of the target server) in the script to take a session on the target server and then perform a file copy? Or will Octopus be able to manage that with the tentacles installed on the target servers?
That is the part where I was having doubts about it too, as it will bypass the tentacles and require the entire transfer session creating in Powershell.
An easier option may be to setup some form of shared storage in your environment and have the deployment targets retrieve the package directly from there. Could cut out the Worker altogether and if the shared storage was mapped on the targets, the scripting would be a lot easier too.
