Hello,
We recently upgraded from 3.12.5 to 3.12.9 and ran in to numerous issues during the tentacle upgrade process. We have multiple tentacles running on our Octopus Deploy server under different accounts which have different access to different network files. Tentacles on other servers worked fine, but this cluster of tentacles ran in to numerous issues. After diving through the logs, it appears that tentacles were wrongly trying to shut each other down, and failing. The task log gave us this tidbit:
Host Name: OCTOPUSPRD
June 7th 2017 19:22:12Verbose
Running As: Domain\Proxy01 (Local Administrator: False)
However, in the actual Local Proxy01 tentacle upgradelog, it is trying to stop the wrong tentacle
> Octopus upgrader version 3.12.9.0
> Current directory: D:\Octopus\Proxy01\Upgrade\20170608022158-3C2PQ
> Arguments:
> [0] = "OctopusDeploy Tentacle"
> [1] = "D:\Octopus\Proxy01\Upgrade\20170608022158-3C2PQ\Octopus.Tentacle.msi"
> [2] = "D:\Octopus\Proxy01\Upgrade\20170608022158-3C2PQ\Octopus.Tentacle-x64.msi"
> Upgrade mutex acquired
> Stopping service: OctopusDeploy Tentacle: Proxy02
> Stopping service...
> Error: System.InvalidOperationException: Cannot open OctopusDeploy Tentacle:Proxy02 service on computer '.'. ---> > System.ComponentModel.Win32Exception: Access is denied
> --- End of inner exception stack trace ---
> at System.ServiceProcess.ServiceController.GetServiceHandle(Int32 desiredAccess)
> at System.ServiceProcess.ServiceController.Stop()
> at Octopus.Upgrader.ServiceBouncer.StopService(ServiceController service) in Z:\buildAgent\workDir\eec88466c176b607\source\Octopus.Upgrader\ServiceBouncer.cs:line 51
> at Octopus.Upgrader.ServiceBouncer.StopAll() in Z:\buildAgent\workDir\eec88466c176b607\source\Octopus.Upgrader\ServiceBouncer.cs:line 25
> at Octopus.Upgrader.Program.PerformUpgrade(String[] args) in Z:\buildAgent\workDir\eec88466c176b607\source\Octopus.Upgrader\Program.cs:line 43
> Starting: OctopusDeploy Tentacle: Proxy02
> Service started
Note that the Proxy01 tentacle is attempting to shut down the Proxy02 service. This type of error occurred for all the ‘proxy’ tentacles we have running on this server (of which there are ~15).
The next day we took one of the proxy accounts and added it to the Local Admin group. We then went back to Octopus and upgraded that lone tentacle successfully. When we checked back in later, all the other tentacles on the machine were reporting in healthy and upgraded, without any task log event on the Octopus server. In part, we suspect that the upgrades succeeded but was having trouble reporting that back to the server, and were all exiting -1, but that’s for you to decide.