After the successful update to Octopus Deploy v3.5.1, I tried updating Calamari on all the tentacles. All went well except for a few named tentacles we have configured. These tentacles use a dedicated user and of course a different port.
Below is the upgrade log of one of those tentacles.
I ended up installing the Octopus.Tentacle-x64.exe msi (located in that same folder) manually and after that clicking the Update Calamari button in Environment. Octopus then recognized the new version was installed and it was good to go.
I’d like to know what went wrong and what we need to fix to prevent this from happening in the future.
Kind regards,
Emiel
Octopus upgrader version 3.5.1.0
Current directory: C:\Octopus\Backup Operator\Upgrade\20161110091347-5DS8H
Arguments:
[0] = "OctopusDeploy Tentacle"
[1] = "C:\Octopus\Backup Operator\Upgrade\20161110091347-5DS8H\Octopus.Tentacle.msi"
[2] = "C:\Octopus\Backup Operator\Upgrade\20161110091347-5DS8H\Octopus.Tentacle-x64.msi"
Upgrade mutex acquired
Stopping service: OctopusDeploy Tentacle
Stopping service...
Starting: OctopusDeploy Tentacle
Service started
Error: System.InvalidOperationException: Cannot open OctopusDeploy Tentacle service on computer '.'. ---> System.ComponentModel.Win32Exception: Access is denied
--- End of inner exception stack trace ---
at System.ServiceProcess.ServiceController.GetServiceHandle(Int32 desiredAccess)
at System.ServiceProcess.ServiceController.Stop()
at Octopus.Upgrader.ServiceBouncer.StopService(ServiceController service) in Z:\BuildAgent\work\cd33152532ae115f\source\Octopus.Upgrader\ServiceBouncer.cs:line 50
at Octopus.Upgrader.ServiceBouncer.StopAll() in Z:\BuildAgent\work\cd33152532ae115f\source\Octopus.Upgrader\ServiceBouncer.cs:line 29
at Octopus.Upgrader.Program.Main(String[] args) in Z:\BuildAgent\work\cd33152532ae115f\source\Octopus.Upgrader\Program.cs:line 38
Hi Emiel,
Thanks for getting in touch.
It sounds like the dedicated user that Tentacle is running as (for those Tentacles that failed) does not have service rights / permissions to start/stop services. The “Permissions” section of the Tentacles documentation has a list of requirements. Can you please confirm if the dedicated user meets those requirements?
I’m guessing when you logged in and ran the installer manually you were an Administrator with full rights, whereas the Tentacle user does not have the service rights it requires.
We have created a Github issue to improve logging in the future. You can track this issue if you wish to be notified when this logging has been improved.
Hope this helps.
Cheers
Mark
Hi Mark!
Thank you for your prompt response. I looked at that Permissions section. I think it’s folder/file permission(s) that is getting in our way. I will modify the user’s permissions and try again.
Is there an easy way to test if the user operating the Tentacle has all the permissions it needs other than waiting for a new Calamari update to come along?
Kind regards,
Emiel
Hi Emiel,
If you manually install an older version of Tentacle, Octopus should detect that it’s running an older version and give you the option to upgrade on the environments screen.
I’d recommend setting up a test environment and test Tentacle (grab an older version of Tentacle from the downloads page, then once Octopus completes a health check it should give you the option to upgrade and you can test that your permissions work.
Hope this helps.
Cheers
Mark
Hello Mark!
Yesterday I had another go at this. I successfully upgraded Octopus Deploy to version 3.7.8. The problem described above still occurs on two of our machines.
When I looked at the log file I noticed that it says Cannot open OctopusDeploy Tentacle service on computer. However, the tentacle running under the dedicated domain account and using a different port is called OctopusDeploy Tentacle: Backup Operator.
Since we’re not using the default tentacle on that particular machine, I decided to delete that instance through the Tentacle Manager and trying the update again. The result seems to be the same. The Task log in Octopus shows this:
Running Tentacle version 3.5.1
December 20th 2016 10:19:57
Info Beginning upgrade
December 20th 2016 10:19:59
Info Waiting for Tentacle to shut down and restart with the new version 3.7.8...
December 20th 2016 10:20:01
Info Running Tentacle version 3.5.1
December 20th 2016 10:20:06
Info Running Tentacle version 3.5.1
December 20th 2016 10:20:12
Info Running Tentacle version 3.5.1
December 20th 2016 10:20:17
Info Running Tentacle version 3.5.1
December 20th 2016 10:20:23
Info Running Tentacle version 3.5.1
...
and it keeps going like that.
The log files in C:\Octopus\Backup Operator\Upgrade\20161220091956-7XXT2 have different contents now:
UpgradeLog-20161220092001-2aca9.txt
Octopus upgrader version 3.7.8.0
Current directory: C:\Octopus\Backup Operator\Upgrade\20161220091956-7XXT2
Arguments:
[0] = "OctopusDeploy Tentacle"
[1] = "C:\Octopus\Backup Operator\Upgrade\20161220091956-7XXT2\Octopus.Tentacle.msi"
[2] = "C:\Octopus\Backup Operator\Upgrade\20161220091956-7XXT2\Octopus.Tentacle-x64.msi"
Upgrade mutex acquired
Running MSIEXEC on MSI: C:\Octopus\Backup Operator\Upgrade\20161220091956-7XXT2\Octopus.Tentacle-x64.msi, output will go to: UpgradeLog-MsiExec-fef5ae8e-bc99-46e7-beb5-284e2a647da8.txt
MSIEXEC exit code was: 1601
UpgradeLog-MsiExec-fef5ae8e-bc99-46e7-beb5-284e2a647da8.txt
MSI (c) (A8:40) [10:20:04:559]: Failed to connect to server. Error: 0x80070005
1: 2774 2: 0x80070005
I hope you can help!
Kind regards,
Emiel
Hi Emiel,
That last MSI error is interesting: Failed to connect to server. Error: 0x80070005
Do you have any anti-virus software that could be interfering here? (A quick test would be to temporarily disable any AV software and see if that then allows the MSI to run).
Also, does your Tentacle user have permissions to install software / run MSIs? You could try running the installer as the Tentacle user (which shouldn’t work, then you can try and trace down the permissions problem from there). Alternatively, you could elevate the permissions on your Tentacle user to run as Local System and see if that then lets the Tentacle user run the MSI.
Let me know how you go.
Cheers
Mark
Hi Mark,
There’s no anti-virus software installed on the machine. I did however disable the firewall software completely and manually installed an older version of the tentacle (3.7.6) before having a go again.
Same results.
Then I tried the following. Open a cmd.exe with Run as a different User option, using the credentials of the user ODBackupOntw that the Tentacle is running under.
I started the upgrader with the arguments shown in the original post.
Octopus.Upgrader.exe "OctopusDeploy Tentacle" "C:\Octopus\Backup Operator\Upgrade\20161110091347-5DS8H\Octopus.Tentacle.msi" "C:\Octopus\Backup Operator\Upgrade\20161110091347-5DS8H\Octopus.Tentacle-x64.msi"
That resulted in the following logs.
Octopus upgrader version 3.7.8.0
Current directory: C:\Octopus\Backup Operator\Upgrade\20161223113725-4Q8L3
Arguments:
[0] = "OctopusDeploy Tentacle"
[1] = "C:\Octopus\Backup Operator\Upgrade\20161223113725-4Q8L3\Octopus.Tentacle.msi"
[2] = "C:\Octopus\Backup Operator\Upgrade\20161223113725-4Q8L3\Octopus.Tentacle-x64.msi"
Upgrade mutex acquired
Stopping service: OctopusDeploy Tentacle: Backup Operator
Stopping service...
Waiting for service to stop. Current status: StopPending
Waiting for service to stop. Current status: Stopped
Service stopped
Running MSIEXEC on MSI: C:\Octopus\Backup Operator\Upgrade\20161223113725-4Q8L3\Octopus.Tentacle-x64.msi, output will go to: UpgradeLog-MsiExec-416ab02c-064e-4d20-b05a-db80e273c12b.txt
MSIEXEC exit code was: 1625
Updating executable path of OctopusDeploy Tentacle: Backup Operator, from "C:\Program Files\Octopus Deploy\Tentacle\Tentacle.exe" run --instance="Backup Operator" to "C:\Program Files\Octopus Deploy\Tentacle\\Tentacle.exe" run --instance="Backup Operator"
Running SC.exe config "OctopusDeploy Tentacle: Backup Operator" binpath= "\"C:\Program Files\Octopus Deploy\Tentacle\\Tentacle.exe\" run --instance=\"Backup Operator\""
SC.exe exit code was: 5
Starting: OctopusDeploy Tentacle: Backup Operator
Waiting for service to start. Current status: StartPending
Waiting for service to start. Current status: Running
Service started
and
MSI (s) (E0:4C) [14:39:37:571]: Product: Octopus Deploy Tentacle -- Installation failed.
MSI (s) (E0:4C) [14:39:37:571]: Windows Installer installed the product. Product Name: Octopus Deploy Tentacle. Product Version: 3.7.8. Product Language: 1033. Manufacturer: Octopus Deploy Pty. Ltd.. Installation success or error status: 1625.
Info 1625. This installation is forbidden by system policy. Contact your system administrator.
C:\Octopus\Backup Operator\Upgrade\20161223113725-4Q8L3\Octopus.Tentacle-x64.msi
In the EventLog 4 events are logged:
Beginning a Windows Installer transaction: C:\Octopus\Backup Operator\Upgrade\20161223113725-4Q8L3\Octopus.Tentacle-x64.msi. Client Process Id: 8972.Product: Octopus Deploy Tentacle -- Installation failed.Windows Installer installed the product. Product Name: Octopus Deploy Tentacle. Product Version: 3.7.8. Product Language: 1033. Manufacturer: Octopus Deploy Pty. Ltd.. Installation success or error status: 1625.Ending a Windows Installer transaction: C:\Octopus\Backup Operator\Upgrade\20161223113725-4Q8L3\Octopus.Tentacle-x64.msi. Client Process Id: 8972.
Kind regards,
Emiel
Hi Emiel,
Good work. This appears to be a Windows issue and it looks like you’ve got a message to help track down the problem: “Info 1625. This installation is forbidden by system policy. Contact your system administrator.”
We can’t advise on your Windows software restriction policies, but if you Google that error message e.g. “This installation is forbidden by system policy”, you can review various Microsoft forums and see if any of the answers help to fix this problem for you.
Once you’re able to manually run Octopus.Upgrader.exe successfully using those credentials/arguments you mentioned (ODBackupOntw), you can re-try from the Octopus UI.
Hope this helps.
Cheers
Mark