Tentacles won't upgrade since upgrade to 3.4.13

hsessom · 18 October 2016 18:44

Yesterday I upgraded from version 3.4.10 to 3.4.13 and created an HA cluster with a new Octopus node and since then tentacles won’t complete the update process. I chose a single environment and click the green upgrade these machines button and 98% of them will finish but one or two will hang up and say “Cannot start this task yet. There is already another task running that cannot be run in conjunction with any other task. Please wait…”. I’ve got about 300 tentacles and at this point only a small set of them have finished upgrading. There are no deployments running when I try to upgrade but sometimes the tentacle health check task will kick off mid way through my upgrade task. I am wondering if this is causing the conflict and if there is a way to stop it from running temporarily so I can get the tentacles upgraded. Is this a common issue with this new build? What can I do to automatically get the tentacles upgraded without disruption? Thank you!

P.S. Another common error is below. It just keeps running through this and never fails or completes.

Escapia_astweb200
October 18th 2016 13:43:04Info
Running Tentacle version 3.3.4
October 18th 2016 13:43:04Info
Beginning upgrade
October 18th 2016 13:43:17Info
Waiting for Tentacle to shut down and restart with the new version 3.4.13…
October 18th 2016 13:43:23Info
Tentacle has stopped. Waiting for it to start…
October 18th 2016 13:43:27Info
Tentacle has stopped. Waiting for it to start…
October 18th 2016 13:43:32Info
Tentacle has stopped. Waiting for it to start…
October 18th 2016 13:43:36Info
Tentacle has stopped. Waiting for it to start…
October 18th 2016 13:43:40Info
Tentacle has stopped. Waiting for it to start…
October 18th 2016 13:43:50Info
Tentacle has stopped. Waiting for it to start…
October 18th 2016 13:44:00Info
Tentacle has stopped. Waiting for it to start…
October 18th 2016 13:44:22Info
Running Tentacle version 3.3.4
October 18th 2016 13:44:30Info
Running Tentacle version 3.3.4
October 18th 2016 13:44:39Info
Running Tentacle version 3.3.4
October 18th 2016 13:44:47Info
Running Tentacle version 3.3.4
October 18th 2016 13:44:54Info
Running Tentacle version 3.3.4
October 18th 2016 13:45:02Info
Running Tentacle version 3.3.4
October 18th 2016 13:45:11Info
Running Tentacle version 3.3.4
October 18th 2016 13:45:20Info
Running Tentacle version 3.3.4
October 18th 2016 13:45:29Info
Running Tentacle version 3.3.4
October 18th 2016 13:45:38Info
Running Tentacle version 3.3.4
October 18th 2016 13:45:45Info
Running Tentacle version 3.3.4
October 18th 2016 13:45:54Info
Running Tentacle version 3.3.4
October 18th 2016 13:46:02Info
Running Tentacle version 3.3.4
October 18th 2016 13:46:12Info
Running Tentacle version 3.3.4
October 18th 2016 13:46:20Info
Running Tentacle version 3.3.4
October 18th 2016 13:46:29Info
Running Tentacle version 3.3.4
October 18th 2016 13:46:39Info
Running Tentacle version 3.3.4
October 18th 2016 13:46:47Info
Running Tentacle version 3.3.4
October 18th 2016 13:46:55Info
Running Tentacle version 3.3.4
October 18th 2016 13:47:03Info
Running Tentacle version 3.3.4
October 18th 2016 13:47:12Info
Running Tentacle version 3.3.4
October 18th 2016 13:47:20Info
Running Tentacle version 3.3.4
October 18th 2016 13:47:29Info
Running Tentacle version 3.3.4
October 18th 2016 13:47:39Info
Running Tentacle version 3.3.4
October 18th 2016 13:47:47Info
Running Tentacle version 3.3.4
October 18th 2016 13:47:55Info
Running Tentacle version 3.3.4
October 18th 2016 13:48:03Info
Running Tentacle version 3.3.4
October 18th 2016 13:48:12Info
Running Tentacle version 3.3.4
October 18th 2016 13:48:21Info
Running Tentacle version 3.3.4
October 18th 2016 13:48:30Info
Running Tentacle version 3.3.4
October 18th 2016 13:48:40Info
Running Tentacle version 3.3.4
October 18th 2016 13:48:48Info
Running Tentacle version 3.3.4
October 18th 2016 13:48:56Info
Running Tentacle version 3.3.4
October 18th 2016 13:49:04Info
Running Tentacle version 3.3.4
October 18th 2016 13:49:12Info
Running Tentacle version 3.3.4

Shane_Gill · 19 October 2016 02:35

Hi,

The second issue is easier so let’s start there. Sometimes the Tentacle MSI fails to install the new version of Tentacle but we continue to expect the new version to come online. There is an issue open here to address it: https://github.com/OctopusDeploy/Issues/issues/2468

To help that cause, when a Tentacle is stuck during an upgrade would you mind providing the .txt and .log files located on the Tentacle in C:\Octopus\Upgrade? They will help us determine why the installation failed.

The log “Cannot start this task yet. There is already another task running that cannot be run in conjunction with any other task. Please wait…” is supposed to indicate that there is a script already running on that Tentacle. In 3.4.13 we added checks to make sure that an upgrade doesn’t happen while a deployment is in progress but the log message is different. Would you mind sending your full raw Tentacle upgrade log?

Cheers,
Shane