Tentacle upgrade hangs, and blocks any other tasks to be processed

Legacy Execution
1

We have recently upgraded to version 3.11.18, and have started to notice various issues with tasks queuing in the environment one of this versions of this I have been able to isolate is this situation.

We have one tentacle that we pushed out the tentacle upgrade process for, as it was causing issues performing the health check. This task starts and hangs (see attached log). While this was running two deployment tasks (for projects, machines, environments not connected to this tentacle in anyway) joined the queue. However these tasks would not start (so only one task was running in the environment at this time). After 20 minutes we cancelled the tentacle upgrade task, and immediately the two deployment tasks started to process.

Do upgrade tasks really block all other tasks from running?
Do you know what is causing the tentacle to fail the upgrade? We are now trying to run the MSI manually to do the upgrade.

ServerTasks-68469.log.txt (3 KB)

2

Hi Stuart,

Thanks for getting in touch! Yes upgrades block tasks. Upgrades require a Tentacle restart and there were occasions that deployments were being run while upgrades were also being run or vice versa that would have very detrimental effects on the deployments.
However the current solution is a bit clumsy and we will be updating it to put a mutex around the machine and not around the tasks. So any machines not included in the upgrade will be available for other tasks.
You can track the issue here: https://github.com/OctopusDeploy/Issues/issues/3535

As for your error I haven’t seen that specifically, how did the manual MSI upgrade go?

Regards,
Vanessa