Hello there. We have a step on our deployment process which executes a Bash script. When pressing the “Cancel” button the process looks like it is stopping but in fact the worker continues the execution of the Bash script. What is the best way to force the worker to stop the execution of the Bash script please (like pressing Ctrl+C on Linux)?
Hey @georgios.goniotakis,
Thanks for reaching out.
When you click the Cancel button, the Octopus Server does reach out to the Tentacle/Worker and does attempt to stop the current task, but unfortunately depending on the script/task, the cancel sometimes isnt respected and the task continues to completion.
As a potential workaround, you could use the script console to target the specific machine in question and get the process id then kill it. You could also have a runbook with a prompted variable to do the same. Is this something that happens often, or just a rare instance that you want to have an option for next time?
Please let me know what you think.
Best,
Jeremy
Sorry to hijack, but I have a similar issue and a follow up question regarding script console.
With the scenario, there is already a process running in the target and executing a kill command via the script console will get you “waiting for script to finish in task…” so it is currently not the workaround for me.
Any way we can make the script console ignore the mutex i.e OctopusBypassDeploymentMutex = TRUE on the fly on the script console?
Hello @jeremy.miller
Thank you for your prompt response. I can confirm that this is not something that is happening all the time. However, we would still need to be able to have full control over the process. Could you kindly elaborate on the runbook solutions please?
Kind regards,
Georgios
Hi @earland,
Thanks for the suggestion, that’s a very good point. Can you please create an entry for it on uservoice? https://octopusdeploy.uservoice.com/
I will also pass it along to our product team but it’s good for it to be on uservoice.
Hey @georgios.goniotakis,
My apologies for the Script Console suggestion as that will only work if a worker is doing it, not an SSH target/tentacle, due to the lack of mutex bypass.
To start, the account that runs the ssh connection will need password-less sudo as our scripts are non-interactive and you won’t be able to provide the password.
Second, the project in question will need the following variables added to the project variables:
OctopusBypassDeploymentMutex with a value of True (This will allow the machine to be targeted even though a script is running on it. )
PromptedPIDtoKill with a blank value, but a prompted variable (unless you want to change the variable each time you run the Kill runbook)
In my quick mockup of the solution, what I did was created a runbook that runs ps -aux and shows you the results, then a second runbook that has a prompted variable with the script being sudo kill -9 #{PromptedPIDtoKill}. You will run the first one, find the process that is hanging up the deployment, then run the second one and enter the PID in the prompted variable field. This will kill the process and fail the deployment so that you can remediate the issue that was causing it to be stuck.
Please let me know if you have any questions about the above and what you think.
Best,
Jeremy
Hey @georgios.goniotakis,
I actually came up with a bit of a more elegant solution here. You can do this in one runbook programmatically with this code.
ps axf | grep Calamari | grep -v grep | head -1 | awk '{print "killing pid " $1}'
ps axf | grep Calamari | grep -v grep | head -1 | awk '{print "kill -9 " $1}' | sh
Please let me know how it goes or if you have any questions.
Best,
Jeremy
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.