Octopus process hanging, even if step is complete

When deploying from Octopus using some of the built in Octopus steps, our deployment processes are regularly freezing.

Two Examples

Example One

In this case, it seems that the Octopus deployment is not triggering deployment of the ARM template when using the built in Deploy an Azure Resource Manager template step. However, this is not consistent behaviour, and I can not reliably reproduce it.

As you can see in the screenshot Octopus deployment hung, the Deploy Resource Group step has been running for nearly 12 minutes. You can then see from the ARM deployment not triggered screenshot that no ARM template deployment has not been triggered. Notice however that it was triggered a few minutes before, the last time that I ran the deployment.

ARM Step Hung

ARM Deployment Not Triggered

Example Two

This time the deployment has hung on the Run an Azure PowerShell Script step. Again, I am unable to reliably reproduce this behaviour.

As you can see in the screenshot PowerShell step hung, the Add API to Products step has been running for more than 21 hours, yet the scrip that it runs has output a success message (the last line of the script).

PowerShell Step Hung

Is this behaviour part of a know bug? If not, is anyone able to suggest what may be the root cause and how I’d go about fixing the issue?

I’m happy to supply raw logs if they will help.

Hi David,

Sorry to hear you’re having trouble with the ARM templates. It isn’t a known bug that I’m aware of.

From what I can see in the screenshot there, you have an OnPrem Octopus instance and I’m guessing you’re deploying into an Azure data centre nearby?

The deployment logs may be useful. Would you be able to email them to support@octopus.com, with attn: Shannon?

Regards
Shannon

Hi Shannon,

Thanks for the reply.

We are indeed self-hosted (Azure North Europe), and we typically deploy to North/West Europe and UK South/West, so all regions are pretty close.

It seems that it’s not just ARM templates that are affected - the second example above is around a PowerShell script hanging, even though the script seemingly completes.

I’ll get the logs to you via email momentarily.

Thanks,
David