AMSI Stalling Tentacle-spawned Powershell Scripts

reliability
#1

Hello,

Our deployments periodically stall on a random server (not the same one ever), with the Powershell script spawned for the step running on that server never finishing. We did a memory dump of one of the Powershell scripts and found that it was erroring with an “Unable to load amsi.dll: the specified module could not be found. (Exception from HRESULT: 0x8007007E)” error. Stopping and restarting the Tentacle and re-running the step allows it to continue, so the DLL isn’t actually missing. This isn’t ideal of course. Has anyone at Octopus Deploy seen this issue before?

We’re on version 2019.3.3 LTS. The Tentacles are running on Windows 2012 R2, and the Powershell is 5.1.14409.1018 (it’s mixed, however, but always 5.1 as our deployment process requires it).

(Michael Noonan) #3

Hi,

Thanks for getting in touch! Intermittently stalling deployments are really annoying. I’m glad you took a memory dump of the PowerShell.exe process running your stalled script. This is the best way to see why the deployment has stalled.

amsi.dll is related to anti-malware detection on Windows. The most common cause of intermittently stalling deployments is virus protection. Depending on your scenario, a good next step would be to disable the virus protection completely and test, or exclude your Tentacle Home folder and any other folders you deploy to from virus protection, then test.

I agree, it is strange that the error is appearing even though the DLL is available. I don’t have any better advice right now apart from what I’ve already offered, and try to narrow down whether our old nemesis is truly at fault, or if it’s something else entirely.

Hope that helps!
Mike