Service Delayed Start Fails Release


(kira.resari) #1

Hello everyone,

this is a problem that randomly keeps appearing in our Octopus environment, and has been appearing for several months so, pretty much since we set up the Project.

The Project deploys the latest build of our software to a testing server every night and starts the relevant services. This works pretty much all of the time.

However, every few weeks or so the release fails with the same error message:

Installing Company Foo Bar Service... 
Stopped  Company Foo ... Company Foo Bar Service             
The Company Foo Bar Service service is starting........ 
The Company Foo Bar Service service could not be started. 
The service did not report an error. 
More help is available by typing NET HELPMSG 3534. 
Finished installing Company Foo Bar Service! 
The remote script failed with exit code 2 
The action Install and Start Services on Company Test Server failed

This happens in a PowerShell Step, where the following code is executed (excerpt, the full script starts several other services as well, none of which caused this error thus far):

Function ServiceExists([string] $ServiceName) {

[bool] $Return = $False

if ( Get-Service "$ServiceName" -Include $ServiceName ) {

$Return = $True

}

Return $Return

}

[...]

# Installs and starts the Windows Service "Company Foo Bar Service"


if(ServiceExists("Company Foo Bar Service")){

echo "[NOTICE] The service 'Company Foo Bar Service' does appear to be already installed. I'll skip this one."

}else{

New-Service -Name "Company Foo Bar Service" -BinaryPathName "C:\Company\CPFooBarServiceHost.exe -k netsvcs" -StartupType Automatic

net start "Company Foo Bar Service"

}

echo "Finished installing Company Foo Bar Service!"

Now, the interesting part here is that even if the octopus Deploy Step fails with above error messing, saying that the service could not be started, up until now whenever I checked the Test Server, the Company Foo Bar Service was running after all, so at first I thought it might be a problem with the Service taking too long to start up.

So I created the Registry Entry HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ServicesPipeTimeout on the Test Server and set it to 300,000 (5 minutes), and verified that it doesn’t get overwritten or deleted or anything, but this same error still occurs all the same.

So I was wondering, is there any setting similar to ServicesPipeTimeout in Octopus Deplyo that might cause this “false negative” Error? I mean, the Service ends up running just fine in the end. I am just trying to understand why we are getting an error message despite that, and if there’s any way we can get around that.

Thanks in advance,
Kira Resari


(Henrik Andersson) #2

Hi Kira,

There is nothing in Octopus that I can think of that would interfere with that timeout.

Instead of using net start you could try and use the code below to wait for the service to start:

Start-Service -Name "Company Foo Bar Service"

$service = Get-Service -Name "Company Foo Bar Service"
$service.WaitForStatus([System.ServiceProcess.ServiceControllerStatus]::Running, "00:05:00")

I hope that helps.

Thank you and best regards,
Henrik


(kira.resari) #3

Hi Henrik,

thank you very much for your reply. It’s good to know that I can strike Octopus from the list of possible causes, and I will definitely try your suggestiong.

Thank you very much,
Kira
=^,^=


(kira.resari) #4

Hi Henrik,

today I implemented your suggestion after the error occurred again. I tested it and it appears to work nicely, but since the error only manifests itself randomly, I can’t say for sure whether this fixed the problem.

However, I’ve noticed that the error always manifests during the deployment on Monday nights, which I assume is because the server is also busy doing other things at that day of the week, so I am looking forward to next week to see if this resolved the issue.

Thanks again,
Kira
=^,^=


(Henrik Andersson) #5

Hi Kira,

Thanks for the update, fingers crossed it keeps working for you. If not, let us know and we’ll keep investigating.

Thanks,
Henrik