Error from Octopus Server (HTTP 503 ServiceUnavailable)

bhagyashreesarode21 · 6 May 2022 13:50

hi team,

Currently, I am trying to deploy my package in octopus but i get this error in the middle of execution.

Error Log:
rror: The process ‘C:\hostedtoolcache\windows\octo\7.4.3663\x64\octo.cmd’ failed with exit code 4294967289

at ExecState._setResult (D:\a\_tasks\OctopusDeployRelease_8ca1d96a-151d-44b7-bc4f-9251e2ea6971\3.1.131\node_modules\azure-pipelines-task-lib\toolrunner.js:942:25)

at ExecState.CheckComplete (D:\a\_tasks\OctopusDeployRelease_8ca1d96a-151d-44b7-bc4f-9251e2ea6971\3.1.131\node_modules\azure-pipelines-task-lib\toolrunner.js:925:18)

at ChildProcess.<anonymous> (D:\a\_tasks\OctopusDeployRelease_8ca1d96a-151d-44b7-bc4f-9251e2ea6971\3.1.131\node_modules\azure-pipelines-task-lib\toolrunner.js:838:19)

at ChildProcess.emit (events.js:198:13)

at maybeClose (internal/child_process.js:982:16)

at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)

dan_close · 6 May 2022 14:43

Hi @bhagyashreesarode21,

Thanks for reaching out and welcome to the Octopus community! When you receive this error in your deployment task are you still able to login to your Octopus instance? I just want to verify the Octopus service isn’t actually down.

Based on the output you gave us, it looks like this could be either a network disruption with your Octopus instance or perhaps a timeout with one of your deployments. I mention the task timeout because we have a 5-minute gap in the log which is a typical timeout used for deployment targets.

To rule out a network issue would you be able to kick off the same deployments within Octopus to see if you run into a similar issue?

For the task timeout, do you have a deployment task in Octopus that corresponds with this error? If so could you send us the raw output from the task? I can DM you a secure upload link rather than attach the task log here.

I look forward to hearing back and please let me know if you have any questions.

Thanks!
Dan

bhagyashreesarode21 · 9 May 2022 08:24

hi @dan_close ,

To add more points here, I got this error in the Azure DevOps site,
once I got this error no new releases were created in Octopus and the old package was getting deployed.
To mitigate this issue, I manually created a new release and deployed the latest package for deployment.

Is it possible to add a retry mechanism here?

Kindly share me secure link, so that i can share raw output to you!!

thanks for the quick reply!!!

clare.martin · 9 May 2022 08:48

Hi @bhagyashreesarode21,

Just jumping in here for Dan whilst he is currently off shift as part of our US Based Team.

I believe one of your other colleagues has asked a similar question here?

One of our Support staff has replied to that with a detailed answer which does include some information about ‘retry’ logic so are you able to give that a read and see if it fits your scenario?

If not we would be happy to share a secure link here for you to send us some raw outputs. If you do need to share those with us can you let us know when you have uploaded them please as we do not get notifications when customers upload files to our secure site.

Kind Regards,

Clare Martin

ritvij05 · 9 May 2022 12:08

Hi @clare.martin,
Replying on behalf of @bhagyashreesarode21.

Before sharing RAW output, We want to know which raw output your expecting. failed tenant one or success tenant raw output will do? Because Tenants are not getting failed. The error is getting reflected on ADO and not on octopus.

Few Tenants are getting deployed and and rest are not deployed since we have not checked the condition in ADO i.e. continue on failure.

API Calls which are getting successful for those tenants are getting deployed but API call sent to octopus which is not getting completed does not deploy the tenant. Please refer below image.

Today Again Our Pipeline got failed with Connection Issue. But this time the error message was different. This pipeline is scheduled from more than a month now but it has started failing from 5th MAY

Error message:
2022-05-09T10:39:27.4261485Z System.Net.Http.HttpRequestException: Error while copying content to a stream.

2022-05-09T10:39:27.4262184Z —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host…

2022-05-09T10:39:27.4262922Z —> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.

2022-05-09T10:39:27.4263480Z — End of inner exception stack trace —

2022-05-09T10:39:27.4264068Z at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)

2022-05-09T10:39:27.4264739Z at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)

2022-05-09T10:39:27.4265543Z at System.Net.Security.SslStream.g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask`1 task, Int32 min, Int32 initial)

2022-05-09T10:39:27.4268100Z at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer)

2022-05-09T10:39:27.4268732Z at System.Net.Http.HttpConnection.FillAsync()

2022-05-09T10:39:27.4269354Z at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.CopyToAsyncCore(Stream destination, CancellationToken cancellationToken)

2022-05-09T10:39:27.4270493Z at System.Net.Http.HttpConnectionResponseContent.SerializeToStreamAsync(Stream stream, TransportContext context, CancellationToken cancellationToken)

2022-05-09T10:39:27.4272745Z at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)

2022-05-09T10:39:27.4273538Z — End of inner exception stack trace —

2022-05-09T10:39:27.4274172Z at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)

2022-05-09T10:39:27.4274868Z at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)

2022-05-09T10:39:27.4275678Z at Octopus.Client.OctopusAsyncClient.DispatchRequest[TResponseResource](OctopusRequest request, Boolean readResponse, CancellationToken cancellationToken)

2022-05-09T10:39:27.4276344Z at Octopus.Client.OctopusAsyncClient.Create[TResource](String path, TResource resource, Object pathParameters)

2022-05-09T10:39:27.4276913Z at Octopus.Client.Repositories.Async.BasicRepository`1.Create(TResource resource, Object pathParameters)

2022-05-09T10:39:27.4278258Z at Octopus.Cli.Commands.Deployment.DeploymentCommandBase.CreateDeploymentTask(ProjectResource project, ReleaseResource release, DeploymentPromotionTarget promotionTarget, ReferenceCollection specificMachineIds, ReferenceCollection excludedMachineIds, TenantResource tenant) in C:\buildAgent\workDir\289bf0fca31007af\source\Octopus.Cli\Commands\Deployment\DeploymentCommandBase.cs:line 444

2022-05-09T10:39:27.4279696Z at Octopus.Cli.Commands.Deployment.DeploymentCommandBase.DeployTenantedRelease(ProjectResource project, ReleaseResource release) in C:\buildAgent\workDir\289bf0fca31007af\source\Octopus.Cli\Commands\Deployment\DeploymentCommandBase.cs:line 182

2022-05-09T10:39:27.4280866Z at Octopus.Cli.Commands.Deployment.DeploymentCommandBase.DeployRelease(ProjectResource project, ReleaseResource release) in C:\buildAgent\workDir\289bf0fca31007af\source\Octopus.Cli\Commands\Deployment\DeploymentCommandBase.cs:line 233

2022-05-09T10:39:27.4281934Z at Octopus.Cli.Commands.Deployment.DeployReleaseCommand.Request() in C:\buildAgent\workDir\289bf0fca31007af\source\Octopus.Cli\Commands\Deployment\DeployReleaseCommand.cs:line 66

2022-05-09T10:39:27.4282897Z at Octopus.Cli.Commands.ApiCommand.Execute() in C:\buildAgent\workDir\289bf0fca31007af\source\Octopus.Cli\Commands\ApiCommand.cs:line 250

2022-05-09T10:39:27.4283729Z at Octopus.Cli.Commands.ApiCommand.Execute(String[] commandLineArguments) in C:\buildAgent\workDir\289bf0fca31007af\source\Octopus.Cli\Commands\ApiCommand.cs:line 236

2022-05-09T10:39:27.4284542Z at Octopus.Cli.CliProgram.Run(String[] args) in C:\buildAgent\workDir\289bf0fca31007af\source\Octopus.Cli\CliProgram.cs:line 52

2022-05-09T10:39:27.4285081Z Exit code: -3

2022-05-09T10:39:27.4602578Z ##[error]"Failed to deploy release. The process ‘C:\hostedtoolcache\windows\octo\7.4.3663\x64\octo.cmd’ failed with exit code 4294967293

Error: The process ‘C:\hostedtoolcache\windows\octo\7.4.3663\x64\octo.cmd’ failed with exit code 4294967293

at ExecState._setResult (D:\a\_tasks\OctopusDeployRelease_8ca1d96a-151d-44b7-bc4f-9251e2ea6971\3.1.131\node_modules\azure-pipelines-task-lib\toolrunner.js:942:25)

at ExecState.CheckComplete (D:\a\_tasks\OctopusDeployRelease_8ca1d96a-151d-44b7-bc4f-9251e2ea6971\3.1.131\node_modules\azure-pipelines-task-lib\toolrunner.js:925:18)

at ChildProcess.<anonymous> (D:\a\_tasks\OctopusDeployRelease_8ca1d96a-151d-44b7-bc4f-9251e2ea6971\3.1.131\node_modules\azure-pipelines-task-lib\toolrunner.js:838:19)

at ChildProcess.emit (events.js:198:13)

at maybeClose (internal/child_process.js:982:16)

at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)

Regards,
Ritvij Iyer

system · 9 June 2022 12:09

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.