We have 27 deployment targets on our QA, Staging and Live environments. Most work without a problem, but one or two fail during Acquire Packages.
each server gets multiple packages, and this doesn’t necessarily occur on the first package, but often later, after several have uploaded successfully.
On an identical machine, the download of packages is successful.
the error is this:
Uploading package Code.Source.Services.ShowtimeService (227.529 MB)…
June 11th 2018 11:07:22Info
Beginning streaming transfer of Code.Source.Services.ShowtimeService@S0.992.0.25832-ChannelQA2@FED7529606BEED45968D8F29A6EAC460.nupkg
June 11th 2018 11:07:43Info
An error occurred when sending a request to ‘https://iisieawssrc01qa:10933/’, after the request began: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.
Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.
An existing connection was forcibly closed by the remote host An error occurred when sending a request to ‘https://iisieawssrc01qa:10933/’, after the request began: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.
Halibut.HalibutClientException: An error occurred when sending a request to ‘https://iisieawssrc01qa:10933/’, after the request began: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. —> System.IO.IOException: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.EndSend(IAsyncResult asyncResult)
at System.Net.Sockets.NetworkStream.EndWrite(IAsyncResult asyncResult)
— End of inner exception stack trace —
at System.Net.Security._SslStream.EndWrite(IAsyncResult asyncResult)
at Halibut.DataStream.StreamingDataStream.CopyAndReportProgress(Stream destination)
at Halibut.Transport.Protocol.MessageExchangeStream.WriteEachStream(IEnumerable1 streams) at Halibut.Transport.Protocol.MessageExchangeStream.Send[T](T message) at Halibut.Transport.Protocol.MessageExchangeProtocol.ExchangeAsClient(RequestMessage request) at Halibut.HalibutRuntime.<>c__DisplayClass28_0.<SendOutgoingHttpsRequest>b__0(MessageExchangeProtocol protocol) at Halibut.Transport.SecureClient.ExecuteTransaction(Action1 protocolHandler)
— End of inner exception stack trace —
Server stack trace:
at Halibut.Transport.SecureClient.HandleError(Exception lastError, Boolean retryAllowed)
at Halibut.HalibutRuntime.SendOutgoingHttpsRequest(RequestMessage request)
at Halibut.ServiceModel.HalibutProxy.Invoke(IMessage msg)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Octopus.Shared.Contracts.IFileTransferService.UploadFile(String remotePath, DataStream upload)
at Octopus.Server.Orchestration.Targets.Tentacles.TentacleRemoteEndpointFacade.UploadFile(String fileName, DataStream package)
Octopus.Server version 2018.5.7 (2018.5.7+Branch.master.Sha.d3aea6debd7681084721e692589d665bcbf0c24f)
June 11th 2018 11:07:45Info
File upload failed. Retry attempt 1 of 5…
We tried to increase the timeout:
Halibut.PollingRequestMaximumMessageProcessingTimeout
Halibut.PollingRequestQueueTimeout
, but the result is the same
Could you help in solving this problem?
Full deployment log.txt (101.2 KB)
OctopusServer.txt (258.6 KB)
OctopusTentacle.txt (70.0 KB)