So I have a deployment process which is launching an AMI and installing various applications, in this particular example the process steps before succeeded on the target and then one fails with -
Server exception:
System.IO.IOException: The write operation failed, see inner exception.
—> System.ObjectDisposedException: Cannot access a disposed object.
Object name: ‘System.Net.Sockets.NetworkStream’.
at System.Net.Sockets.NetworkStream.g__ThrowObjectDisposedException|63_0()
at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security.SslStream.WriteSingleChunk[TIOAdapter](TIOAdapter writeAdapter, ReadOnlyMemory1 buffer) at System.Net.Security.SslStream.WriteAsyncInternal[TIOAdapter](TIOAdapter writeAdapter, ReadOnlyMemory1 buffer)
— End of inner exception stack trace —
at System.Net.Security.SslStream.WriteAsyncInternal[TIOAdapter](TIOAdapter writeAdapter, ReadOnlyMemory`1 buffer)
at System.Net.Security.SslStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Compression.DeflateStream.Dispose(Boolean disposing)
at Halibut.Transport.Protocol.MessageSerializer.WriteMessage[T](Stream stream, T message)
at Halibut.Transport.Protocol.MessageExchangeStream.Send[T](T message)
at Halibut.Transport.Protocol.MessageExchangeProtocol.ProcessReceiverInternalAsync(IPendingRequestQueue pendingRequests, RequestMessage nextRequest)
Thanks for reaching out to Octopus Support, and I’m sorry you’re experiencing this issue with your deployments.
When you have a moment, could you send us a copy of the raw task log from one of these failed deployments? It should give us a picture of your deployment process and where the breakdown may occur.
Do you see this failure on every deployment, or does it sometimes succeed? A task log from a successful deployment would be helpful if there is one. I’ve created a secure upload link you can use to send us any files.
I look forward to hearing back, and please let me know if you have any questions.
Typically, when I’ve hit this issue, if I re-deploy rather than re-trying the failed step, the re-deployment succeeded, but because this project is depolyed in a chain, that is not a workaround
Thanks for sending that over. Could you also send us the tentacle log from that deployment? I’m curious if that indicates why it cannot connect to the Octopus server.
Are you using guided failure or the new retry feature that was recently added for the step retry you attempted?
I look forward to hearing back, and please let me know if you have any questions.
It seems there is a network communication breakdown happening when your deployment is failing. In your Tentacle log, we see System.Exception: messageEnvelope is null at the time of the deployment failure when the Tentacle attempts to communicate back to your Octopus server.
Do any previous steps in your deployment reboot the EC2 instance before you get this failure? A reboot happening mid-deployment could cause similar issues.
You could try implementing the new step retry feature I mentioned previously, as this was introduced to help alleviate deployment failures from transient networking issues. You would need to enable the setting in any steps necessary and create a new release for it to take effect.
