Octopus service cannot connect to tentacle

[neo]

Hi,

We have a problem with Octopus service connecting to the tentacle. The error is

Check deployment target: ADGCMDEVOPS01 Tentacle

An error occurred when sending a request to ‘https://10.10.50.85:10933/’, before the request could begin: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine. Unable to read data from the transport connection: An established connection was aborted by the software in your host machine. Unable to read data from the transport connection: An established connection was aborted by the software in your host machine. An established connection was aborted by the software in your host machine

Summary

One or more machines were not available. Please see the output Log for details.

I have tried everything to troubleshoot including steps in your website, IIS Crypto etc but nothing works.
The Octopus installation sits on the C drive but the application files on teh D drive.

Any ideas what may be going wrong?

Thanks!
Neo

[rafal.gapski]

Did you checked firewall settings at the client machine?

[jeremy.miller]

Hi @neo,

Like @rafal.gapski said, this seems like a port issue. Have you ensured that port 10933 is open on all firewalls from end to end?

[neo]

Hi Jeremy,

Thank you very much for your reply, much appreciated.

Yes I have spoken with the Admin team and we have even disabled the firewall on the box for testing but still the same result.

When I try the telnet command on the server, for example,

telnet server_name 10933

that works as well.

Thanks
Neo

[jeremy.miller]

Hi Neo,

Do you have any other software that could be blocking it like antivirus?

Would you be able to attach the tentacle logs from that timeframe? If they need to be private please feel free to direct message them to me.

Thanks,
Jeremy

[neo]

Hi Jeremy,

Yes we have Sophos running, but this is managed by the IT Admin team and I am not aware of what Sophos settings we have in place that could be causing this. Please find the logs below:

2020-06-25 00:30:36.2024 21108 482 INFO https://10.10.50.85:10933/ 482 Secure connection established. Server at [::ffff:10.10.50.85]:10933 identified by thumbprint: 3B215E0FB632AB2470F8F3CA759BF63C8A98E492, using protocol Tls12
2020-06-25 00:30:36.2024 21108 482 INFO https://10.10.50.85:10933/ 482 Secure connection established. Server at [::ffff:10.10.50.85]:10933 identified by thumbprint: 3B215E0FB632AB2470F8F3CA759BF63C8A98E492, using protocol Tls12
2020-06-25 00:30:36.2024 21108 482 INFO https://10.10.50.85:10933/ 482 Secure connection established. Server at [::ffff:10.10.50.85]:10933 identified by thumbprint: 3B215E0FB632AB2470F8F3CA759BF63C8A98E492, using protocol Tls12
2020-06-25 00:30:36.2164 21108 482 ERROR https://10.10.50.85:10933/ 482 Connection initialization failed while connecting to https://10.10.50.85:10933/
Halibut.Transport.Protocol.ConnectionInitializationFailedException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
— End of inner exception stack trace —
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadLine()
at Halibut.Transport.Protocol.MessageExchangeStream.ReadRemoteIdentity() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 168
at Halibut.Transport.Protocol.MessageExchangeStream.ExpectServerIdentity() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 231
at Halibut.Transport.Protocol.MessageExchangeProtocol.PrepareExchangeAsClient() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 56
— End of inner exception stack trace —
at Halibut.Transport.Protocol.MessageExchangeProtocol.PrepareExchangeAsClient() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 66
at Halibut.Transport.Protocol.MessageExchangeProtocol.ExchangeAsClient(RequestMessage request) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 35
at Halibut.HalibutRuntime.<>c__DisplayClass34_0.b__0(MessageExchangeProtocol protocol) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\HalibutRuntime.cs:line 157
at Halibut.Transport.SecureListeningClient.ExecuteTransaction(Action1 protocolHandler) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\SecureListeningClient.cs:line 65 2020-06-25 00:30:36.2164 21108 482 ERROR https://10.10.50.85:10933/ 482 Connection initialization failed while connecting to https://10.10.50.85:10933/ Halibut.Transport.Protocol.ConnectionInitializationFailedException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count) at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.Read(Byte[] buffer, Int32 offset, Int32 count) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadLine() at Halibut.Transport.Protocol.MessageExchangeStream.ReadRemoteIdentity() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 168 at Halibut.Transport.Protocol.MessageExchangeStream.ExpectServerIdentity() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 231 at Halibut.Transport.Protocol.MessageExchangeProtocol.PrepareExchangeAsClient() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 56 --- End of inner exception stack trace --- at Halibut.Transport.Protocol.MessageExchangeProtocol.PrepareExchangeAsClient() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 66 at Halibut.Transport.Protocol.MessageExchangeProtocol.ExchangeAsClient(RequestMessage request) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 35 at Halibut.HalibutRuntime.<>c__DisplayClass34_0.<SendOutgoingHttpsRequest>b__0(MessageExchangeProtocol protocol) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\HalibutRuntime.cs:line 157 at Halibut.Transport.SecureListeningClient.ExecuteTransaction(Action1 protocolHandler) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\SecureListeningClient.cs:line 65
2020-06-25 00:30:36.2164 21108 482 ERROR https://10.10.50.85:10933/ 482 Connection initialization failed while connecting to https://10.10.50.85:10933/
Halibut.Transport.Protocol.ConnectionInitializationFailedException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
— End of inner exception stack trace —
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadLine()
at Halibut.Transport.Protocol.MessageExchangeStream.ReadRemoteIdentity() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 168
at Halibut.Transport.Protocol.MessageExchangeStream.ExpectServerIdentity() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 231
at Halibut.Transport.Protocol.MessageExchangeProtocol.PrepareExchangeAsClient() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 56
— End of inner exception stack trace —
at Halibut.Transport.Protocol.MessageExchangeProtocol.PrepareExchangeAsClient() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 66
at Halibut.Transport.Protocol.MessageExchangeProtocol.ExchangeAsClient(RequestMessage request) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 35
at Halibut.HalibutRuntime.<>c__DisplayClass34_0.b__0(MessageExchangeProtocol protocol) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\HalibutRuntime.cs:line 157
at Halibut.Transport.SecureListeningClient.ExecuteTransaction(Action1 protocolHandler) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\SecureListeningClient.cs:line 65 2020-06-25 00:30:36.2164 21108 482 ERROR https://10.10.50.85:10933/ 482 Connection initialization failed while connecting to https://10.10.50.85:10933/ Halibut.Transport.Protocol.ConnectionInitializationFailedException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count) at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.Read(Byte[] buffer, Int32 offset, Int32 count) at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadLine() at Halibut.Transport.Protocol.MessageExchangeStream.ReadRemoteIdentity() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 168 at Halibut.Transport.Protocol.MessageExchangeStream.ExpectServerIdentity() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 231 at Halibut.Transport.Protocol.MessageExchangeProtocol.PrepareExchangeAsClient() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 56 --- End of inner exception stack trace --- at Halibut.Transport.Protocol.MessageExchangeProtocol.PrepareExchangeAsClient() in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 66 at Halibut.Transport.Protocol.MessageExchangeProtocol.ExchangeAsClient(RequestMessage request) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 35 at Halibut.HalibutRuntime.<>c__DisplayClass34_0.<SendOutgoingHttpsRequest>b__0(MessageExchangeProtocol protocol) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\HalibutRuntime.cs:line 157 at Halibut.Transport.SecureListeningClient.ExecuteTransaction(Action1 protocolHandler) in C:\buildAgent\work\fe2b45bbd4978f75\source\Halibut\Transport\SecureListeningClient.cs:line 65
2020-06-25 00:30:36.2396 21108 538 INFO https://adgcmvmprdsql01:10933/ 538 Secure connection established. Server at [::ffff:10.10.41.94]:10933 identified by thumbprint: 85BD42E8E99981FA7756C7C73FCB1FB8B0A75FA8, using protocol Tls12
2020-06-25 00:30:36.2396 21108 538 INFO https://adgcmvmprdsql01:10933/ 538 Secure connection established. Server at [::ffff:10.10.41.94]:10933 identified by thumbprint: 85BD42E8E99981FA7756C7C73FCB1FB8B0A75FA8, using protocol Tls12
2020-06-25 00:30:36.2396 21108 538 INFO https://adgcmvmprdsql01:10933/ 538 Secure connection established. Server at [::ffff:10.10.41.94]:10933 identified by thumbprint: 85BD42E8E99981FA7756C7C73FCB1FB8B0A75FA8, using protocol Tls12
2020-06-25 00:30:37.2207 21108 482 ERROR https://10.10.50.85:10933/ 482 Retry attempt 1
2020-06-25 00:30:37.2207 21108 482 ERROR https://10.10.50.85:10933/ 482 Retry attempt 1
2020-06-25 00:30:37.2207 21108 482 ERROR https://10.10.50.85:10933/ 482 Retry attempt 1

[jeremy.miller]

Hi Neo,

This definitely looks like it could be caused by Sophos. Would you be able to contact your IT admins and tell them what’s happening here and see if they can take a look at the settings and logs? Have them ensure that both the tentacle and server are white listed within Sophos.

Please let me know if you have any questions or concerns.

Thanks,
Jeremy