Octopus seems to be working but tentacles are logging errors

Everything seems to be working but just noticed that all the tentacles are logging errors. Health checks are fine. Here is a snippet from the tentacle log file.

2015-09-08 00:38:44.9542 38 INFO listen://0.0.0.0:10933/ 38 Unhandled error when handling request from client: xx.xx.xx.xx:56516
System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. —> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
— End of inner exception stack trace —
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadLine()
at Halibut.Transport.Protocol.MessageExchangeStream.ReadLine() in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 91
at Halibut.Transport.Protocol.MessageExchangeStream.ExpectNextOrEnd() in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 65
at Halibut.Transport.Protocol.MessageExchangeProtocol.ProcessClientRequests(Func2 incomingRequestProcessor) in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 110 at Halibut.Transport.Protocol.MessageExchangeProtocol.ExchangeAsServer(Func2 incomingRequestProcessor, Func`2 pendingRequests) in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 90
at Halibut.Transport.SecureListener.ExecuteRequest(TcpClient client) in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\SecureListener.cs:line 122
2015-09-08 01:28:43.9503 62 INFO listen://0.0.0.0:10933/ 62 Accepted TCP client: xx.xx.xx.xx:49600

I was on 3.0.6, I just updated to 3.0.21. Will update posting a little while.

Hi Doug,

They look to me like transient connection errors. Were these happening on 3.0.6 or only since 3.0.21? Are they happening frequently?

Cheers,
Shane

Shane, I upgraded to 3.0.21 last night. The errors are still happening and they are logged every hour. The tentacles are setup to be listening.

2015-09-09 02:07:49.2733 18 INFO listen://0.0.0.0:10933/ 18 Accepted TCP client: xx.xx.xx.xx:53040
2015-09-09 02:07:49.2889 20 INFO listen://0.0.0.0:10933/ 20 Performing SSL (TLS 1.0) server handshake
2015-09-09 02:07:49.2889 20 INFO listen://0.0.0.0:10933/ 20 Secure connection established, client is not yet authenticated
2015-09-09 02:07:49.5073 20 INFO listen://0.0.0.0:10933/ 20 Client at xx.xx.xx.xx:53040 authenticated as A757339212ADBA8771A8FDBF7AE63EB000000000
2015-09-09 02:17:52.7125 20 INFO listen://0.0.0.0:10933/ 20 Unhandled error when handling request from client: xx.xx.xx.xx:53040
System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. —> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
— End of inner exception stack trace —
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadLine()
at Halibut.Transport.Protocol.MessageExchangeStream.ReadLine() in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 91
at Halibut.Transport.Protocol.MessageExchangeStream.ExpectNextOrEnd() in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeStream.cs:line 65
at Halibut.Transport.Protocol.MessageExchangeProtocol.ProcessClientRequests(Func2 incomingRequestProcessor) in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 110 at Halibut.Transport.Protocol.MessageExchangeProtocol.ExchangeAsServer(Func2 incomingRequestProcessor, Func`2 pendingRequests) in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\Protocol\MessageExchangeProtocol.cs:line 90
at Halibut.Transport.SecureListener.ExecuteRequest(TcpClient client) in y:\work\7ab39c94136bc5c6\source\Halibut\Transport\SecureListener.cs:line 122

Hi Doug,

I am getting the same exceptions in my Tentacle log files. I think they are just transient faults, the underlying connection is timing out or being dropped and the server is reconnecting. If everything else looks fine I wouldn’t worry about them. They do create a lot of noise in the log files, I’ll see if we can handle them better.

Cheers,
Shane