Polling Tentacles Don't Respond

Legacy Uncategorised
1

I have some polling tentacles running on servers in a public cloud. Occasionally, a deploy starts and they seem to not pick up the job. For the most part, these are just powershell scripts to run.

I found this link that may be related and was wondering if there is a patch available. http://help.octopusdeploy.com/discussions/problems/19302-release-gets-stuck-when-uploading-packages-to-polling-tentacle

From the tentacle server, I can open a browser and see the mx page on the server, so firewalls shouldn’t be an issue. This diagnostics tab show nothing until I try to cancel the job, then I get the following entries:

Unhandled error when processing request from client
System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at Pipefish.Transport.SecureTcp.Server.SecureTcpServer.ExecuteRequest(TcpClient client) in y:\work\3cbe05672d69a231\source\Pipefish.Transport.SecureTcp\Server\SecureTcpServer.cs:line 111
Error 2014-08-12 16:40:24
Unhandled error when processing request from client
System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at Pipefish.Transport.SecureTcp.Server.SecureTcpServer.ExecuteRequest(TcpClient client) in y:\work\3cbe05672d69a231\source\Pipefish.Transport.SecureTcp\Server\SecureTcpServer.cs:line 111
Error 2014-08-12 16:40:27
Invalid request
System.Net.ProtocolViolationException: Request syntax was invalid
at Pipefish.Transport.SecureTcp.ProtocolParser.ReadPrelude(Stream clientStream) in y:\work\3cbe05672d69a231\source\Pipefish.Transport.SecureTcp\ProtocolParser.cs:line 111
at Pipefish.Transport.SecureTcp.ProtocolParser.ParseRequest(Stream clientStream, Method& method, Uri& uri, RequestHeaders& headers, String& protocol) in y:\work\3cbe05672d69a231\source\Pipefish.Transport.SecureTcp\ProtocolParser.cs:line 52
at Pipefish.Transport.SecureTcp.Server.SecureTcpServer.ApplyProtocol(AuthorizationResult authorizationResult, EndPoint clientEndPoint, String clientThumbprint, Stream clientStream) in y:\work\3cbe05672d69a231\source\Pipefish.Transport.SecureTcp\Server\SecureTcpServer.cs:line 140
Error 2014-08-12 16:40:31
Invalid request
System.Net.ProtocolViolationException: Request syntax was invalid
at Pipefish.Transport.SecureTcp.ProtocolParser.ReadPrelude(Stream clientStream) in y:\work\3cbe05672d69a231\source\Pipefish.Transport.SecureTcp\ProtocolParser.cs:line 111
at Pipefish.Transport.SecureTcp.ProtocolParser.ParseRequest(Stream clientStream, Method& method, Uri& uri, RequestHeaders& headers, String& protocol) in y:\work\3cbe05672d69a231\source\Pipefish.Transport.SecureTcp\ProtocolParser.cs:line 52
at Pipefish.Transport.SecureTcp.Server.SecureTcpServer.ApplyProtocol(AuthorizationResult authorizationResult, EndPoint clientEndPoint, String clientThumbprint, Stream clientStream) in y:\work\3cbe05672d69a231\source\Pipefish.Transport.SecureTcp\Server\SecureTcpServer.cs:line 140

I have no control of the cloud provider network and the transfer is over the internet. I have seen a lot of file corruption trying to deploy a package or using ftp through Octopus. I am able to compress the file with 7zip and use powershell to ftp the file with no issues so far though.

2

Hi Pete,

Thanks for getting in touch! Could you please explain to us a bit about the environment and the server.
Such as: are the tentacles behind or use a proxy server or load balancer? How do you gain access, is it via public IP address?
Where is it hosted and what does the network look like?

I know that you have answered a few of these questions, but giving us as many specifics into this situation will help.
Thanks!
Vanessa

3

The tentacles are not behind a load balancer or proxy that I am aware of. They are hosted in Rackspace, so I’m not sure what the physical network looks like. The logical layout is a cisco asa 5500 series firewall and then my servers. There are load balancers for the web sites, but they are on separate ips and only answer on 80 and 443. I can access all port of the server directly by public ip.

My server is version 2.5.5.318 and tentacles are a mix of 2.5.5.318 and 2.5.7.384.

4

Hi Pete,

We are really at a bit of a loss.
May I ask why you went the polling tentacle route? If they are available via public IP and port, listening tentacle would be a more ideal solution and may resolve this issue.

Vanessa