Communication to Octopus Tentacle failing

We have a previously functioning Octopus Server set up with a number of environments that we were successfully deploying too.

However recently two tentacles in the same environment have stopped behaving. They respond to the connectivity check (if I restart a tentacle, it shows as disconnected on the server then connected) but if we try a deployment or a healthcheck, it just hangs without response.

Logs on each of the tentacles don’t know anything of interest however the log on the server shows:
2018-05-23 12:54:35.3846 68 WARN It looks as though conversations with XXXX-FAILING-TENTACLE are leaking
Google hasn’t aided me in locating the reason for this message. It only shows that message for the two failing tentacles and not for the other 8 successful tentacles.

Help appreciated!

Hi,

Could you please let me know what version of Octopus Server (and Tentacles) you are running?

Could you also send through both Octopus Server and Tentacle logs, this should help us figure out what is causing this issue.

Thank you and kind regards,
Henrik

We are using Octopus Server: 2.6.5.1010.
GoodOctopusTentacle.txt (31.0 KB)
BadOctopusTentacle.txt (8.8 KB)
OctopusServer.txt (3.1 KB)

I’ve attached logs from server, a tentacle which is working nicely (GoodOctopusTentacle.txt) and one that is not working (BadOctopusTentacle.txt) as there are splats of text in both the Tentacle logs which I discarded seeing as they appear on the good and the bad tentacle.

Hi,

I have not been able to figure out what is causing this issue yet.

While I keep investigating the issue, could you try the Reinstall option for your Tentacle service in the Octopus Tentacle Manager application and see if that fixes the issue?

Thank you and kind regards,
Henrik

Heya,

I’ve already tried that as part of my earlier testing steps.

Do you know what the “It looks as though conversations with XX are leaking” means? As I’m suspicious because that message only happens on the failing tentacle that it might at least guide us towards the problem, even if it’s just a symptom of something generally benign.

Hey,

The It looks as though conversations with ... are leaking message is printed when there are > 100 messages on the go at a time between the Octopus server and Tentacle. Which makes sense as the Tentacle receives the message, but fails to process it (the errors in the bad Tentacle log) and never returns the response.

I’ll keep investigating if I can figure out what else we can try and do to get to the root cause of this issue.

Thanks,
Henrik

Hi,

I had a thought, might be nothing, but could you check if TLS1.0 has been turned off on the servers that are failing?

Thank you and kind regards,
Henrik

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.