Tentacle connection issues

We are also trying out these settings. Will have to run a couple of weeks to see if it helps.

/Jan

Hi,

any updates on your experiences with these settings?

We have encountered some timeout issues with a server we have a poor connection to and will use these settings to see what difference they make.

Hi,

These settings seem to have helped a lot with these issues during Acquire packages, but we now see this issue during Apply retention policy on Tentacles.

An error occurred when sending a request to ‘https://xxxxxxx:10933/’, before the request could begin: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

/Jan

For us this actually happened while running a deployment script, the packages had already been uploaded.

If the settings don’t apply to the connection post-upload is there something else that could be tried?

It does look a lot better now, although we still see some issues from time to time.

-Thomas

I’m glad to hear the configuration changes have helped.

Vasily: Those settings do apply to the Tentacle connections post package-upload. If you are still getting errors, you can follow the instructions in this comment to upload the task log of a failure. Let me know if you do, and I’ll take a look.

Can you please make this the default settings in Octopus.Server.exe.config or fix this in some other way so that we don’t have to manually update the Octopus.Server.exe.config file on every upgrade of Octopus

/Jan

Jan,

I apologize, I didn’t realize those settings were wiped with each upgrade.

We’ll certainly make it so they are persisted.
We are also currently evaluating if there are sane changes we can make to the default settings. I’ll keep you posted on both.

We are at 3.2.9 as of this Monday morning, and up until yesterday hadn’t seen any issues. However yesterday afternoon during a CD we got a message part of the way through the deployment we saw this error on step 24 of a project:
“An error occurred when sending a request to …”

In the Octo logs we saw this:
“Unable to read data from the transport connection: A connection attempt failed because the connected party…”

After this happened, I tried the release again and it failed on step 1. And from there every health check for every tentacle started to fail (the errors were the same as above). I tried restarting a tentacle service. I tried restarting the server service. Neither fixed anything. Based off of a thread I found in the support forum, I tried a reinstall/repair of the server which forced me to do a server restart. Since this point everything has been working again.

Just had it happen again. I’m going to try to reinstall again (of course that requires a reboot which is happening right now). This time it was only one tentacle. I could connect to the tentacle and the machine from the Octopus Manager server if I visited via a browser.

In my case (unlike the above it appears), the environment/tentacle is offline. No deployments work to the environment.

The reinstall function through the Manager didn’t work.
Adding the settings above did not seem to do anything. I restarted the service after adding it, although I should note my overall install is not on C:\ and I’m not running as the local service, but as a domain service user.

Now that the box is back up, I can confirm that the connection did work (this time) and at least on 3.2.9, the repair function did not wipe out the appSettings mentioned above.

I’ll see what I can figure out for a tasklog.

As part of the above, I did upload a tasklog. We haven’t seen the problem again, but I can tell you having taken a vacation last week, I was about as nervous as I’ve been because I worried that someone else on my team was going to get stuck dealing with this.

Is it believed this is fixed in a particular release? Is there some other information that is being waited on (or worked)? Just would like to feel like there is progress being made.

Mike,

As of release 3.2.14, we updated some of the default connection timeouts to make them more tolerant. You can see specific values changed here.

Feedback is welcomed (from anyone on this thread) as to whether you have noticed any improvements lately.

On the flip-side, if you are still experiencing issues, can you upload your logs to here and post in this thread to let me know.

Hi!

We are still seeing issues with this quite often. We recently updated to 3.3.14, and this seemed to make the problem worse, as it used to work putting in a workaround in the area in the Octopus.Server.exe.config file:

Seeing that you have changed around a bit in the code, should I apply this workaround with some other levels? Are the limits now interpreted as hours instead of minutes?

Just happened again:

Fatal
An error occurred when sending a request to ‘https:///’, after the request began: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

Another potential cause for this is simply that the tentacle doesn’t trust the Octopus server - I just spent 10 minutes scratching my head over this before I realised I was using a machine image designed to work with a different octopus server.

Worth checking in the Octopus logs, which are pretty good at recording connection attempts and how they respond to them and why.

Thanks Nikki!

Our problem is unfortunately more intermittent than that, as we have agents some times working and some times dropping out, so it seems to be more related to the connection between the server and the agents being rickety.

Hi Thomas,

Can you send through the relevant deployment logs and Tentacle logs please so we can see what might cause the issue for you.

Thanks!
Vanessa

Vanessa,

I realize I’m intercepting, however we just had this happen again this morning (and yesterday morning), so I should still have the logs, but can you remind me which logs you are looking for and then where I should reply to (or send to) so I don’t plaster them on the internet?

-Mike

Mike

OK, I grabbed server/tentacle/deployment log.

Mike

@Mike,

Thanks for the logs - yes the deployment log then the Tentacle log (c:\Octopus\Logs) from the relevant Tentacle are what I needed. I downloaded the file then deleted it from here. You are always welcome to email things like logs to support @ octopus.com when its part of a public or shared thread. I will look into it now.

Thanks again
Vanessa