Intermittent Linux Connectivity

Legacy Execution
1

I’m having trouble with a pair of linux deployment targets that keep intermittently failing health checks. I don’t have much else to go on other than the following,

image
image.png898×925 55.8 KB

image
image.png1261×441 26.7 KB 

Apr  2 19:23:00 ubuntu-16042-server-amd64 sshd[419]: Accepted password for octopus from 10.31.184.155 port 62832 ssh2
Apr  2 19:23:00 ubuntu-16042-server-amd64 sshd[419]: pam_unix(sshd:session): session opened for user octopus by (uid=0)
Apr  2 19:23:00 ubuntu-16042-server-amd64 systemd: pam_unix(systemd-user:session): session opened for user octopus by (uid=0)
Apr  2 19:23:00 ubuntu-16042-server-amd64 systemd-logind[1122]: New session 623 of user octopus.
Apr  2 19:23:22 ubuntu-16042-server-amd64 sshd[458]: Received disconnect from 10.31.184.155 port 62832:11: Connection terminated by the client.
Apr  2 19:23:22 ubuntu-16042-server-amd64 sshd[458]: Disconnected from 10.31.184.155 port 62832
Apr  2 19:23:22 ubuntu-16042-server-amd64 sshd[419]: pam_unix(sshd:session): session closed for user octopus
Apr  2 19:23:22 ubuntu-16042-server-amd64 systemd-logind[1122]: Removed session 623.
2

Hi James,

It looks like a network problem, or a problem with that target. You could try:

  • Keeping a SSH connection open to the target and seeing if it get’s dropped
  • Setting up a continuous ping between the server and target to see if there are drop outs
  • Check whether the target is refusing the connection due to resource limits (also check how many sshd processes are running)
  • Create a new machine policy for these targets and set the health check interval to something really short (eg 1 minute) to see how long this problem lasts and if there is a pattern (eg gets worse at the end of an hour and suddenly corrects itself). I’ve found problems with network infrastructure getting overloaded (and resetting) this way.
3

Thank you for the response and suggestions!

I am working on trying each suggestion, but limited in what I can do with SSH on the server itself (without having to install ssh via windows update). Is there a program/script in the Octopus program file(s) that I can run to either emulate the health check, or do and ad hoc SSH connection?

I found this thread, https://help.octopusdeploy.com/discussions/problems/45621-ssh-connection-with-failed-check-deployment-target. I ran the shell script on my linux endpoints and everything checks out.

I also ran Ping/telnet (port 22) and have discovered issues there I’m investigating with our networking team.

4

Hi James,

The best way to test the ssh connection is to download PuTTY (https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html), there is a stand alone version. There isn’t a command line tool bundled with Octopus unfortunately.

Rob

5

Thanks, finding it’s our internal networking.

6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.