Unable to connect to sts.amazonaws.com:443

Hello,

Now we are able to connect to aws account , authentication worked , but our eks cluster health check is failed .PFA log and suggest us ASAP as production deployments are getting impacted
ServerTasks-953.log.txt (17.5 KB)

Hi @yogitha.kakarla,

Having looked at your logs this again unfortunately points to a network issue.

In your logs I can see this error when Octopus is trying to connect to that remote machine you are performing the health check on:

13:57:52   Error    |     System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
13:57:52   Error    |     ---> System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

The System.Net.Sockets.SocketException (10060) error relates to this issue.

Did your networking team allow access to that machine? Are you able to ping it from the Octopus Server? I do not know how your networking team have setup the ‘allow’ for the aws account but if they have just let through sts.amazonaws.com they may have locked it down too much and you may need to add your servers into the rules too.

Let me know if you can ping those AWS machines from the Octopus server, if not unfortunately you will need to liaise with your networking team again to ensure you can ping those machines in order for Octopus to be able to deploy to them and run health checks.

I look forward to hearing the results of the ping.

Please reach out in the meantime if you have any further queries.

Kind Regards,

Clare Martin

Hello,

Are you suggesting us to ping eks endpoint from the octopus server ?

I am unable to connect to eks cluster .Kindly suggest me .

And i can see that from eks cluster Security group , the 443 port is opened to octopus server

Hi @yogitha.kakarla,

You need to see if you can ping the machine DNS name you are trying to do a health check on in Octopus (eks-t5npint1) or its IP address, this will tell you if you can connect to it or not. The ping needs to come from the Octopus Server itself.

Your network team may have allowed the eks endpoint through (as you can now ping sts.amazonaws.com) but that does not mean you can then access all of the resources within that endpoint.

You need to allow access to the resources located within your AWS account. So the VMs will need to be individually granted access. I do not know how your networking team have alllowed access to the EKS endpoint but if they have locked the server down a lot you will be experiencing these issues.

Unfortunately we cannot advise users how to setup their network, all we can do is try and point you in the right direction, it is down to your network and security team to allow / deny access to resources in order to get Octopus to talk to your external VMs (tentacles).

Let me know if you can ping eks-t5npint1 (or its IP address), if you cannot you will need to talk to your networking team and they will need to allow that IP through whatever firewall or access point they have setup.

I look forward to hearing from you,

Kind Regards,

Clare Martin

Hello,

we can see that we are able to telnet EKS API endpoint to 443 port from Octopus server

Hi @yogitha.kakarla,

Thank you for getting back to us and confirming you can telnet into that EKS API endpoint, the support team are having a deeper look into this for you but it will take us some time to reproduce and try and figure out what needs letting through so you can access what you need to.

We will be in touch as soon as we have some information for you,

Kind regards,

Clare Martin

Hi @yogitha.kakarla!

Just jumping in for Clare here - I did some checks to verify here on our end, when we do the healthcheck, it was just the two sequential connections, one to sts.amazonaws.com on tcp/443 and then one to the cluster’s API management endpoint (in our case x.eks.amazonaws.com on tcp/443). If those are open and accessible from the Octopus server, then health checks should function correctly.

I wonder if using a tool like Wireshark (to analyze the connections being made) might be helpful in this scenario, then verifying that your network team has opened up the appropriate network traffic.

I hope this helps!

Hello Clare,

Thanks for the update.
We will wait for your response.

Thanks ,
Yogitha

1 Like

Hi @yogitha.kakarla,

I am not sure if you managed to give Justin’s response a read, he mentions what endpoints and ports he had to open to get that working, you seem to have those open from our discussions so Justin has suggested using a program such as Wireshark to help you with your investigations, this will tell you where network traffic is being routed and will help you analyze the connections being made.

After that you can verify your networking team has opened up the appropriate traffic - as per Justins comment of the two connections - sts.amazonaws.com on tcp/443 and one to the cluster’s API management endpoint (in our case x.eks.amazonaws.com on tcp/443) .

Let me know if you need anything in the meantime, getting your network traffic analyzed by a network analyser such as wireshark will help in this investigation.

Kind Regards,

Clare Martin

Hello,

Thanks for update , Apologies for late reply

we do not have wireshark in our env /server

If you could tell us any other alternative , it would be helpful

Hello,

Thanks for update , Apologies for late reply

we do not have wireshark in our env /server

If you could tell us any other alternative , it would be helpful

Thanks,

Yogitha

Hi @yogitha.kakarla,

Unfortunately, I cannot recommend any pre-installed products that would come built-in with Windows or Linux that would allow you to see more advanced network traffic information.

Wireshark is commonly used on networks and gives you a more in-depth network traffic analysis of your network, that is why I mentioned it. If you are unable to install that product you would need to liaise with your network and security team to see if they can advise a similar product.

I can only advise on Octopus issues and unfortunately until we know exactly where you are getting blocked network traffic-wise we are unable to help you any further, even then we can only advise on the Octopus side of things, for security reasons we cannot advise you on what ports to open with regards to Azure (other than the ones we already have as they are common ones which we know we have to let through to allow Octopus to talk to Azure).

Justin has confirmed the only things you need to allow through are the ones he listed in his previous comment on the thread and if your networking team have advised those rules are already in place to let those through and you are still having issues then you would need to install a more advanced network traffic product in order to see where these issues lie.

I am really sorry we cannot help you any further with regards to this, once you have analysed your network traffic you should be able to see where you are getting blocked but this, unfortunately, does now go beyond the scope of what Octopus Support can advise you on.

I hope you get this issue sorted and you are able to connect to Azure with that VM, the only other way to test this would be to open that server up completely to the internet and see if you can connect to Azure, if you can you know for a fact this is a network issue, however, I cannot advise you to do that, that would be down to the network and security team on your domain to advise, it would mean you would know that your server is locked down somewhere so your networking team could then re-investigate from their end.

Let me know if there is anything else we can do for you but for now, we would need your networking team to analyse your network traffic from the Octopus Server to Azure and see where the connectivity issues lie before we investigate any further,

Kind Regards,

Clare Martin

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.