We have a working Octopus setup (server + worker tentacle) under Windows Server(s) but now we have to install one worker under docker, in Kubernetes cluster. Before moving tentacle to K8S I started with a simple step of running it on my local Windows laptop in 3 scenarios:
Just installed windows tentancle with MSI → successful (with latest tentacle msi: 6.3.417)
On the same windows laptop, but under the Ubuntu/WSL engine → Failed to connect (using this instruction tried, the latest version)
On the same windows laptop, but under linux docker container → Failed to connect same way (using instruction on dockerhub, tried with 3 versions: 6.3.451, 6.3.417 and 6.1.961)
The server version we’re currently running is 2021.1.
In both failed scenarios (#2 and #3) I received the same error, it looks like it has problems with SSL connectivity: “Checking that server communications are open failed with message The SSL connection could not be established, see inner exception”
The only thing that looks suspicious to me is the SSL Certificate indeed of the Web Portal (is that a correct name to use?) under port 10943 of our Octopus server. When entering in chrome it shows a problem with certificate
I would instantly blame our SSL/DNS settings, however, looking at the cert itself, it looks like it belongs to Octopus, as the cn is “Octopus Portal”.
The certificate itself is self-signed for that port, hence why it will be coming up as it does, so that error is to be expected.
One thing that stands out in your log is Received an unexpected EOF or 0 bytes from the transport stream. This usually indicates a TLS mismatch between the application and server. We have a documentation page here on troubleshooting and potential solutions for this. It might also be worth trying this command from WSL and checking the results: curl -vvv https://samples.octopus.app:10943
Let me know how this goes and we’ll see if we can help further if this doesn’t help identify the cause.
Kind Regards
Sean
Just a quick extra bit, if you add -k as an argument after -vvv that should get you past the self-signed certificate. You can also try on port 443 as well.
The curl request is sending a TLS1.3 request on the OUT, but it fails after that with either the server returning nothing or getting invalid data. We’re wondering if there the machine your Octopus Server is hosted on is an older version of windows that includes protocols that OpenSSL3 might consider unsecure. Linux deprecates a lot faster than windows, so that might be the issue maybe, but I cannot be too sure.
While we’d love to help out further, it will be difficult for us to help diagnose this since it falls into more of an infrastructure problem. If anything comes back after diagnosing the curl connectivity issue we’d be more than happy to help with further help regarding the tentacle.
Just jumping in for Sean quickly - I was doing some further research here, and it does look like the SYSCALL and what you’re seeing is the same issue.
From the OpenSSL documentation: The SSL_ERROR_SYSCALL with errno value of 0 indicates unexpected EOF from the peer. - this lines up with what Octopus was seeing ---> System.IO.IOException: Received an unexpected EOF or 0 bytes from the transport stream..
As Sean mentioned, this does look like something infrastructure-wise (potentially something like a proxy or a WAF?) that’s blocking these connections.
Thank you guys (@Justin_Walsh and @sean.stanway ) for looking at this, I’m closer! There is no proxy, nor WAF as
the option #1 - communication from my windows laptop is ok
but options #2 and #3 which are from the same laptop, but from the Ubuntu/WSL (2) or Docker(3) do not succeed.
Unfortunately, I have limited abilities to do anything on the Octopus Server. But I can modify anything in my WSL or docker engines.
I do wonder if it’s an incompatibility with OpenSSL and the configuration of your Octopus server. I assume you were using Ubuntu 22.04 with your WSL install, and our Tentacle docker container uses mcr.microsoft.com/dotnet/runtime-deps which is currently based on 22.04 as well. This ships with OpenSSL3, and it could be that there’s some incompatibility with that and what your Octopus server supports.
Do you know what OS your Octopus Server is running? You might also be able to use sslscan from your WSL install to scan your Octopus server to see what TLS versions etc. it supports.
Testing SSL server samples.octopus.app on port 10943 using SNI name samples.octopus.app
SSL/TLS Protocols:
SSLv2 disabled
SSLv3 disabled
TLSv1.0 enabled
TLSv1.1 enabled
TLSv1.2 enabled
TLSv1.3 disabled
Hope this gives you some ideas for investigation avenues, and please let us know if you have any further questions.
Sadly, that all looks normal, and nothing immediately sticks out as problematic - I was mostly looking at the TLS version support, which seems fine. I’m a little puzzled by this one - does the scan on port 10943 show anything different (if you have a load balancer or anything else in front of the http(s) ports?
Apologies for the late reply. I had a look at the port scan, but can’t see anything that stands out aside from TLS1.3 being disabled.
Justin messaged me when he came online just now and stated he tried to curl to your server and got the same result that you are currently seeing when trying to connect.
The only thing we can suggest is to look at the Octopus Server logs when a connection is attempted to be made and see if there is any calls at all being processed? It might not even be getting into the servers API. You should be able to find this under “Logs” in your Octopus install folder.
I finally got the connectivity by lowering TLS requirements in my docker container.
Lowering minimum TLS requirement in my docker image did the work (I found the solution here)
sed -i -e “s|^MinProtocol = .*|MinProtocol = TLSv1.0|g” “/etc/ssl/openssl.cnf”
Adding this finally allowed my polling tentacle to connect and successfully register against the server:
I know that this isn’t the right solution, but at least I can move forward with some PoC work. The Octopus Server I’m connecting to will be removed anyway soon and we’ll end up with fresh installation, under docker so the missing SSL requirements should be gone then.
@sean.stanway and @Justin_Walsh - big thanks for helping me here, I really appreciate your commitment guys!
