Workers in Kubernetes aren't configured properly

Hi everybody,

I’m trying to deploy workers in our kubernetes cluster but the workers crash at their start.
Here’s the error:
Worker_error.txt (26.4 KB)

I’m following the tuto give by Octopus: https://octopus.com/blog/kubernetes-workers

I don’t know if the SSL error comes from the K8S cluster or from our self-hosted server (local DNS and local SSL certificate from our own authority).

Thanks in advance if you can help me.

Hi @douchet.pantoine,

Thanks for reaching out, I’d be happy to help with using a self signed certificate with Kubernetes workers!

It sounds like this will require you to create a custom docker image with your root CA configured if you haven’t already:

ADD your_ca_root.crt /usr/local/share/ca-certificates/foo.crt
RUN chmod 644 /usr/local/share/ca-certificates/foo.crt && update-ca-certificates

However from the error it looks like there could be a TLS issue, implying that it’s TLS v1 being used:

Interop+Crypto+OpenSslCryptographicException: error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version

Our docs about Tentacle Server Communication outline how TLS is negotiated from the underlying host:

Octopus Server and Tentacle rely on the host OS for the available TLS version to use when establishing a secure TLS connection when communicating.
The TLS implementation uses the .NET SslStream class, and uses the best available of TLS 1.2, TLS 1.1 or TLS 1.0. Fallback to SSL is disallowed.

So it could also be that the Octopus Server has been configured to not allow TLSv1 however I’d also like to confirm if the certificate Octopus Server is using is SHA1 or SHA256?

Feel free to reach out if you have any questions or have any additional details about your process for configuring the certificate!

Best Regards,

Hi,

The root certificate of our authority is already added to the container.

Our server is already harden by following the instructions here : Hardening Octopus - Octopus Deploy
So TLSv1 is disabled.

The certificate for our server DNS is using SHA-256.

Hi @douchet.pantoine,

Cheers for confirming that!

By default the Tentacle image should be using TLS1.2 so I’m not quite sure where TLS1 is coming from:
cat /etc/ssl/openssl.cnf

Could you please confirm which version of the Tentacle Docker image you are using and if you’ve made any other modifications to the image besides adding the certificate?

It might require explicitly setting the openssl system_default_sect setting to explicitly exclude protocols:

[system_default_sect]
Protocol = -SSLv3, -TLSv1, -TLSv1.1, TLSv1.2

Let me know if you have any questions at all!

Best Regards,

Hi,

I’m using the latest image. No other modification than the certificate.

I check the openssl.cnf file and it looks like your screenshot.
I added the Protocol config line and I still got the issue.

image

Hey @douchet.pantoine,

Thanks for checking that out for us.

I wonder if you’d be able to create a non-containerised worker on your network, to connect up to Octopus in the same way.

This would hopefully rule out or confirm the container image as being the culprit behind the issue.

Kind Regards,
Adam

Hi,

I try to install a new polling tentacle on our current Windows-based Worker and I have an other error:
error.txt (1.2 KB)

Here’s where I’ve got the error:
image

FYI, the firewall is deactivated on our Octopus server.

EDIT:
I also do a test on a CentOS server and it’s working this time. (by using this script : Linux Tentacle - Octopus Deploy)

Hey @douchet.pantoine,

Thanks for trying that out.

I’m not sure why the windows tentacle wouldn’t work and throw an object reference error.

Would you be able to send over Octopus Server logs from around the time you tried to register the tentacle? It may provide us more information about what went wrong there.

If you don’t mind sending over server logs, please feel free to upload them to our secure file upload.

Kind Regards,
Adam

Hi,

I just upload the log file (named OctopusServer.txt).

Regards,

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.

For any customer following this thread @douchet.pantoine was kind enough to post the fix for this in this post here so head on over there to see what they did to fix it.