Error getting SSL certificate

(Alex) #1

Hi Octo Support,

Please assist with the below issue:

Version: 2019.3.2 LTS but we do not see anything in the 2019.3.3 LTS that hints it would address this issue.
Expected behavior: Kubernetes should deploy a TLS secret for a Kubernetes Ingress. The secret should be able to be used by our Kubernetes objects and the site should be able to use a deployed certificate to allow our site to use HTTPS.
Actual Behavior: The deployment succeeds and a secret is deployed out to Kubernetes with the Ingress. However, when we try to access the site, we get the following Kubernetes logs:

  • Error getting SSL certificate " [Scrubbed for SensitiveData] -services/octopus-certificate-certificates-[Scrubbed for SensitiveData]": local SSL certificate [Scrubbed for SensitiveData]-services/octopus-certificate-certificates-[ Scrubbed for SensitiveData] was not found. Using default certificate
  • Error obtaining X.509 certificate: unexpected error creating pem file: tls: failed to find any PEM data in key input

When we try to access the site, it gives us an “invalid certificate” error since its using the default certificate. When we deploy the same certificate in the Octopus Library manually, it succeeds and everything works.

We also notice that certificates that are deployed to our 2018.4.7 instance have the “Private Key” field equal to “Yes” while the same certs in our 2019.3.3 instance have “Private Key” field equal to “No”

We have an outstanding issue where there are some corrupted certs in our Cert library which may be affecting this (see other posted thread).

Steps to Reproduce: Deploy Kubernetes ingress, service, and deployment with a TLS certificate into Azure Kubernetes Service. The certificate is loaded into the Certificate Library and loaded via the Octopus Deployment. We are using an OOTB Deploy Kubernetes ingress action:

“Id”: “c37b1b77-b7ab-4940-bccf-7ff8c11c44e2”,
“Name”: “[Scrubbed for SensitiveData] Service Ingress”,
“PackageRequirement”: “LetOctopusDecide”,
“Properties”: {
“Octopus.Action.TargetRoles”: “[Scrubbed for SensitiveData].Service”,
“Octopus.Action.ConditionVariableExpression”: “#{K8.ShouldRunFullDeployment}”
},
“Condition”: “Variable”,
“StartTrigger”: “StartAfterPrevious”,
“Actions”: [
{
“Id”: “36c5035b-ad74-4ed9-b848-dec91e9305dc”,
“Name”: “[Scrubbed for SensitiveData] Service Ingress”,
“ActionType”: “Octopus.KubernetesDeployIngress”,
“IsDisabled”: false,
“CanBeUsedForProjectVersioning”: false,
“IsRequired”: false,
“WorkerPoolId”: null,
“Environments”: [],
“ExcludedEnvironments”: [],
“Channels”: [],
“TenantTags”: [],
“Packages”: [],
“Properties”: {
“Octopus.Action.KubernetesContainers.IngressAnnotations”: “[]”,
“Octopus.Action.KubernetesContainers.IngressName”: “[Scrubbed for Sensitive Data]-service”,
“Octopus.Action.KubernetesContainers.IngressRules”: “[{“host”:”#{K8.Ingress.Host}",“http”:{“paths”:[{“key”:"/",“keyError”:null,“value”:“80”,“valueError”:null,“option”:"[Scrubbed for Sensitive Data]-service",“optionError”:null,“option2”:"",“option2Error”:null}]}}]",
“Octopus.Action.KubernetesContainers.IngressTlsCertificates”: “[{“hosts”:[”#{K8.Ingress.Host}"],“certificateVariableName”:“K8.Ingress.TLS.Certificate”}]"
},
“Links”: {}
}
]
}

Actual Behavior

(Michael Richardson) #3

Hi Alex,

Thank you for the excellent issue description, though I’m sorry to hear you’re encountering the problem.

We believe both the issue with the k8s TLS certificate not being used and your certificates being shown as not having a private key, were resolved in 2019.4.2.

The private-key issue can be seen here. The k8s tls certificate issue was resolved as part of another change and doesn’t have a public GitHub issue.

You’re correct, these issues are not resolved in the latest LTS release.
If you are willing to upgrade to the latest version, this will be the quickest option. Alternatively, we are willing to apply these changes onto the LTS branch, and release a new LTS patch if this will assist you? Let me know if you would like us to do so.

For your reference, the k8s tls issue was that the secrets created to hold the tls certificate were being created as Type = "Opaque" rather than Type = "kubernetes.io/tls"

I hope that helps. We do sincerely apologize for the inconvenience.

(Alex) #4

Hi Michael,

Thank you for getting back to us. We are going to stay in the 2019.3 LTS lane, and we will need the backport patch made in 2019.4.2 available for our LTS lane.

Please let me know if you need more information on this from us on this request.

Thank you,

Alex

(Michael Richardson) #5

Alex,

These changes have been included in the LTS 2019.3.4 release, which should be available early next week.

1 Like
(Alex) #6

Fantastic. We will proceed with the upgrade next week.

Thank you for the fast response and fix inclusion.

Best,
Alex

(Alex) #7

Hi Michael,

Will the LTS 2019.3.4 release be available in this location: https://octopus.com/downloads/previous or in another folder in GitHub?

Thanks,
Alex

(Alex) #8

Just saw the upgrade. Thank you again!

(Michael Richardson) #9

You’re most welcome Alex :slight_smile: