We have an on prem Octopus deploy server and wish to deploy to an RBAC-enabled Kuberenetes cluster running on AKS. I’ve configured a Kuberenetes deploymemt target, but the health check hangs. I first specified a namespace, which gave no clue as to why. But when I removed the namespace, I saw that the Azure cli device login prompt is issued, which of course causes the health check to hang. Is there a way to provide the --admin flag when Octopus is retrieving the credentials? We are using a service principal by the way.
Ideally we would like to avoid the --admin flag, but that problem is on Microsoft’s end for not supporting connections using service principals without it.
Unfortunately we haven’t been able to reproduce this.
We setup a cluster on AKS using service principal authentication and no namespace, installed the latest version of kubectl (v1.15.0) into our Octopus Server and setup a Kubernetes Cluster target in Octopus. The target’s health check successfully authenticated from the PowerShellContext and passed a connectivity test.
Could this be an issue with the version of kubectl you have installed? Or possibly a permission issue with the service principal being used?
In regards to possible parameters, Octopus calls through to our open-source Calamari library, and will issue a command using this KubectlPowershellContext.
Were you able to determine what command causes the device login prompt to be issues? Is this happening when PowerShell issues the az aks get-credentials command, or when it tries to test connectivity?
Could you please include the raw task log when you deploy with no namespace specified, in case that offers us any clues?
A little more insight into how our cluster is set up:
We have deployed the cluster in our own private VNET using the azure network-plugin (as recommended by MS). Furhtermore we have set up an AAD server app and client app to use for RBAC in our AAD. This ensures that whenever someone does az aks get-credentials and then tries a kubectl command they are promted to login (the first time). The same occurs for a SP account, which is a known “issue”: https://github.com/Azure/AKS/issues/600
But when I look at this issue now, it may seem that the issue might be my inexperience with Kubernetes. I’ve found several sources, including from you guys, talking about using service accounts for authentication. Something in the lines of this: https://devopscube.com/kubernetes-api-access-service-account/. May that be the issue here? That we cannot use a target with a SP account with our current setup? In that case, I’m a little unsure on how to set up a target using a service account, which (I guess) will “bypass” the RBAC login prompt.
For the record: If I just create an Azure Powershell step running with our general Azure target (that we use for deploying Web Apps and such), using az aks get-credentials --admin works, so we do have a workaround. It feels a bit hacky though.
kubectl version: 1.15.0. Installed via the az aks install-cli command
To workaround the issue of the login prompt, you can create an access token, create a Token account in Octopus and change your Kubernetes target to use the Token account.
We have instructions on our documentation which has some information on getting the access token from your cluster.