Deployment targets in multiple regions using one account

Running into an issue, and unsure if i am doing something wrong or missing something obvious.

Setting up a new environment in Azure and we have 2 azure regions in play. AusEast and AusSouthEast.

The web apps deployed to these regions are in the same subscription, and the account i am using to add the deployment targets has contributor permissions for the resource groups in both subscriptions.

Yet when i try to add an app service from this account, i only ever see the resources in AusSouthEast and not the AusEast resources.

Is there something i am missing where i cannot use the same account across multiple regions in azure? Or is this some other arbitrary limitation.

Hi @paul2,

Welcome to the Octopus Community, thanks for reaching out!

You definitely should be able to see any resources that the service principal is configured access to without any restrictions such as regions.

I’d like to confirm if the Service Principal also has Reader permission on the Subscription? Our docs about least privilege for Azure accounts mentions:

Firstly, you might want to constrain the service principal to a single resource group, in which case, you just need to assign it the Contributor role on the resource group.

Next, if you want to get even more granular you can constrain the service principal to a single resource, e.g. a Web App. In this case, you have to assign the Contributor role on the Web App and explicitly assign the Reader role on the subscription itself.

It sounds like it might be only using one Resource Group instead of checking the Subscription for any other Resource Groups to use, if you use the Azure Account in a Run an Azure Script step to list the resource groups & web apps, are you able to see all the resources?

az group list --query "[?location=='australiaeast']"
az group list --query "[?location=='australiasoutheast']"

I’ll dig into this and see if I can spot anything odd going on, feel free to reach out with any questions!

Best Regards,

Hey Finnian, thanks for your response.

I have since altered the roles and added Reader to the subscription, this resulted in a minor change, it now shows me an additional service in a different resource group in AustraliaSouthEast. So i know the reader change took effect, however it is still only showing targets in AustraliaSouthEast and none from AustraliaEast.

I tried adding you az query to a Run an Azure Script Step, but it fails saying

ObjectNotFound: The term 'az' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. 

It looks like it is still fixating on resources in one site and not aware of the other, even though it has reader across the sub

Hi @paul2,

No problem, thanks for testing that!

I haven’t been able to reproduce this and have confirmed that it doesn’t actually require Reader on the Subscription Group and should work fine with just the Contributor role on each Resource Group only. It seems like there’s something preventing access on your end, could there be any Deny Assignments configured, or condition?

The Azure script looks like it failed to find the Az CLI tooling on the target, if the option ‘Use Azure tools pre-installed on the worker’ is selected then the Azure CLI az command will need to be available in the $PATH, check out our docs about it here:

If you’d like to send through any screenshots of your configuration to our Secure Upload Portal, I’d be happy to see if I can spot what’s going on. I’d also like to confirm which version of Octopus you are using?

Example of the role assignments I configured on each Resource Group:

Feel free to reach out with any questions!

Best Regards,

Finnian, i am stumped, i tried a few things since the last update and am still in the same boat.

I tested with some of our other accounts that have multi region targets and they were all fine.

I tried adding a new account and giving it permission, but it too can only see the targets in AustraliaSouthEast.

I deleted the targets and accounts from octopus, and created a new app account in Azure, and still get the same issue.

This leads me to think it is something weird on our azure, but i can find no Deny Assignments either.

Its more than a little perplexing.

Oh forgot to add, we are on the cloud version v2023.1.

I just tried adding a target is East US, and that worked.

Hi @paul2,

Agreed, this is definitely one of the more perplexing issues I’ve seen!

If you use the “Check Access” tool under Access Control (IAM), are you able to confirm that the Service Principal has access to all of the resources inside the Resource Group?

Otherwise using the Azure CLI with that service principal using the --debug flag might reveal more info about what’s going on. Is it possible there is Conditional Access configured? E.g. MFA required

It might be worth getting Azure support involved if you aren’t able to locate the issue, unfortunately I’m not sure how much help I can provide since it seems Octopus is working fine with the other targets.

Let me know if you have any questions!

Best Regards,

We’ve raised a support ticket with MS.

In testing even my account cannot see the app services in Aus East via the cli tool, even though they are there in the portal. But the issue is only present in one of our subscriptions. Very strange times indeed

So further follow up, this appears to be an issue within the az api’s, but it also highlights that the octopus cli relies upon that also.

After playing around for a while we realised the ‘az webapp list’ command was only returning the endpoints from one region for this subscription, however if we ran ‘az webapp list -l australiaeast’ and explicitly set the location it shows the web apps from that region. It also worked when i specified the resource group for the deployment target.

Now based upon that we tried using the octopus cli, interactively that must use the equivalent of az webapp list to find the list of webapps. But here is where i think the octopus cli is letting users down is when i use the cli arguments it doesn’t appear to pass through the resource group (even though i am specifying the name of the RG).

Is there some other way i can add the target given i have all the values available whilst we wait for an MS reply, deployments to the targets are working for us from the cli, so expect if we can get it registered in Octopus, we will be able to deploy.

Hi @paul2,

I appreciate the follow up, thanks for sharing that info!

To add the target manually, check our docs for an example of adding a WebApp Target via the API, which should hopefully allow for the target to be added otherwise we have also released a new feature called “Cloud Target Discovery” which can locate Azure Web Apps at deployment time based on tags.

I’ll dig into this and check into how we are handling location with our Azure targets and check for any improvements we can make. I’ll keep you posted with any findings or updates!

Best Regards,

Turned out for anyone who gets here, this was an error in the API service from azure.

1 Like