Tenant not appearing as a deployment target

[mike]

Hi,

I’ve hit a specific problem a couple of times now, so I thought I’d mention it as it’s a really dastardly problem to troubleshoot.

In short, it’s the scenario specified by bullet point #4 on this page: https://octopus.com/docs/deployment-patterns/multi-tenant-deployments/troubleshooting-multi-tenant-deployments, but I’ve lost several hours over the last couple of days trying to diagnose it, only to face-palm when I eventually found that page (and once I did find it I remembered reading it last time I had the same problem!).

I have two projects that are connected to multiple tenants - one was deploying fine to all tenants in our PROD environment, but the other wouldn’t let me deploy to a specific tenant in PROD. I spent a long time checking and double-checking the config for tenants, tag sets, connected projects, tentacle roles and tenants, tenants and tag sets assigned to an account, and a bunch of other config that I remember affects what tenants can be deployed to where,

It turns out the project that was working had all environments in the lifecycle assigned to one tenant (which was promoted up through the lifecycle phases), and the others only had PROD, thereby bypassing the lifecycle completely, which meant that any tenant could be deployed to PROD.

The project with issues had one tenant being deployed up the lifecycle phases, and the others being deployed to our DEV and TEST phases, but not PREPROD. As result it wasn’t available to deploy to PROD because the lifecycle was in effect due to multiple environments, but also not satisfied because it hadn’t been deployed to PREPROD for the “missing” tenants.

Something that would have helped me massively would be if projects connected to tenants with a single environment maybe had a warning icon on that page saying something like “connected projects assigned to only one environment will bypass the lifecycle”. That way when I review connected projects for a tenants, I’d easily see that one had the warning and one didn’t and it’d be immediately obvious that having only one environment connected causes different lifecycle behaviour to when multiple environments are connected.

Sorry if this sounds like a bit of a rant - I appreciate why the behaviour is different, I just keep forgetting that the difference exists and it would be really helpful to have a visual reminder in the UI.

Cheers,

M

[mike]

In case my description above isn’t very obvious, my config was something like this:

My Lifecycle
DEV -> TEST -> PREPROD -> PROD

Tenant 1
Project A - connected to DEV, TEST, PREPROD, PROD
Project B - connected to DEV, TEST, PREPROD, PROD

Tenant 2
Project A - connected to PROD
Project B - connected to DEV, TEST, PROD

Project A
Tenant 1 - deployed to DEV, TEST, PREPROD
Tenant 2 - not deployed

Project B
Tenant 1 - deployed to DEV, TEST, PREPROD
Tenant 2 - deployed to DEV, TEST

Just looking at that config, it wasn’t immediately obvious (to me, at least) that lifecycles didn’t apply to Tenant 2 for Project A, but did for Project B, and since Project B for Tenant 2 wasn’t in PREPROD it couldn’t be deployed to PROD, whereas Project A could be deployed to PROD because it bypassed lifecycles.

[TomPeters]

Hi,

Thanks for getting in touch, and thank you for mentioning this problem. My investigation of this issue helped to refine my own understanding of tenanted deployments

This sounds like it could actually be a bug. If I understand correctly, you are saying that you cannot deploy your release to PROD for Tenant 2 in Project B.

Let me explain why I think you should be able to create that deployment, and therefore why I think this sounds like a bug. In my following description I will assume that each lifecycle phase contains just one required environment.

In order to deploy a release to a tenant in a particular environment, it must first “pass” each required lifecycle phase that precedes that environment. The rules for passing a phase for a tenant are slightly more complex than for untenanted projects.

In order to progress a release past a lifecycle phase for a tenant:

1. If the tenant is connected to that environment, then that release must have been deployed to that environment for that tenant
2. If the tenant is not connected to that environment, then that release must have been deployed to that environment for any tenant or as an untentanted deployment

In your scenario, you should be able to deploy to Prod for Tenant 2 in Project B because all of the preceeding phases have been passed:

• DEV has been passed because you deployed to Tenant 2 (rule 1)
• TEST has been passed because you deployed to Tenant 2 (rule 1)
• PREPROD has been passed because you deployed to Tenant 1 (rule 2). This is because Tenant 2 is not connected to PREPROD

Because these three phases have been passed for Tenant 2, you should be able to deploy Tenant 2 to Prod.

I tried configuring a similar scenario to demonstrate how this works. In this project my lifecycle is Dev -> Test -> Prod. Tenant D is connected to Dev and Prod only (not Test).

I can deploy to Prod for Tenant D even though I have not yet deployed to Test for Tenant D. This is because Tenant D has passed the Test part of the lifecycle as a result of me deploying Tenant B to Test.

So if you really can’t deploy to PROD, then this might be a bug. I wonder if there might be other factors at play. Do you use channels in your deployment? Or do you have more complex lifecycle phases (optional phases or multiple environments per phase)?

Could you also let me know what version of Octopus you are using?

Look forward to hearing from you,
Tom

[mike]

Hi Tom,

Thanks for the response. Your additional detail makes it a lot clearer to me what’s going on, and it does actually behave as you’ve described. It turns out there’s a glitch in my example and it wasn’t actually identical to our project setup in a couple of critical places.

I picked through everything and this is more like our current setup:

Lifecycle

Test
  env-248t
  env-258t
PreProd
  preprod
Prod
  prod

Connected tenants / projects / environments

Tenant-EU
+ Project A - [env-248t] [env-258t] [preprod] [prod]
+ Project B - [env-248t] [env-258t] [preprod] [prod]

Tenant-US
+ Project A - [env-248t] [env-258t] [preprod] [prod]
+ Project B - [env-248t] [env-258t] [preprod] [prod]

Overview

Project A - Overview
           [env-248t]  [env-258t]   [preprod]     [prod]
Tenant-EU   1.4.85.0     1.4.1.4     1.4.1.4   1.4.85.0
Tenant-US   1.4.69.0    1.4.85.0    1.4.85.0   1.4.85.0

I iterated through the deployment options and this was the available targets for the version we were trying to deploy (1.4.1.4 - the version numbers are a bit screwy… don’t ask!):

Project A - Available Tenants for release 1.4.1.4

env-248t - Tenant-EU, Tenant-US
env-258t - Tenant-EU, Tenant-US
preprod  - Tenant-EU
prod     - Tenant-EU

Using the promotion rules you’ve described above, this makes sense because:

• Tenant-EU can deploy everywhere because it currently satisfies all phases of the lifecycle.

• env-248t and env-258t are connected to Project A for Tenant-US, and are in the first phase of the lifcycle (i.e. Test) so are available

• preprod is connected to Project A for Tenant-US and so the lifecycle phase Preprod is active, but not satisfied, so cannot be deployed to for Tenant-US.

• prod is connected to Project A for Tenant-US and so the lifecycle phase Prod is active, but not satisfied, so cannot be deployed to for Tenant-US.

As a first step, I removed env-248t and env-258t from Tenant-US for Project A, then checked deployment options and preprod had become available to deploy to because the “Test” lifecycle phase was satisfied by Tenant-EU.

Next, I removed preprod from Tenant-US for Project A then checked deployment options again - prod had become available for the same reason (i.e. the “preprod” lifecycle phase was satisfied by Tenant-EU).

To summarise, everything in Octopus is working as designed so there’s no bug - it’s my understanding that was at fault. However, some more detailed documentation (maybe with some examples and edge-cases of when each phase in the lifecycle does or doesn’t apply) or some additional guidance in the UI would be very much appreciated as it’s a really knotty piece of functionality and I’ll probably have forgotten all of this again the next time I come across it!

Thanks for taking the time to investigate though, and please feel free to close this ticket.

Cheers,

M

[mike]

Oh, and to answer your other questions - yes, we do also use channels in this project, and Azure Accounts (different ones for different environments, with different tenant tag set filters on each account!). And global variable sets that are used to identify which account is scoped to each environment :-(.

That’s what make it so difficult for me to get a handle on why I couldn’t deploy - there were too many free variables to pin down exactly where the issue was. It’s something for me to feed back to our internal dev team I guess - simplify your projects and it’ll be easier to troubleshoot!

Thanks again,

M

[TomPeters]

I’m so glad you worked it out Happy to help.

However, some more detailed documentation […] or some additional guidance in the UI would be very much appreciated

Couldn’t agree with you more. I’ll raise an issue with the team and see if we can expand the documentation a little bit in the short term. In the long term, I think we could definitely improve the usability in this area to make it clear what you can deploy to and, perhaps more importantly, why you can’t deploy to certain tenants.

Have a good one,
Tom