Dynamic Package ID using Tenant Variable

(Robert Davies) #1

We have a nuget package which has been named after our tenants.

We want to deploy the package as a part of our overall pipeline, however when we generate the release it wants a version number for that package, and each package has its own individual version, as some tenants have their package updated more often than others.

Am i missing something, or is there a way to specify that you want the latest release of a certain package without knowing the number

(Michael Noonan) #2

Hi Robert,

Thanks for getting in touch! I think I can see what you mean. In the screenshot below, I’ve attempted to replicate a simple scenario like yours, and I can see that while you can vary the Package ID, you are required to select a specific version of that unknown package.

Admittedly this feature was designed so you could build MyApp.Dev.2.3.0.nupkg and MyApp.Test.2.3.0.nupkg and MyApp.Production.2.3.0.nupkg and use MyApp.#{Octopus.Environment.Name} selecting 2.3.0 as the specific version, and deploy that “release” through your environments.

It seems like a logical extension that if we’re letting you “late bind” the Package Id, you should be able to add some kind of expression and “late bind” the Version also… but we don’t allow that now because we’ve never had a compelling use case for it.

I’m really interested to learn some more specifics about your situation and what you’re trying to achieve so I can help you best. A brief description and some screenshots would be wonderful. :slight_smile:

Hope that helps!
Mike

(Robert Davies) #3

Hi Mike,

The situation in your screenshot is pretty much exactly how it is being set up.

But we are trying to deploy a package containing client configuration files, so that when we deploy a clients site, we can also deploy our clients configuration files.

We version each package based on changes to the configuration files, so one client version may be ahead of a different clients version, which is why we really just need a way of selecting the latest version as it’s not a number we have much control over (not worried about redeploying the same version)

(Michael Noonan) #4

Hi Robert,

Thanks for getting back to me. I can understand the desire to only rev the tenant-specific package versions based on when they actually change. I had a talk with some of the others in the development team, and we think it makes logical sense: “If you can bind the Package ID, why can’t you bind the Version?” However, we didn’t think it would be a “normal” variable binding, but more some kind of version range expression like NuGet’s [1.0-2.0), or something like 1.0.* where * could mean latest.

While we think it’s a good idea, we do need to treat carefully around design issues like this, so I’ve created a User Voice suggestion for us to gather support and consider our options with yourself and other customers.

I’m sorry this sounds like it’s all bad news in the short-term. I have thought of a way to work around this in the meantime, which depending on the number of customers with customizations could be annoying to handle: In your project you could create a Package Step for each customer with a customized package, and then scope each step so it is only run when deploying to that specific tenant. Would something like this be a workable solution for now?

Hope that helps!
Mike

(Robert Davies) #5

Hi Mike,

We have 100+ clients, this would not be a feasible workaround.

Is this not a bug rather than a feature request though, as there is no way to deploy a package without the version, so the fact that you can bind the package id, but not the version, then the feature is broken.

Thanks

(Michael Noonan) #6

Hi Robert,

Thanks for getting back to me. I’ve been talking today with a few other developers about the feature that could alleviate this pain. The ability to provide a variable binding for the Package ID was originally to solve a different problem, where people build a different package per-environment, but with the same version number. This is the first time I’ve seen the desire to have a variable Package ID and Version, but I can also see how multi-tenant deployments will make this a common practice, for white-labelling scenarios.

One question I’d like to understand from your perspective is about the tension we have between repeatability-vs-volatility: If you re-deploy the same Release to a Tenant/Environment, but you have pushed a new version of the tenant-specific package, which version of the tenant-specific package would you expect to be deployed? (assuming you specified latest)

At this point I’d suggest we will end up resolving the Package ID and Version when the deployment starts (which is where we resolve most variables), but while the Package ID should remain stable across re-deploys due to variable snapshots, the interpretation of latest may have changed.

Look forward to hearing back from you.
Mike

(Michael Noonan) #7

Hi Robert,

I’ve created a GitHub Issue for you to follow and get involved with the design of this feature: https://github.com/OctopusDeploy/Issues/issues/2755

Hope that helps!
Mike

(jmo) #8

Hi Mike, Robert,

Sorry for jumping on this topic but I believe we are possibly in a somewhat related situation. (Vanessa Love referenced this discussion in my thread)

Very simple explanation of our situation:
We also have 100+ clients/tenants and have multiple packages that can be deployed in the exact same way (same step) for each of these. We thus have a tenanted project that handles the deployment and it would be very cool if at deployment time we could specify the package we want to deploy.

Even if we could create a variable or select the package variably, currently at release time we would require to select a specific package and version which in our case could be multiple packages and their respective versions.

Looking forward to hear from you if this is related according to you guys.

Thanks!

Kr,
Joren

(Mark Siedle) #9

Hi Joren,

Thanks for getting in touch. It definitely sounds related.

Would the proposed solution in our GitHub issue work from your perspective? ie. where we allow you to specify a version range expression such as 1.0.* or latest and Octopus would resolve the package at deployment-time to find the highest available version?

So you could specify a variable package ID like #{Octopus.Deployment.Tenant.Name} and then enter latest for the package version and Octopus would figure out the latest package version per tenant when the deployment starts.

Let me know your thoughts.

Cheers
Mark Siedle

(jmo) #10

Hi Mark,

I think that would be excellent.

If I understand correctly, this would allow us to specify a package by using a variable ID and with the “latest” or range expression allow us to deploy the latest version of the desired package through the same project release for each tenant.

This would in theory also allow us to create a release of a specific, older version when needed?

Kr,
Joren

(Mark Siedle) #11

Hi Joren,

Yes, using the latest expression will deploy the latest package for each tenant. If we allow a range expression eg. 2.0.*, then you could potentially deploy older versions of packages when needed. Do you find yourself needing to deploy older versions of packages for tenants (not latest)?

Hope this helps
Mark

(Roger Nesheim) #12

Hi.

After starting with tenants I seem to have come across a very similar issue. In our case we have our main application as a nuget package referenced from a tenant specific host project where we customize the application to the needs of the customer (for instance adding specific authentication through the owin pipeline, implementing custom rule engines etc.).

The deployment process for each customer is however identical, and as a result we get into the same scenario where we must have a dynamic package id, and versions will be different across each tenant.

As a option to what you have already described here, wouldn’t it be possible to create a tenant specific release (as opposed to deployment) where you resolve the package id based on a tenant parameter and then select the version of the package you want to deploy?

(Mark Siedle) #13

Hi Roger,

Thanks for getting in touch and thanks for your suggestion.

You are correct, that is a potential option.

With the way the multi-tenancy feature has evolved, we believe this problem can be solved without modifying the release table itself. If we can keep this to being a ‘packaging problem’ that we solve at a packaging level, then we hope to provide a solution that fits (and is flexible) for everyone’s tenant scenarios, while having minimal disruption to the rest of Octopus.

When designing this feature, we’ll definitely consider all options though :slight_smile:

Cheers
Mark

(jake.rote1995) #14

Hi

We are now starting to use tenants in our deployment process, but the main problem we face with dynamic package names is we would like to use the latest version of the package for each tenant at the time of creating the release, so give the following scenario.
Tenants:
Tenant1
Tenant2

Package name: #{TenantName}
Tenant1.1.0.0.nupkg
Tenant2.1.1.0.nupkg

and creating a release for 1.1.0 then the two following packages should be locked in and version should not change even if newer version is added, until a new release is created. Wondering if this would be possible with the current ideas for the new version constraints.

Thanks

Jake

(Mark Siedle) #15

Hi Jake,

Thanks for your feedback.

With the current proposal, while the package is calculated at deployment time per tenant, if we considered supporting a version range expression, that might ensure you can lock in version 1.1.0 to always get picked up for your 1.1.0 release.

We’ll definitely take all of this feedback into consideration before starting on the feature. I’ve added an additional note to the GitHub issue.

Cheers
Mark

(jake.rote1995) #16

Hi Mark,

Just to check would the current proposal allow me to use the latest version available up to the specified version like how the create release page works?

Thanks

Jake

(Michael Noonan) #17

Hi Jake,

I’ll continue the conversation from here for now. :slight_smile:

We haven’t locked anything in yet, so this is the best time to show us some more details about the example you have in mind.

For example, the NuGet Version Range format is very expressive, and we use this for Channel Version Rules, but we find some people get confused by it.

Other ecosystems support “star notation” for version tolerance, like 1.2.* and that comes with some tradeoffs also.

From what I interpreted from your message, you’d like something like [,1.1.0] which would allow you to specify a maximum of 1.1.0 inclusive.

One of the original suggestions was to simply allow for * which means “use any matching package, preferring the latest version available”.

Let me know what specific scenario you have in mind - that’s the most useful thing to help us validate our design!

Hope that helps!
Mike

(Roger Nesheim) #18

If anyone is looking for a workaround I just got one working today. Basically it is to create a tagset with a tag for each tenant, as well as channel for each tenant which will be limited to the tenant tag. From there you can create a project variable for the package, and scope that to the tenant’s channel.

When you create a release you then get the option to select channel/tenant and the correct package will appear with the option to select any that matches the channel version rule etc. This also makes the mapping of release version with package version work (that you can set on the project setting page).

It may not be the ideal case if you have a lot of tenants, or if you want to deploy many tenants at the same time, but at least it fits nicely with our process so perhaps it can be of use for others as well.

(Michael Noonan) #19

Hi Roger,

Thanks for sharing that idea! It’s actually a really neat concept if you don’t have a large number of tenants. The best thing is it removes some of the downsides to using dynamically resolved package details - specifically retention policies will work as expected.

For those still waiting on this feature, watch the GitHub Issue.

Thanks for sharing!
Mike

(system) closed #21