Multitenancy, config transforms, and you

(Ryan Adler) #1

We are very excited about leveraging multitenancy (once the mix-and-match issue has been solved: http://help.octopusdeploy.com/discussions/questions/9031-tennant-and-non-tennant-deploys-to-the-same-server) in order to make our deploys and variables/configurations easier to manage. In our case, we will be using the feature to cover an entire environment (ie. Prod or QAT) which may span multiple data centers in multiple countries. We currently handle this with one environment per data center.

Now comes the interesting part. How will our config transforms work with multitenancy? I have brought this up in the past, before the feature was completed, but I noticed that there is no mention of it in the final docs. I suppose I can use the tenant system variables (ie. Octopus.Deployment.Tenant.Name) to apply custom transforms, but then I will have to manage the order, hierarchy, etc. implicitly in our processes, and do so for each step individually. I guess the question is - will config transforms ever be “first-class citizens” in Octopus, or does the team recommend leaning toward user variables in Octopus? It’s starting to look like this is the way forward - I am just not sure about removing configs from source control/change tracking. I would like to get some insight as to the Octopus team’s feelings.

Thanks!

(Michael Noonan) #2

Hi Ryan,

Thanks for getting in touch to talk about config transforms and tenants. Forgive me if I go over things you’re already comfortable with, I know you’ve been asking us for help/guidance/insight regarding configuration management for a while…

Config transforms first-class

I would personally like to understand what behaviour you’ve seen in Octopus that causes you to interpret that Config Transforms are not first-class citizens, or that we have any plans to deprecate support for them. As part of our effort to port over to .NET Core we’ve had to build a .NET Core port of the config transform engine, and we’ll continue to evolve our support for XML config transforms as long as they’re relevant to our customers. :slight_smile:

If you are comfortable perhaps you could share some specific details about how you use Octopus + Config Transforms etc today? (Perhaps in a Private thread if that would be more suitable)

Config transforms and Tenants

Back to configuration transforms specifically, Octopus will run some of your configuration transform files automatically based on the conventions we describe here - but we haven’t currently implemented a convention that would run a transform file for a tenant, like web.TenantName.config. Would that be something you’re interested in, and help deal with the “first-class citizen” situation you’ve mentioned?

If so, when it comes to ordering, here is the code in Calamari that finds the automatic, convention-based transforms: https://github.com/OctopusDeploy/Calamari/blob/master/source/Calamari/Deployment/Conventions/ConfigurationTransformsConvention.cs#L43

Since no precedence has been set previously, we could order the transforms like this, by adding Tenant after the existing conventions:

  1. Release
  2. Environment
  3. Tenant

How I like to configure applications

When it comes to configuring applications personally, I try to use the simplest approach that could possibly work, which in Octopus terms is updating AppSettings and ConnectionStrings by convention. I will usually end up layering some static configuration transforms on top of that to change the ASP.NET hosting environment using web.Release.config and web.MyProductionEnvironment.config style files which will be applied automatically by convention, which is also really convenient.

If I can’t get away with that approach, and I need custom configuration transforms, I usually like to leverage the deployment feature ordering and take a layered approach leveraging several features in coordination, something like this example. This kind of approach usually helps me tame the complexity of my configuration, and keep sensitive values in Octopus.

When considering where to source the configuration, I personally consider a few different things:

  • Sensitive values go in Octopus (or another key vault)
  • Values that I don’t intend to change at deploy-time go in source
  • Values that I intend to vary by tenant, environment, role or deployment target go in Octopus
  • I try to keep configuration to a minimum and use conventions wherever possible (to tame blowout and complexity)
  • I try to use appSettings in preference to configuration transforms wherever possible, again for simplicity
  • I use configuration transforms only when I actually need to transform something outside appSettings, and I will most often pair Substitute Variables in Files and config transforms in tandem as I described earlier

Future

Looking to the future, if Variable Templates are proving to be a good idea, we’d like to consider extending them into Environments, Deployment Targets, and potentially Roles - so you can define variables where they vary, again taming the Project and Library Variables.

Hope that helps!
Mike

Multi-Tenant deployments Update web.config appSettings
(Michael Noonan) #3

Hi Ryan,

I just wanted to let you know I’ve added this GitHub Issue after considering this enhancement for a few days: https://github.com/OctopusDeploy/Issues/issues/2754

I’d still be interested in your thoughts to my previous post. :slight_smile:

Hope that helps!
Mike

(Ryan Adler) #4

Hi Michael,

My apologies for not responding sooner - things have been somewhat busy here, and I wanted to give you my full attention. Thank you so much for the detailed answer, and beginning work on support for tenant transforms. I think this will go a long way to achieving what we are looking for. I will try to address each item you mentioned above. Sorry if it’s a big long-winded.

I would personally like to understand what behaviour you’ve seen in Octopus that causes you to interpret that Config Transforms are not first-class citizens, or that we have any plans to deprecate support for them.

To be honest, I guess I am referring to the company support perspective: it seems that transform support is being handled as an afterthought , rather than a v1 release feature. Of course, this is clear from the current discussion, where transform support was not included in the first tenants feature release. I appreciate that it is being pursued now, and that work was done in the background to support it, but I think you can understand what I mean. While variables are fully supported, transforms are not, and this makes it feel like a second-class feature from a strategy perspective. I just wanted to be sure that we can count on the transform support going forward with new features.

If you are comfortable perhaps you could share some specific details about how you use Octopus + Config Transforms etc today?

Sure! Our team (there are others using Octopus internally, but I am speaking just for our team) has 9 projects, 7 of which use transforms (the other two are Linux-based deployments). In those 7 projects together, we deploy about 40 packages, across 6 enviroments (and this is growing). Some of these environments are tenants-as-environments for different data-centers around the world (Production-City1, Production-City2, etc.). We plan on merging these into tenants once the feature set it there, of course. As you can already imagine, each environment requires various differing configurations - connectionStrings, but also internal variables, different logging configurations, email server settings, etc.etc. It can get very unwieldy (imagine nlog configs which differ by tenant), which is why we manage all the configs with transforms.
We leverage SlowCheetah internally so that we can transform config files while developing and building projects. Once we build on the TFS server, we disable the transforms, and let Octopus take care of that, since we don’t want to have to produce tons of different build configurations.
Keeping configs in source control has a huge advantage for us, which is traceability. Octopus simply doesn’t offer such a view for when someone changes a config, and we need to know that the config was changed due to requirement/work item XYZ. This is important for us, and moving the variables out of the software lifecycle process can be an issue.

On a related note, as I mentioned in another thread somewhere, I think that Octopus (as a company) needs to decide where they stand with relation to configuration management specifically. It’s nice to support all the workflows, but at some point, you will be focusing more on one workflow or the other, intentionally or not. I feel it’s also a huge potential for the company if they can offer a robust solution for config management (the other solutions in this space are seriously lacking, IMO, and Octopus is in a unique position to take advantage of that, with its install base). But maybe this is a discussion for another time. :slight_smile:

Would that be something you’re interested in, and help deal with the “first-class citizen” situation you’ve mentioned?

Yes, I think this is the best overall way to do this. Convention-based transforms, plus the ability to specify custom transforms should cover our use-case pretty well. We (or others) will just have to be careful about naming Environments and Tenants with unique names, or there could be some problems (maybe you have a solution for that).

How I like to configure applications

This sounds like a logical way to split up the configs - but again, the disadvantage here is that you have split up the configs. :slight_smile: Now, if someone wants to find out why something is configured wrong, they have to check:
The root app.config
The transform(s) for the environment(s)
(Later, the transform(s) for the Tenant(s))
Octopus variables
Octopus Confguration Variables steps
Octopus Custom Transforms steps

And now, even if I found that something is wrong in one of the Octopus items, it is non-trivial to determine who, what, when, and why it was changed. I may now break someone else’s deployment if I put it back (especially with regards to variables and tenants), and they will have the same issue.

I hope this gives an idea of how we work, and where our concerns lie. You and your team do great work, and truly listen to customer feedback, and we really appreciate that. Please keep it up, and keep the channels open!

(Mike Jacobs) #5

We have the same issue in our enterprise, where config files (and many of them) vary by tenant as well as by environment. Our solution was to develop our own config transform engine. It uses the same XDT syntax that is already used for config transforms, but we extended it by creating our own ‘transform’ file (XML) that has sections in it which we flag with tenant ID’s and environment ID’s.

When we deploy, we push both the raw, un-transformed, config file to the server, as well as the transform file(s). We then run our transform engine on it, using the tenant ID and environment of the server (using Octopus variables and now tenant variables, scoped to the tenant and environment) as parameters. The transform engine applies the appropriate sections (based on the tenant and/or environment) and produces the config file appropriate for this server.

If anyone (other Octo users or OctoCorp itself) is interested in learning more about it, let us know.

(system) closed #7