Transformation - substitution target pattern

I have since a long time a working config transformation in my deploy step:

Web.config.Transform => Web.config

I now need to split this into 2 files:

SAML.Web.Transform.config
WAM.Web.Transform.config

For this I’ve specified an environment specific variable WebAuthTransform = SAML or WAM.
In my deploy step I specify the transformation

#{WebAuthTransform}.Web.Transform.config => Web.config

But when the deploy step is invoked I get this warning, and no transformation is performed:

No files were found that match the substitution target pattern 'Web.Transform.config

I also tried changing the filename to SAMLWeb.Transform.config, didn’t work.
Where is this target pattern specifified and how can I change it?

Hi @kenth.jonsson,

Firstly, let me welcome you to the Octopus community. It is great to see you here :wave:

I have set up a test in my local instance to try and replicate your question (if I have understood this correctly) and found that I got this to work.

This was my setup:

Created a test package with my transform files and uploaded to my Octopus instance:

image

Created a test project and added my scoped variables:

Next I added a “Deploy a Package” step to my project process and included the “.NET Configuration Transforms” feature:

For my Transform I added #{WebAuthTransform}.Transform.config => Web.config:

I created a release and deployed this to Development and Test and I can see the transform take place:

Development:

Test:

As you can see, Octopus transformed the file between my environments.

Is this what you are trying to do? Or have I misunderstood?

If it is of any value to you, here are the docs that go into more detail about how transforms work within Octopus: .NET Configuration transforms - Octopus Deploy.

While I am unaware of any issues with older version of Octopus, just in case, are you able to let me know what version of Octopus you are running? (I am using v2022.3).

If you are still having issues with this, are you able to Get the Raw Output From a Task and share this with me (omitting any sensitive details).

I look forward to hearing from you.

All the best
Doug

Apologies, I just noticed I had missed the .Web element from my file name (to match your scenario).

I have updated my package and modified my step and checked that this also works:

This worked for me also.

Regards
Doug

Thanks, you have done exactly what I’m trying to achieve.
Our Octopus is managed by the company and the version number I see in the browser is v4.1.5 which seems really old compared to your version.

I did a test according your first example, SAML.Transform.config, didn’t work.

Maybe what I’m trying to do is not supported in very old versions of Octopus

Hi @kenth.jonsson,

Given the difference between versions the logic may have changed.

I would have to test against your version, but I suspect this may be the case.

I will see if I can set this up locally with your version to see if I get the same and if I can find a work around.

Unfortunately here, if there are issues with the version you are running then the recommendation would be to look at upgrading your Octopus instance. I appreciate this is a big jump between versions and would consider a large amount of planning & resource from your side. But this is something we can also help to support you with.

Let me get back to you in (hopefully within the next day or so) with what I find and see if there is a workaround.

All the best,
Doug

Hi @kenth.jonsson,

I have just set up a test instance with your version and I can confirm this does work on your version (following the same steps as above).

I did have a few thoughts here:

Are you able to confirm if the files are present within your package or if the are stored within any sub-folders.

Are your variables set up and scoped correctly?

Finally, can you confirm if you are doing a new release since modifying your variables so that Octopus picks them up?

I look forward to hearing from you.

Doug

Hi,

I got it working with the SAML.Transform.config filename after rebuilding and creating a new deployment package.

The warning message still appears but the log confirms that transformation is done:

Is it possible to suppress the warning message?

Regards

Kenth Jonsson

Hi @kenth.jonsson,

I am glad to hear you got this working.

The warning comes from Octopus as it appears you are trying to use the “Substitute Variables in Files” feature within the deployment step, and the file that has been defined doesn’t exist within your package.

My assumption here is you have this configured:

Which will result in the warning as the file is not present:

A couple of options here for you.

  1. Is the feature needed within your deployment step? If not, you could disable the feature?
  2. If you want to ignore the warning message, you can add in the Octopus.Action.SubstituteInFiles.EnableNoMatchWarning variable set to False within your project to ignore this.

However, testing this against your version, this doesn’t appear to be supported so will not work in your version. I am unsure which release this variable was included, but I tested against my current version (2022.3) and this worked as expected.

More information can be found here for Octopus System variables.

Unfortunately, with older versions, you will be missing that availability of quite a number of these as they have been introduced throughout the newer releases.

I hope this has helped.
Doug

It didn’t work to suppress the warning message, I guess it’s not supported in our old version.
It’s not a big problem, I’ll leave it as it is.

so, I got the configuration transformation working with the SAML transformation but unfortunately
not with the WAM transformation.

I have a couple of variables declared in Octopus:

The WAM transformation file refers to these variables:

The web.config that is put into the deploy package do not contain the authorization and authentication elements

The idea now is to have the transformation insert the authorization and authentication elements with the Octopus variable values.

For some reason the variable names are inserted instead of the values so this is what we get in web.config when the deployment
is done:

The WAM transformation is our original file which has been working for several years so I don’t understand why it’s not working now.

Any idea what the problem might be?

Regards

Kenth

Hi @kenth.jonsson,

This should work using the “Substitute Variables in Templates” feature within your step.

You will need to make sure you point to the correct files within your package.

So for example my config files sit within a sub-folder called “dotnet_transforms”:

image

To reference this within my step I add the following(under “Substitute Variables in Templates”):

I added in a test variable to see this in action:

Within my .config files I have included my test variable:

Created a new release to pick up my variables, after the deployment has completed, after inspecting the Web.Config file, I can see the transform has worked (this is the same for both my files, picking up the scoped variable):

I did test this against your version and all worked as expected here.

We do have further documentation surrounding this which can be found here: Substitute variables in templates - Octopus Deploy

I hope this helps.

All the best,
Doug

Hello, thanks a lot for all the input.

However, my variable substitution refuses to work even with everything setup as required
so I’m reaching out once more to see if it is possible to figure out what’s wrong.

I have 2 elements in my Web.config file that must be updated by the variable substitution:

<system.net>

<proxy usesystemdefault=“True”

proxyaddress=“”

bypassonlocal=“True”

autoDetect=“False” />

</system.net>

The complete transformation file that is used, my Octopus variables in bold:

<?xml version="1.0"?>

<system.webServer>

<add accessType=“Allow” users**=“#{IISUsers**}” xdt:Transform=“SetAttributes(users)”/>

<basicAuthentication enabled=“true”

defaultLogonDomain**=“#{DefaultLogonDomain**}”

xdt:Transform=“SetAttributes(defaultLogonDomain)” />

</system.webServer>

<system.net>

<proxy usesystemdefault=“True” proxyaddress**=“#{ProxyAddress**}” bypassonlocal=“True” autoDetect=“False” />

</system.net>

The Web.config that is deployed:

<system.webServer>

<add accessType=“Allow” users**=“#{IISUsers**}” />

<basicAuthentication enabled=“true” defaultLogonDomain**=“#{DefaultLogonDomain**}”/>

</system.webServer>

<system.net>

</system.net>

The log output from the build step confirms that the transformation file has been loaded:

Transforming ‘W:\Octopus\ODS Test\OdsWeb\1.1.891\Web.config’ using ‘W:\Octopus\ODS Test\OdsWeb\1.1.891\WAM.Transform.config’.

All 3 variables are declared and scoped but for some reason only the #ProxyAddress variable has been replaced.

Any idea what I can do to get this working?

Regards
Kenth Jonsson

Hi @kenth.jonsson,

As you say you have 1 working, my assumption here may be something misconfigured with the scoping of the variables themselves or the variables are not being passed in correctly. It may be worth double-checking this.

A good thing to check here would be to write your variables out to the task log:

You can add the following variables to your variables section within your project using the Octopus System Variables:

OctopusPrintEvaluatedVariables = True
OctopusPrintVariables = True

I made a mock of your variables like below:

After my “new” deployment has completed I can inspect the verbose task logs for my deployment to see what variables Octopus has used for my deployment:

I also did a new transform test here:

Here is my code for my fictional config file “SAML.Web.Transform.config”:

<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <connectionStrings>
    <add name="DefaultConnection" providerName="System.Data.SqlClient" connectionString="DataSource=localhost\SQLExpress;Databasename=master;Integrated Security=true;" xdt:Transform="SetAttributes" xdt:Locator="Match(name)"/>
  </connectionStrings>
  <MyCustomSettingsSection>
    <TestMode xdt:Transform="Replace">#{My.Octopus.Variable}</TestMode>
  </MyCustomSettingsSection>
  <system.web>
    <compilation debug="true" targetFramework="x.0" xdt:Transform="InsertIfMissing"/>
    <identity impersonate="false" xdt:Transform="Replace"/>
  </system.web>
  <system.webServer>
    <add accessType="Allow" users="#{IISUsers}" xdt:Transform="SetAttributes(users)"/>
    <basicAuthentication enabled="true" defaultLogonDomain="#{DefaultLogonDomain}" xdt:Transform="SetAttributes(defaultLogonDomain)" />
  </system.webServer>
</configuration>

Here is my “Web.config” file:

<?xml version="1.0"?>
<configuration>
  <connectionStrings>
    <add name="DefaultConnection" providerName="System.Data.SqlClient" connectionString=""/>
  </connectionStrings>
  <MyCustomSettingsSection>
    <TestMode>True</TestMode>
  </MyCustomSettingsSection>
  <system.web>
    <identity impersonate="true" other="true"/>
    <compilation debug="true" targetFramework="x.0">
      <assemblies>
        <add assembly="System.Web.Abstractions, Version=x.0.0.0, Culture=neutral, PublicKeyToken=xxxxxxxxxxxxxxxxxxxx" />
        <add assembly="System.Web.Helpers, Version=x.0.0.0, Culture=neutral, PublicKeyToken=xxxxxxxxxxxxxxxxxxxx" />
        <add assembly="System.Web.Routing, Version=x.0.0.0, Culture=neutral, PublicKeyToken=xxxxxxxxxxxxxxxxxxxx" />
        <add assembly="System.Web.Mvc, Version=x.0.0.0, Culture=neutral, PublicKeyToken=xxxxxxxxxxxxxxxxxxxx" />
        <add assembly="System.Web.WebPages, Version=x.0.0.0, Culture=neutral, PublicKeyToken=xxxxxxxxxxxxxxxxxxxx" />
      </assemblies>
    </compilation>
  </system.web>
    <system.webServer>
    <add accessType="Allow" users="VariableToReplace" />
    <basicAuthentication enabled="true" defaultLogonDomain="VariableToReplace" />
  </system.webServer>
</configuration>

After deployment, my config file resembles the following (with my variables substituted):

If I run the deployment and my variable has been scoped incorrectly (in my case I scoped this to another environment that was not the environment I was deploying to) then I can see the following in my config (where my variable has not been substituted for users)

  </system.web>
    <system.webServer>
    <add accessType="Allow" users="#{IISUsers}" />
    <basicAuthentication enabled="true" defaultLogonDomain="saml.org" />
  </system.webServer>

And I am missing the variable within my task log as Octopus has not picked this up due not being scoped to my environment I am deploying to:

image

This was tested against the version you are running.

Please also note the warning regarding printing out your variables with production environments. These variables should only be used to help debug issues and used as a temporary measure.

I hope this helps
Doug

Hi Dough,

my variables are setup correctly so I wonder if I’m using the configuration features incorrectly?

Here’s my setup that is not working:

Regards
Kenth Jonsson

Hello again,

I’ve been able to get my configuration features working so that at least the transformation files are located without errors or warnings:

I’ve also added the Octopus debug print variables, scoped to the test environment where the app is being deployed:

But in the deploy log there are no variables printed:

So, in the web.config that has been created no variable substitution has been done:

We also have a Log4net config file that isn’t updated by variable substitution.

This is really strange, especially since the transformation file for Log4Net hasn’t been touched.
The last deploy to production occurred Dec 2nd with all variables substituted as expected.

What can be the problem?

Regards
Kenth Jonsson

Hello dough.pipe,

happy to inform you that I’ve got things working now.
Thanks for all your input.

Regards
Kenth Jonsson

Hi @kenth.jonsson,

This is great news that you now have this working and thank you for the update.

I hope you are able to continue with your deployments successfully.

All the best,
Doug