Octopus doesn't substitute variables in the package

I have a package which has a script in it. That script will look for all JSON files and process them. Before that, I need Octopus to substitute the variables in the JSON files which have the format like:

{
    "name": "#{MyVariable}"
}

and then they are ready for my script. Assume that I have defined MyVariable in the project.

Here is the step configuration:

When I look at the working folder which is C:\Octopus\Work\20201211124219-286528-241, I can see the contents of the JSON files are not touched.

Can you please help me figure out the problem?
Do I need to add the same package again in Additional Packages?

Hi Hossein.Margani,

Thank you for reaching out to us with your query.

The two different types of variable replacement feature available in Octopus Deploy are structured configuration variables and substitute variables in templates. The example JSON code you have posted suggests that it is the latter option that you need. It might be worth disabling the former for clarity if you aren’t using it.

The most likely explanation for why the substitutions aren’t happening as expected is that the value provided for “Target Files” isn’t correct. Could you please try changing the “Target Files” value to explcitiy point to each JSON file relative to the installation directory (e.g. path/to/file.json)? If that doesn’t resolve the issue could you please check the task log and advise if any warnings are shown regarding no suitable files being found for variable substitution?

Best Regards,

Charles

Thank you Charles.

I removed the former feature, and added the relative path to the json file, but Octopus still isn’t substituting the variables. Here is the deployment log:

                    |     == Failed: Worker ==
13:23:53   Verbose  |       Octopus Server version: 2020.4.0+Branch.tags-2020.4.0.Sha.6fcc5868ffcdca3e5d2bb1a2071964dfa43d218a
13:23:53   Verbose  |       Environment Information:
                    |       IsRunningInContainer: True
                    |       OperatingSystem: Linux 5.4.0-1040-azure #42~18.04.1-Ubuntu SMP Mon Feb 8 19:05:32 UTC 2021
                    |       OsBitVersion: x64
                    |       Is64BitProcess: True
                    |       CurrentUser: root
                    |       MachineName: octopusdeploy-***
                    |       ProcessorCount: 2
                    |       CurrentDirectory: /Octopus
                    |       TempDirectory: /tmp/
                    |       HostProcessName: Octopus.Server
                    |       PID: 1
13:23:53   Verbose  |       Executing Validate Legacy Deployment (type Run an Azure Script) on nl-ams-build03
13:23:53   Verbose  |       Using account ID 'azureserviceprincipal-azure-local-***'
13:23:53   Verbose  |       Using Calamari.AzureScripting.netfx 8.3.1
13:23:53   Verbose  |       Using Calamari.AzureScripting.linux-x64 8.3.1
13:23:53   Verbose  |       Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory 'C:\Octopus\Work\20210421132353-306497-263' using 'OEM United States' encoding running as 'NT AUTHORITY\SYSTEM' with the same environment variables as the launching process
13:23:53   Verbose  |       Calamari Version: 1.0.0
13:23:53   Verbose  |       Environment Information:
13:23:53   Verbose  |       OperatingSystem: Microsoft Windows NT 6.2.9200.0
13:23:53   Verbose  |       OsBitVersion: x64
13:23:53   Verbose  |       Is64BitProcess: True
13:23:53   Verbose  |       CurrentUser: NT AUTHORITY\SYSTEM
13:23:53   Verbose  |       MachineName: NL-AMS-BUILD03
13:23:53   Verbose  |       ProcessorCount: 4
13:23:53   Verbose  |       CurrentDirectory: C:\Octopus\Work\20210421132353-306497-263
13:23:53   Verbose  |       TempDirectory: C:\Windows\TEMP\
13:23:53   Verbose  |       HostProcess: Calamari.AzureScripting (6168)
13:23:54   Verbose  |       Extracting package to: C:\Octopus\Work\20210421132353-306497-263\staging
13:23:54   Verbose  |       Extracted 32 files
13:23:54   Info     |       Extracting package 'C:\Octopus\Files\***@44778BC7E89999438187CA47EF7F42B6.nupkg' to 'C:\Octopus\Work\20210421132353-306497-263\staging'
13:23:54   Verbose  |       Performing variable substitution on 'C:\Octopus\Work\20210421132353-306497-263\staging\tests\cd-test-legacy.ps1'
13:23:54   Verbose  |       Executing 'C:\Octopus\Work\20210421132353-306497-263\staging\tests/cd-test-legacy.ps1'
13:23:54   Verbose  |       Setting Proxy Environment Variables
13:23:55   Verbose  |       Name                           Value
13:23:55   Verbose  |       ----                           -----
13:23:55   Verbose  |       PSVersion                      5.1.14409.1018
13:23:55   Verbose  |       PSEdition                      Desktop
13:23:55   Verbose  |       PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
13:23:55   Verbose  |       BuildVersion                   10.0.14409.1018
13:23:55   Verbose  |       CLRVersion                     4.0.30319.42000
13:23:55   Verbose  |       WSManStackVersion              3.0
13:23:55   Verbose  |       PSRemotingProtocolVersion      2.3
13:23:55   Verbose  |       SerializationVersion           1.1.0.1
13:23:55   Verbose  |       PowerShell Environment Information:
13:23:55   Verbose  |       OperatingSystem: Microsoft Windows NT 6.3.9600.0
13:23:55   Verbose  |       OsBitVersion: x64
13:23:55   Verbose  |       Is64BitProcess: True
13:23:55   Verbose  |       CurrentUser: NT AUTHORITY\SYSTEM
13:23:55   Verbose  |       MachineName: NL-AMS-BUILD03
13:23:55   Verbose  |       ProcessorCount: 4
13:23:55   Verbose  |       CurrentDirectory: C:\Octopus\Work\20210421132353-306497-263\staging\tests
13:23:55   Verbose  |       CurrentLocation: C:\Octopus\Work\20210421132353-306497-263\staging\tests
13:23:55   Verbose  |       TempDirectory: C:\Windows\TEMP\
13:23:55   Verbose  |       HostProcess: powershell (5588)
13:23:55   Verbose  |       TotalPhysicalMemory: 9436660 KB
13:23:55   Verbose  |       AvailablePhysicalMemory: 5699424 KB
13:24:15   Info     |       Mode             : Process
13:24:15   Info     |       ContextDirectory :
13:24:15   Info     |       ContextFile      :
13:24:15   Info     |       CacheDirectory   :
13:24:15   Info     |       CacheFile        :
13:24:15   Info     |       Settings         : {}
13:24:17   Verbose  |       Az Modules: Authenticating with Service Principal
13:24:17   Verbose  |       Environments : {[AzureChinaCloud, AzureChinaCloud], [AzureCloud, AzureCloud],
13:24:17   Verbose  |       [AzureGermanCloud, AzureGermanCloud], [AzureUSGovernment,
13:24:17   Verbose  |       AzureUSGovernment]}
13:24:17   Verbose  |       Context      : Microsoft.Azure.Commands.Profile.Models.Core.PSAzureContext
13:24:20   Verbose  |       Azure CLI: Authenticating with Service Principal
13:24:20   Verbose  |       [
								***
13:09:38   Verbose  |       ]
13:24:20   Verbose  |       Azure CLI: Setting active subscription to ***
13:24:24   Verbose  |       Successfully authenticated with the Azure CLI
13:24:24   Verbose  |       Invoking target script C:\Octopus\Work\20210421132353-306497-263\staging\tests\Octopus.FunctionAppenderContext.ps1 with  parameters
13:24:24   Verbose  |       Invoking target script C:\Octopus\Work\20210421132353-306497-263\staging\tests/cd-test-legacy.ps1 with -ResourceGroupName *** -EnvironmentName *** parameters.
13:24:24   Info     |       Template file path: C:\Octopus\Work\20210421132353-306497-263\staging\src\legacy\Non-Production\azuredeploy.json
13:24:24   Info     |       Template parameters file path: C:\Octopus\Work\20210421132353-306497-263\staging\src\legacy\Non-Production\azuredeploy.parameters.json
13:24:24   Info     |       Code    : InvalidTemplate
13:24:24   Info     |       Message : Deployment template validation failed: 'The provided value
13:24:24   Info     |       '#{AzureServiceBusPricingTier}' for the template parameter
13:24:24   Info     |       'serviceBusPricingTier' at line '26' and column '28' is not valid.
13:24:24   Info     |       The parameter value is not part of the allowed value(s):
13:24:24   Info     |       'Basic,Standard,Premium'.'.
13:24:24   Info     |       Details :
13:24:24   Error    |       OperationStopped: [C:\Octopus\Work\20210421132353-306497-263\staging\src\legacy\Non-Production] Template is NOT Valid
13:24:24   Error    |       At C:\Octopus\Work\20210421132353-306497-263\staging\tests\helpers\deployment-validation-helper.psm1:23 char:13
13:24:24   Error    |       +             throw "[$TemplatePath] Template is NOT Valid"
13:24:24   Error    |       +             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13:24:24   Error    |       at Confirm-Deployment<Process>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\helpers\deployment-validation-helper.psm1: line 23
13:24:24   Error    |       at Test-LegacyARMDeployment<Process>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\cd-test-legacy.ps1: line 46
13:24:24   Error    |       at <ScriptBlock>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\cd-test-legacy.ps1: line 56
13:24:24   Error    |       at <ScriptBlock>, <No file>: line 1
13:24:24   Error    |       at <ScriptBlock>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\Octopus.FunctionAppenderContext.ps1: line 116
13:24:24   Error    |       at <ScriptBlock>, <No file>: line 1
13:24:24   Error    |       at <ScriptBlock>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\Octopus.AzureContext.ps1: line 190
13:24:24   Error    |       at <ScriptBlock>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\Bootstrap.Octopus.AzureContext.ps1: line 3312
13:24:24   Error    |       at <ScriptBlock>, <No file>: line 1
13:24:24   Error    |       at <ScriptBlock>, <No file>: line 1
13:24:24   Verbose  |       Adding journal entry:
13:24:24   Verbose  |       <Deployment Id="***" EnvironmentId="Environments-421" TenantId="" ProjectId="Projects-1501" InstalledOn="2021-04-21 13:24:23" RetentionPolicySet="Environments-421/Projects-1501/Step-Validate Legacy Deployment/#{Octopus.Machine.Id}/&lt;default&gt;" ExtractedTo="C:\Octopus\Work\20210421132353-306497-263\staging" CustomInstallationDirectory="" WasSuccessful="False">
13:24:24   Verbose  |       <Package PackageId="***" PackageVersion="1.0.0.743-***" DeployedFrom="C:\Octopus\Files\***@44778BC7E89999438187CA47EF7F42B6.nupkg" />
13:24:24   Verbose  |       </Deployment>
13:24:24   Error    |       Running rollback behaviours...
13:24:24   Error    |       Script returned non-zero exit code: 1.
13:24:24   Verbose  |       Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Octopus\Work\20210421132353-306497-263 exited with code 1
13:24:24   Verbose  |       Updating manifest with output variables
13:24:24   Verbose  |       Updating manifest with action evaluated variables
13:24:24   Fatal    |       The remote script failed with exit code 1
13:24:24   Fatal    |       The action Validate Legacy Deployment on a Worker failed

Hi Hossein.Margani,

Thank you for getting back to me.

The log seems to show that no attempt was made to perform the substitution. There should be a line containing the following text for each specified file: Performing variable substitution on [...]. However, this is not the case. There are also no errors indicating a missing file or a permission issue.

Can you please confirm that you created and deployed a new release after making the changes to the step? Could you please also provide a screenshot showing the relevant part of the “Substitute variables in templates” configuration?

Best Regards,

Charles

Thanks Charles.

There is a line for ps1 substitution in the log I sent:

Performing variable substitution on 'C:\Octopus\Work\20210421132353-306497-263\staging\tests\cd-test-legacy.ps1'

In the screenshot I sent, I showed that the config is:

**\*.json

Then I changed it to: src\legacy\path-of-json-file.json

And yes I created a new release.

Here is the screenshot:

The path to two files are relative to the nuget package in the artifact.

Hi Hossein.Margani,

Thank you for getting back to me.

I can see from the log that the full path of one of the files in question is as follows:

C:\Octopus\Work\20210421132353-306497-263\staging\src\legacy\Non-Production\azuredeploy.parameters.json

However, the screenshot shows that the path of the file to run variable replacement on is given as:

src\legacy\Non-Production\azuredeploy.parameters.json

Could you please try updating the variable replacement configuration so the paths include the ‘staging’ directory as well and let me know if that makes a difference?

e.g. staging\src\legacy\Non-Production\azuredeploy.parameters.json

If the above doesn’t help could you please send me a full task log and your process JSON as attachments? You can either upload them here (let me know if you’d like the thread made private) or you can send them to support@octopus.com.

Best Regards,

Charles

Thanks. No it didn’t help.

I just sent an email.

Hi Hossein.Margani,

Thank you for providing the task log and deployment process.

It appears that you are being affected by a known issue with some older versions of Octopus Deploy. You can read more about the issue below:

This issue should be fixed if you upgrade to a newer version than 2020.4.1. There are also some workarounds described in the issue that might help if an upgrade isn’t an option yet, although we’d strongly recommend that you upgrade if possible.

I hope this is helpful. Please let me know if you have any questions.

Best Regards,

Charles

Hi Charles,

Yes I upgraded the Octopus to the latest version and it’s now substituting the variables in JSON files using **\*.json expression.

Thank you.

I appreciate your help.

1 Like