I have a package which has a script in it. That script will look for all JSON files and process them. Before that, I need Octopus to substitute the variables in the JSON files which have the format like:
{
"name": "#{MyVariable}"
}
and then they are ready for my script. Assume that I have defined MyVariable in the project.
The two different types of variable replacement feature available in Octopus Deploy are structured configuration variables and substitute variables in templates. The example JSON code you have posted suggests that it is the latter option that you need. It might be worth disabling the former for clarity if you arenât using it.
The most likely explanation for why the substitutions arenât happening as expected is that the value provided for âTarget Filesâ isnât correct. Could you please try changing the âTarget Filesâ value to explcitiy point to each JSON file relative to the installation directory (e.g. path/to/file.json)? If that doesnât resolve the issue could you please check the task log and advise if any warnings are shown regarding no suitable files being found for variable substitution?
I removed the former feature, and added the relative path to the json file, but Octopus still isnât substituting the variables. Here is the deployment log:
| == Failed: Worker ==
13:23:53 Verbose | Octopus Server version: 2020.4.0+Branch.tags-2020.4.0.Sha.6fcc5868ffcdca3e5d2bb1a2071964dfa43d218a
13:23:53 Verbose | Environment Information:
| IsRunningInContainer: True
| OperatingSystem: Linux 5.4.0-1040-azure #42~18.04.1-Ubuntu SMP Mon Feb 8 19:05:32 UTC 2021
| OsBitVersion: x64
| Is64BitProcess: True
| CurrentUser: root
| MachineName: octopusdeploy-***
| ProcessorCount: 2
| CurrentDirectory: /Octopus
| TempDirectory: /tmp/
| HostProcessName: Octopus.Server
| PID: 1
13:23:53 Verbose | Executing Validate Legacy Deployment (type Run an Azure Script) on nl-ams-build03
13:23:53 Verbose | Using account ID 'azureserviceprincipal-azure-local-***'
13:23:53 Verbose | Using Calamari.AzureScripting.netfx 8.3.1
13:23:53 Verbose | Using Calamari.AzureScripting.linux-x64 8.3.1
13:23:53 Verbose | Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory 'C:\Octopus\Work\20210421132353-306497-263' using 'OEM United States' encoding running as 'NT AUTHORITY\SYSTEM' with the same environment variables as the launching process
13:23:53 Verbose | Calamari Version: 1.0.0
13:23:53 Verbose | Environment Information:
13:23:53 Verbose | OperatingSystem: Microsoft Windows NT 6.2.9200.0
13:23:53 Verbose | OsBitVersion: x64
13:23:53 Verbose | Is64BitProcess: True
13:23:53 Verbose | CurrentUser: NT AUTHORITY\SYSTEM
13:23:53 Verbose | MachineName: NL-AMS-BUILD03
13:23:53 Verbose | ProcessorCount: 4
13:23:53 Verbose | CurrentDirectory: C:\Octopus\Work\20210421132353-306497-263
13:23:53 Verbose | TempDirectory: C:\Windows\TEMP\
13:23:53 Verbose | HostProcess: Calamari.AzureScripting (6168)
13:23:54 Verbose | Extracting package to: C:\Octopus\Work\20210421132353-306497-263\staging
13:23:54 Verbose | Extracted 32 files
13:23:54 Info | Extracting package 'C:\Octopus\Files\***@44778BC7E89999438187CA47EF7F42B6.nupkg' to 'C:\Octopus\Work\20210421132353-306497-263\staging'
13:23:54 Verbose | Performing variable substitution on 'C:\Octopus\Work\20210421132353-306497-263\staging\tests\cd-test-legacy.ps1'
13:23:54 Verbose | Executing 'C:\Octopus\Work\20210421132353-306497-263\staging\tests/cd-test-legacy.ps1'
13:23:54 Verbose | Setting Proxy Environment Variables
13:23:55 Verbose | Name Value
13:23:55 Verbose | ---- -----
13:23:55 Verbose | PSVersion 5.1.14409.1018
13:23:55 Verbose | PSEdition Desktop
13:23:55 Verbose | PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
13:23:55 Verbose | BuildVersion 10.0.14409.1018
13:23:55 Verbose | CLRVersion 4.0.30319.42000
13:23:55 Verbose | WSManStackVersion 3.0
13:23:55 Verbose | PSRemotingProtocolVersion 2.3
13:23:55 Verbose | SerializationVersion 1.1.0.1
13:23:55 Verbose | PowerShell Environment Information:
13:23:55 Verbose | OperatingSystem: Microsoft Windows NT 6.3.9600.0
13:23:55 Verbose | OsBitVersion: x64
13:23:55 Verbose | Is64BitProcess: True
13:23:55 Verbose | CurrentUser: NT AUTHORITY\SYSTEM
13:23:55 Verbose | MachineName: NL-AMS-BUILD03
13:23:55 Verbose | ProcessorCount: 4
13:23:55 Verbose | CurrentDirectory: C:\Octopus\Work\20210421132353-306497-263\staging\tests
13:23:55 Verbose | CurrentLocation: C:\Octopus\Work\20210421132353-306497-263\staging\tests
13:23:55 Verbose | TempDirectory: C:\Windows\TEMP\
13:23:55 Verbose | HostProcess: powershell (5588)
13:23:55 Verbose | TotalPhysicalMemory: 9436660 KB
13:23:55 Verbose | AvailablePhysicalMemory: 5699424 KB
13:24:15 Info | Mode : Process
13:24:15 Info | ContextDirectory :
13:24:15 Info | ContextFile :
13:24:15 Info | CacheDirectory :
13:24:15 Info | CacheFile :
13:24:15 Info | Settings : {}
13:24:17 Verbose | Az Modules: Authenticating with Service Principal
13:24:17 Verbose | Environments : {[AzureChinaCloud, AzureChinaCloud], [AzureCloud, AzureCloud],
13:24:17 Verbose | [AzureGermanCloud, AzureGermanCloud], [AzureUSGovernment,
13:24:17 Verbose | AzureUSGovernment]}
13:24:17 Verbose | Context : Microsoft.Azure.Commands.Profile.Models.Core.PSAzureContext
13:24:20 Verbose | Azure CLI: Authenticating with Service Principal
13:24:20 Verbose | [
***
13:09:38 Verbose | ]
13:24:20 Verbose | Azure CLI: Setting active subscription to ***
13:24:24 Verbose | Successfully authenticated with the Azure CLI
13:24:24 Verbose | Invoking target script C:\Octopus\Work\20210421132353-306497-263\staging\tests\Octopus.FunctionAppenderContext.ps1 with parameters
13:24:24 Verbose | Invoking target script C:\Octopus\Work\20210421132353-306497-263\staging\tests/cd-test-legacy.ps1 with -ResourceGroupName *** -EnvironmentName *** parameters.
13:24:24 Info | Template file path: C:\Octopus\Work\20210421132353-306497-263\staging\src\legacy\Non-Production\azuredeploy.json
13:24:24 Info | Template parameters file path: C:\Octopus\Work\20210421132353-306497-263\staging\src\legacy\Non-Production\azuredeploy.parameters.json
13:24:24 Info | Code : InvalidTemplate
13:24:24 Info | Message : Deployment template validation failed: 'The provided value
13:24:24 Info | '#{AzureServiceBusPricingTier}' for the template parameter
13:24:24 Info | 'serviceBusPricingTier' at line '26' and column '28' is not valid.
13:24:24 Info | The parameter value is not part of the allowed value(s):
13:24:24 Info | 'Basic,Standard,Premium'.'.
13:24:24 Info | Details :
13:24:24 Error | OperationStopped: [C:\Octopus\Work\20210421132353-306497-263\staging\src\legacy\Non-Production] Template is NOT Valid
13:24:24 Error | At C:\Octopus\Work\20210421132353-306497-263\staging\tests\helpers\deployment-validation-helper.psm1:23 char:13
13:24:24 Error | + throw "[$TemplatePath] Template is NOT Valid"
13:24:24 Error | + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13:24:24 Error | at Confirm-Deployment<Process>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\helpers\deployment-validation-helper.psm1: line 23
13:24:24 Error | at Test-LegacyARMDeployment<Process>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\cd-test-legacy.ps1: line 46
13:24:24 Error | at <ScriptBlock>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\cd-test-legacy.ps1: line 56
13:24:24 Error | at <ScriptBlock>, <No file>: line 1
13:24:24 Error | at <ScriptBlock>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\Octopus.FunctionAppenderContext.ps1: line 116
13:24:24 Error | at <ScriptBlock>, <No file>: line 1
13:24:24 Error | at <ScriptBlock>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\Octopus.AzureContext.ps1: line 190
13:24:24 Error | at <ScriptBlock>, C:\Octopus\Work\20210421132353-306497-263\staging\tests\Bootstrap.Octopus.AzureContext.ps1: line 3312
13:24:24 Error | at <ScriptBlock>, <No file>: line 1
13:24:24 Error | at <ScriptBlock>, <No file>: line 1
13:24:24 Verbose | Adding journal entry:
13:24:24 Verbose | <Deployment Id="***" EnvironmentId="Environments-421" TenantId="" ProjectId="Projects-1501" InstalledOn="2021-04-21 13:24:23" RetentionPolicySet="Environments-421/Projects-1501/Step-Validate Legacy Deployment/#{Octopus.Machine.Id}/<default>" ExtractedTo="C:\Octopus\Work\20210421132353-306497-263\staging" CustomInstallationDirectory="" WasSuccessful="False">
13:24:24 Verbose | <Package PackageId="***" PackageVersion="1.0.0.743-***" DeployedFrom="C:\Octopus\Files\***@44778BC7E89999438187CA47EF7F42B6.nupkg" />
13:24:24 Verbose | </Deployment>
13:24:24 Error | Running rollback behaviours...
13:24:24 Error | Script returned non-zero exit code: 1.
13:24:24 Verbose | Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Octopus\Work\20210421132353-306497-263 exited with code 1
13:24:24 Verbose | Updating manifest with output variables
13:24:24 Verbose | Updating manifest with action evaluated variables
13:24:24 Fatal | The remote script failed with exit code 1
13:24:24 Fatal | The action Validate Legacy Deployment on a Worker failed
The log seems to show that no attempt was made to perform the substitution. There should be a line containing the following text for each specified file: Performing variable substitution on [...]. However, this is not the case. There are also no errors indicating a missing file or a permission issue.
Can you please confirm that you created and deployed a new release after making the changes to the step? Could you please also provide a screenshot showing the relevant part of the âSubstitute variables in templatesâ configuration?
Could you please try updating the variable replacement configuration so the paths include the âstagingâ directory as well and let me know if that makes a difference?
e.g. staging\src\legacy\Non-Production\azuredeploy.parameters.json
If the above doesnât help could you please send me a full task log and your process JSON as attachments? You can either upload them here (let me know if youâd like the thread made private) or you can send them to support@octopus.com.
Thank you for providing the task log and deployment process.
It appears that you are being affected by a known issue with some older versions of Octopus Deploy. You can read more about the issue below:
This issue should be fixed if you upgrade to a newer version than 2020.4.1. There are also some workarounds described in the issue that might help if an upgrade isnât an option yet, although weâd strongly recommend that you upgrade if possible.
I hope this is helpful. Please let me know if you have any questions.
