Variable substation is not taking place on files in my package when I use the Deploy an Azure Resource Manager template.
Is this expected, and if so how can I request that this feature be added?
The long version
I’m using the Deploy an Azure Resource Manager template step to successfully deploy an API Management API and several Named Values to Azure. My ‘Template Source’ for this is ‘File inside a package’.
In the same package as the ARM template and it’s associated parameters file, I have several XML Policy files, some of which contain Octopus variable substitution place holders. These files are uploaded to Blob Storage using a deployment script, which should run after variable substitution.
Here is an example of what I’m doing in the XML Policy files - <value>{{#{Octopus.Environment.Name}_HeaderValue}}</value>
After the XML Policy files have been uploaded to Blob Storage, that same script also sets an Octopus variable using the Set-OctopusVariable PowerShell Cmdlet, and that is used by the aforementioned ARM template to link to my XML Policy files at deployment time.
The deployment is successful, but variable substitution is not occurring in the XML Policy files.
There is no configurable feature to allow me to specify files in which variable substitution should take place, so I was expecting it to automatically occur on all files in the package.
Is the behavior that I have described expected, and if so how can I request that this feature be added?
I would recommend splitting these into two different steps.
First step, an Azure Powershell script, in this step you reference the policies package, and enable the file substitution (have a look at https://octopus.com/docs/deployment-examples/custom-scripts/standalone-scripts#accessing-package-references-from-a-custom-script to reference the correct extracted package paths when configuring the target files), you can then use the PS Azure cmdlets (which are included by default) to upload these policy files to blob storage.
Finally the second step is pretty much what you have now, the ARM template step that references the output variables from the first step.
Another option would be to use a Terraform step, obviously this option depends on how familiar you are with Terraform.
Octopus Deploy Command Line Tool, version 4.13.14
Handshaking with Octopus server: https://myserver.myurl.co.uk/api
Handshake successful. Octopus version: 2018.8.6; API version: 3.0.0
Authenticated as: TFS Auto Deploy User <myuser@myurl.co.uk>
This Octopus Server supports channels
Finding project: Raster Mapping Service API Management
Automatically selecting the best channel for this release...
Building a release plan for Channel 'Default'...
Finding deployment process...
Finding release template...
Selected the release plan for Channel 'Default' - it is a perfect match
Using version number provided on command-line: 2.0.2-named-values-parameter-fix
Release plan for Raster Mapping Service API Management 2.0.2-named-values-parameter-fix
Channel: 'Default' (this is the default channel)
# Name Version Source Version rules
--- --------------------------------------- ---------------------------------- ---------------- -------------------
1 Upload Swagger and Policies 2.0.2-named-values-parameter-fix User specified Allow any version
2 Deploy Named Values to API Management 2.0.2-named-values-parameter-fix User specified Allow any version
3 Deploy API to API Management 2.0.2-named-values-parameter-fix User specified Allow any version
Checking for existing release for Raster Mapping Service API Management 2.0.2-named-values-parameter-fix because you specified --ignoreexisting...
No release exists - the coast is clear!
Creating release...
There was a problem with your request.
- No package version was specified for Upload Swagger and Policies:Landmark.RasterMappingService.ApiManagement.AzureDeploy
Error from Octopus server (HTTP 400 BadRequest)
Exit code: -7
I have tried to edit the command to explicitly specify a package for that package reference by adding --package "Upload Swagger and Policies:Landmark.RasterMappingService.ApiManagement.AzureDeploy:$VERSION", but that also fails, albeit with a different error -
The version portion of the package constraint ‘Upload Swagger and Policies:Landmark.RasterMappingService.ApiManagement.AzureDeploy:2.0.2-named-values-parameter-fix’ is not a valid semantic version number.
I’ve updated the Octo CLI, and I can now create a release, so thanks for that.
This does seem to have solved my original issue, although it did throw up one unexpected problem that took a little while to figure out.
Alongside the variable substitution, we run a script to upload various XML policy files and a swagger file to a Storage Account. We then use Set-OctopusVariable to set two variables containing links and other information for consumption by the ARM template. However, because these variables are now created in a different step, simply referencing their variable names didn’t work, and I instead had to reference them as output variables.
It’s all documented, it just took a while to figure it out, and is another reason why I’d like to see additional configurable features added to the ‘Deploy an Azure Resource Manager template’ step in future versions.
I am glad you have it all working.
Yes, output variables need to be fully qualified between steps.
Regarding adding more features to 'Deploy an Azure Resource Manager template’ step, we like to keep steps simple and doing just one thing and one thing only. We prefer the use of multiple steps to perform multiple distinct actions, it makes it more composable and also easier to maintain.