Variable Substitution in the 'Deploy an Azure Resource Manager template' step


(David Gard) #1

Hi,

The short version

Variable substation is not taking place on files in my package when I use the Deploy an Azure Resource Manager template.

Is this expected, and if so how can I request that this feature be added?

The long version

I’m using the Deploy an Azure Resource Manager template step to successfully deploy an API Management API and several Named Values to Azure. My ‘Template Source’ for this is ‘File inside a package’.

In the same package as the ARM template and it’s associated parameters file, I have several XML Policy files, some of which contain Octopus variable substitution place holders. These files are uploaded to Blob Storage using a deployment script, which should run after variable substitution.

Here is an example of what I’m doing in the XML Policy files -
<value>{{#{Octopus.Environment.Name}_HeaderValue}}</value>

After the XML Policy files have been uploaded to Blob Storage, that same script also sets an Octopus variable using the Set-OctopusVariable PowerShell Cmdlet, and that is used by the aforementioned ARM template to link to my XML Policy files at deployment time.

The deployment is successful, but variable substitution is not occurring in the XML Policy files.

There is no configurable feature to allow me to specify files in which variable substitution should take place, so I was expecting it to automatically occur on all files in the package.

Is the behavior that I have described expected, and if so how can I request that this feature be added?

Thanks,
David


(John Simons) #2

Hi David,

Thanks for reaching out to us.

So given you want to do two things:

  • Upload files to blob storage
  • Deploy an Azure RM template

I would recommend splitting these into two different steps.

First step, an Azure Powershell script, in this step you reference the policies package, and enable the file substitution (have a look at https://octopus.com/docs/deployment-examples/custom-scripts/standalone-scripts#accessing-package-references-from-a-custom-script to reference the correct extracted package paths when configuring the target files), you can then use the PS Azure cmdlets (which are included by default) to upload these policy files to blob storage.
Finally the second step is pretty much what you have now, the ARM template step that references the output variables from the first step.

Another option would be to use a Terraform step, obviously this option depends on how familiar you are with Terraform.

Let me know how you go.

Regards
John


(David Gard) #3

Hi John,

Thanks for your reply.

I’m trying to implement your suggested change, but I’m running in to a problem using Referenced Packages.

When trying to create an Octopus release via octo.exe, I’m getting an error -

No package version was specified for Upload Swagger and Policies:my.package.name.here

Here is the command that is generating the error -

/opt/OctopusTools.4.13.14.ubuntu.16.04-x64/Octo create-release --server=$bamboo_OctopusUrl --project="$bamboo_OctopusProject" --apikey=$bamboo_OctopusApiKey --packageversion $VERSION --releaseNumber $VERSION --ignoreexisting --force

Here is the full output from the command -

Octopus Deploy Command Line Tool, version 4.13.14
Handshaking with Octopus server: https://myserver.myurl.co.uk/api
Handshake successful. Octopus version: 2018.8.6; API version: 3.0.0
Authenticated as: TFS Auto Deploy User <myuser@myurl.co.uk> 
This Octopus Server supports channels
Finding project: Raster Mapping Service API Management
Automatically selecting the best channel for this release...
Building a release plan for Channel 'Default'...
Finding deployment process...
Finding release template...
Selected the release plan for Channel 'Default' - it is a perfect match
Using version number provided on command-line: 2.0.2-named-values-parameter-fix
Release plan for Raster Mapping Service API Management 2.0.2-named-values-parameter-fix
Channel: 'Default' (this is the default channel)
  #   Name                                    Version                            Source           Version rules      
  --- --------------------------------------- ---------------------------------- ---------------- -------------------
  1   Upload Swagger and Policies             2.0.2-named-values-parameter-fix   User specified   Allow any version  
  2   Deploy Named Values to API Management   2.0.2-named-values-parameter-fix   User specified   Allow any version  
  3   Deploy API to API Management            2.0.2-named-values-parameter-fix   User specified   Allow any version  

Checking for existing release for Raster Mapping Service API Management 2.0.2-named-values-parameter-fix because you specified --ignoreexisting...
No release exists - the coast is clear!
Creating release...
There was a problem with your request.

 - No package version was specified for Upload Swagger and Policies:Landmark.RasterMappingService.ApiManagement.AzureDeploy

Error from Octopus server (HTTP 400 BadRequest)
Exit code: -7

I have tried to edit the command to explicitly specify a package for that package reference by adding --package "Upload Swagger and Policies:Landmark.RasterMappingService.ApiManagement.AzureDeploy:$VERSION", but that also fails, albeit with a different error -

The version portion of the package constraint ‘Upload Swagger and Policies:Landmark.RasterMappingService.ApiManagement.AzureDeploy:2.0.2-named-values-parameter-fix’ is not a valid semantic version number.


(John Simons) #4

Hi David,

Sorry that you are running into this bug with Octo, this was an issue in v4.13.14 (https://github.com/OctopusDeploy/Issues/issues/5003), to fix this, just install the latest version of Octo (https://octopus.com/downloads).
Once you have updated, the command you are running should work.

BTW, there are a few options to specify the package version, if you are interested have a read about it in https://octopus.com/docs/api-and-integration/octo.exe-command-line/create-release#Creatingreleases-Specifyingthepackageversion

Let me know how this goes.

Regards
John


(David Gard) #5

Hi John,

Thanks again for your reply.

I’ve updated the Octo CLI, and I can now create a release, so thanks for that.

This does seem to have solved my original issue, although it did throw up one unexpected problem that took a little while to figure out.

Alongside the variable substitution, we run a script to upload various XML policy files and a swagger file to a Storage Account. We then use Set-OctopusVariable to set two variables containing links and other information for consumption by the ARM template. However, because these variables are now created in a different step, simply referencing their variable names didn’t work, and I instead had to reference them as output variables.

It’s all documented, it just took a while to figure it out, and is another reason why I’d like to see additional configurable features added to the ‘Deploy an Azure Resource Manager template’ step in future versions.

Thanks, for you help.

Thanks,
David


(John Simons) #6

Hi David,

I am glad you have it all working.
Yes, output variables need to be fully qualified between steps.

Regarding adding more features to 'Deploy an Azure Resource Manager template’ step, we like to keep steps simple and doing just one thing and one thing only. We prefer the use of multiple steps to perform multiple distinct actions, it makes it more composable and also easier to maintain.

Regards
John