Iterate over all variables in a template

[OctopusSchaff]

I am needing to add all the project variables as environment variables to my container. To that end I am overriding the values.yaml file in my Update a Helm Chart step.

But I need a way to iterate over all the variables in the project. I looked at this help file: Variable substitutions - Octopus Deploy. While helpful, I could not see a way to get a list of all the variables in my project. Basically I am looking for something like this:

... other stuff from the values.yaml removed for brevity
Env:
#{each variable in AllProjectVariables]
  #{variable.Name}: #{variable.Value}
#{/each}

So if I had the following variables in my project:

Name                                   Value                           Scope
AppSettings__Environment               Dev
AppSettings__LogginUrl                 https://MyLoggingUrl.com

Then the template would render as:

... again, other stuff from values.yaml removed for brevity
Env:
  AppSettings__Environment: Dev 
  AppSettings__LogginUrl: https://MyLoggingUrl.com

The issues with the above are:

1. I made up the AllProjectVariables variable. I need a variable to iterate over that represents all the variables scoped to the project (both variable sets and project variables),
2. I don’t know how to get the name and value of the variable once I am iterating through it. (Is it as easy as variable.name and variable.value?

How can I do this using the template syntax of Octopus Deploy?

[OctopusSchaff]

I got this going, but it took a bit of work.

The first hurdle was getting a list of the variables. Octopus Deploy has a list of ALL the variables in the $OctopusParameters variable. (And I do mean ALL.) Depending on when in the process you write them out, there can be several hundred. So I had to decide how to filter it down. I opted to put something on each of my variables so I could be sure I got only the ones that I wanted. So, the variables that I plan to be made Environment Variables in my container all start with Env. (ie Env.AppSettings__Environment). Variables that came from a variable set got reassigned:

Name                                   Value                           Scope
Env.VariableNameFromSet                #{VariableNameFromSet}

In my Update a Helm Chart step, I added a pre-deployment script (added via the Configure Features button). My pre-deployment script looks like this:

$index = 0
# Get all the Octopus variables that start with Env.
$envKeys = $OctopusParameters.Keys | where {$_ -like "Env.*"}
# Iterate over all the matching keys
foreach($envKey in $envKeys) 
{	        
    # Store the key (with the "Env." trimmed off) and the value in a yaml compliant key value pair.
    $containerEnvironmentVariable = $envKey.TrimStart("Env.") + ': ' + $OctopusParameters[$envKey]    
    # Get an array style name for the variable
    $varName = "ContainerEnvironmentVariables[$index]"
    # Save the variable as an Octopus Variable
    Set-OctopusVariable -name $varName -value $containerEnvironmentVariable
    $index = $index + 1
}

The second main hurdle was getting Octopus to let me have an array of variables. The type of all variable values in Octopus is String. The trick is in the naming of the variable. We end up calling the variables ContainerEnvironmentVariables[0], ContainerEnvironmentVariables[1], ContainerEnvironmentVariables[2] etc.

Using that naming style gets Octopus Deploy to treat the ContainerEnvironmentVariables variable as an array that can be iterated over (though the values of each of the elements of the array are still strings).

With this in place, now we can get values.yaml can list out the contents of our ContainerEnvironmentVariables variable.

But we can’t just use #{ContainerEnvironmentVariables} directly. We need to control the spacing of the insertion. For that we are going to use Octopus Deploy’s repetition #{each var in vars} syntax. Here is what that part of my values.yaml looks like:

env:
#{each envVar in ContainerEnvironmentVariables}  #{envVar}
#{/each}

This sets up a loop over all the ContainerEnvironmentVariables. As it loops over each entry in the array, it inserts two spaces, the array entry variable value, then a return.

It is important to note that if you write this in a more traditional format like this:

env:
#{each envVar in ContainerEnvironmentVariables}
  #{envVar}
#{/each}

Then you will get an extra line before each variable (because the template system assumes you want all the newlines between the start and stop of the loop tags).

When run, I get output like this in my values.yaml file:

Env:
  AppSettings__Environment: Dev 
  AppSettings__LogginUrl: https://MyLoggingUrl.com

(As a side note, the use of the double __ is due to the fact that most Linux systems can’t support the use of colons : in environment variable names.)

[mark.reeder]

Hi @OctopusSchaff,

Glad to hear you got this working!

Another possibility for you is to use the Octostache templating either directly in the values.yaml or in the Raw Values YAML field on that Helm step. So in your values.yaml or in that Raw Values YAML field for the Env section you could have:

Env:
  AppSettings__Environment: #{Project.Environment.Name}
  AppSettings__LogginUrl: #{Project.LoggingUrl}

Then you could set your variables and scope them accordingly. Those steps would replace the values with how they are scoped. I’m not sure if this would help in your use-case, but another option to consider.

Nice to see you got a solution to your situation!

[OctopusSchaff]

Thank you for the suggestion. Normally what you showed would be an excellent solution. However, my Helm charts are a bit generic in nature. I use a template that the application is based off of.

Because of that, I need the Octopus steps to input all the environment variable names from the list of variables in the Octopus project.