Running: Octopus Deploy 2.6.5.1010
We have an environment with 6 machines defined. 3 of these machines are defined a role, RedisSentinel. I have a variable set with a variable named “RedisSentinel”. The value for the variable is:
#{each machine in Octopus.Environment.MachinesInRole[RedisSentinel]}#{machine}:#{RedisSentinelPort}#{unless Last},#{/unless}#{/each}
and we are trying to figure out why it is not getting substituted.
RedisSentinelPort is also defined in the variable set and has a value 12345
We have a project defined to use the variable set in question.
The project targets a number of machine roles, of which the servers marked as RedisSentinel roles are included by them being included in another role.
The Configuration variables checkbox for Replace appSettings and connectionString entries in any .config files is checked.
We have defined Substitute variables in files. The file specified has the following defined:
When the deployment runs the file ends up with the following:
I have downloaded the Octostache project and added a test that performs the same function and it seems to work there.
I have the 2 variables set and the raw output looks like the following:
15:09:03 Verbose | 97 files were extracted from the package
15:09:03 Verbose | Initial variable evaluation performed.
15:09:03 Verbose | The following variables are available:
…
| - [Octopus.Environment.MachinesInRole[RedisSentinel]] = ‘machines-33,machines-65,machines-66’
…
| - [RedisSentinel] = ‘#{each machine in Octopus.Environment.MachinesInRole[RedisSentinel]}#{machine}:#{RedisSentinelPort}#{unless Last},#{/unless}#{/each}’
| - [RedisSentinelPort] = ‘12345’
…
15:09:03 Verbose | Variables have been fully evaluated.
15:09:03 Verbose | The following evaluated variables are available:
…
| - [Octopus.Environment.MachinesInRole[RedisSentinel]] = ‘machines-33,machines-65,machines-66’
| - [OctopusUseGuidedFailure] = ‘False’
| - [RedisSentinel] = ‘#{each machine in Octopus.Environment.MachinesInRole[RedisSentinel]}#{machine}:56399#{unless Last},#{/unless}#{/each}’
| - [RedisSentinelPort] = ‘12345’
Hi Robert,
Thanks for getting in touch!
The short answer is you should only need the “Replace appSettings and connectionString entries” feature turned on - you shouldn’t need the transformation. The appSettings/connectionStrings feature looks for appSettings keys that match your variable names and sets their values directly.
Can you turn off the configuration transformation and see how that goes?
Hope that helps!
Damo
Hi Damian,
So to provide more details, we have the transformation turned on as we have settings that are configured using variables OUTSIDE of the appSettings and connectionString settings (things like log4net logging directories for listeners). Right now we have about 10 different environments and have been using 1 config transform per environment. We are trying to get to a point where we can have only the web.config in the project (with values for local development and local testing) and a Web.Octo.config transform file and then when OctoDeploy deploys it to an environment, we apply the appropriate variable replacements in Web.Octo.config and then do the transform using that configuration file as the source (hopefully this makes sense).
We are trying to get out of having every one of our solutions having to have more than the web.config and a web.octo.config (which is a transform file with variable substitutions in it). The deployment will actually figure out the variable values and set up the web.config as appropriate. In this way, we can ensure that our production servers, logins, and passwords are only available to the admins that need to know what they are. Currently they are in every config transform in every solution.
If you need any further information, please feel free to ask.
Thanks for the quick response,
Rob Steele
Hi Robert,
Thanks for that, I think I better understand what’s happening.
The problem may be that the variable replacement happens before the configuration transforms. That means Octopus will be replacing variables in your web.config file, and then transforming it with your environment-specific transform - which has #{} style variables set in it - which haven’t been replaced. In other words, it will write over the top of the changes.
To get this to work, you’ll need to do the variable replacement on your transform files first. I’ve recently written a how-to document on how this works (we get this scenario a lot).
I hope this helps!
Damo
I’m new to Octopus and I’ve been unable to get variable substitution to work.
The log message says it processed the file:
Performing variable substitution on ‘C:\Octopus\Applications\UAT\MyApp\1.0.10_1\Configs\myconfig.config’
However, when I open the file it still just has the replacement tokens. I added an XML comment just to try to emit some random variables.
Thoughts?
Thanks!
one other thing - variable substitution seems to be working in files with different extensions. For instance, I created a yml file and just shoved some arbitrary variables in it and it worked fine. It’s only the files with the .config extension that seem to be failing.
Hi Chris,
Are you able to send some through the full deployment log? It might also be useful running through the steps to debug problems with variables.
Thanks,
Damo
In the effort to pull together a reply, I found the problem. One of my variables had a missing ‘}’ character.
This apparently prevented the variable substitution from happening in the file at all. While I can see that this was my error, I wish that an error had been reported by Octopus, or that other variables in the file had been substituted.
Is there a place to request more detailed output during variable substitution?
Thanks!
Hi Chris,
Oh great, I’m glad you found the issue!
Providing more information about what has gone wrong is a bit tricky. Ultimately we look for strings starting with #{ and ending with }, try to evaluate them, and leave everything alone if they can’t be substituted. We do our best to substitute, but if it doesn’t make sense, we assume you know what you’re doing (and maybe that you needed the #{something} in place for the current step).
That said, we could provide a warning (or more likely an info message) if we find a match but no suitable substitution. For the above reasons, we wouldn’t want to fail it. I’ve created an issue in GitHub for it, but it’s unlikely to be a high priority.
Thanks,
Damo
Damian,
I think what would probably work best is if Octopus performed variable substitution on the variables that were formatted correctly. The missing ‘}’ was in the middle of the file. There were variables defined before & after that were formatted correctly. If octopus had substituted those, that would have been a visual cue of what the problem was. Instead, octopus substituted nothing in the file.
I also think this would be easier to implement than warnings about incomplete syntax.
Thanks,
Chris McKenzie
Oh, and I’d submit a PR, but I’m not sure where the code is that actually rewrites the files.
Hi Chris,
I think you’d have the same problem then anyway - if there were variables defined after, the matching closing } would have been at the end of the next variable, so very hard to decide what a “correctly formatted variable” is. But yes, I definitely take your point about it substituting nothing - It would be much better if it substituted what it could and left the rest alone.
If you wanted to submit a PR, that’d be awesome! The code is mainly in the Octostache repository. You’re probably looking for the TemplateParser class that’s ultimately responsible for the variable substitution. The entry point is in Calamari in the FileSubstituter class.
Thanks,
Damian
I looked into editing this code but the parsing exception is happening in Sprache. Octostache just check “WasSuccessful” on the parsing result and does nothing in the false case.
While the parsing result does contain very good information about why parsing failed, it does not appear that Octostache has a means to inject information into the deployment log. Choosing a shared logging framework through Calamari & Octostache is an architectural decision that exceeds the bounds of what what I’d normally put in a PR. I did however document my findings on the github issue you created.
Thanks,
Chris McKenzie
Hi Chris,
Thanks for that, that’s really useful information. We’ll have a look at it with the issue. It’s likely we’ll just pass the exception back and log it from Calamari.
Thanks,
Damo