JSON Variable replacement problem

Hi there !

We are currently evaluating Octupus deploy for our deployment needs.
We are using it on an ASP.NET Core RC1 application and thus we use JSON configuration files.

Now so far everything seems to go smoothly but we are running into a problem with json variable replacements.

The config file in question looks like this:

{
“StorageType”: “local”,
“ThreadCount”: null,
“Queues”: [“smarttagging”,“leafletflightcompilation”,“purging”],

“Integration”: {
“StorageType”: “redis”,
“ThreadCount”: 1
},
“Testing”: {
“StorageType”: “redis”,
“ThreadCount”: 1
},
“Production”: {
“StorageType”: “redis”,
“ThreadCount”: 0
}
}

Now on deployment we want to change the value of Queues to [“smarttagging”] only and the value of
Production:ThreadCount to 8

Basically this works, however all values written to the file by Octupus are always saved as json string values.

Here are our variables:
Production:ThreadCount Value: 8
Queues Value: [“smarttagging”]

We end up with this final config file after replacements:

{
“StorageType”: “local”,
“ThreadCount”: null,
“Queues”: “[“smarttagging”]”,
“Integration”: {
“StorageType”: “redis”,
“ThreadCount”: 1
},
“Testing”: {
“StorageType”: “redis”,
“ThreadCount”: 1
},
“Production”: {
“StorageType”: “redis”,
“ThreadCount”: “8”
}
}

Which is obviously not the desired end result…

Are we doing something wrong ? Is this a bug ? Or is there some way around this ?

PS: We are using Octupus 3.3.4

Regards
Bernhard Glück

Hi Bernhard,

Thanks for reaching out and kudos for providing all the info needed on your first reply.

You probably entered an extra set of double quotes. The JSON file should look as shown below, and the variable has to be set as shown on the attached screenshot.

EDIT: This approach uses the “substitute variables in files” feature and NOT the “JSON Configuration Variables” one, see why in this other comment

{ 
  "StorageType": "local", 
  "ThreadCount": null, 
  "Queues": [#{Queues}],

  "Integration": { 
    "StorageType": "redis", 
    "ThreadCount": 1 
  }, 
  "Testing": { 
    "StorageType": "redis", 
    "ThreadCount": 1 
  }, 
  "Production": { 
    "StorageType": "redis", 
    "ThreadCount": #{ProdThreadCount} 
  } 
}

Cheers!
Dalmiro

Hi !

First of all thanks for your quick reply and tips !

However we are running now into more problems with your tip.
We changed our json config file to basically the exact same format as you suggested.
However the json configuration replacer seems to load the file as json ( of course :slight_smile: )
and thus fails since the file is not valid json anymore ( e.g [#{Queues}] is not valid json obviously…

Here is the exception:
Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: #. Path ‘Queues’, line 4, position 16.
22:39:02Error
at Newtonsoft.Json.JsonTextReader.ParseValue()
22:39:02Error
at Newtonsoft.Json.JsonTextReader.ReadInternal()
22:39:02Error
at Newtonsoft.Json.JsonTextReader.Read()
22:39:02Error
at Newtonsoft.Json.Linq.JContainer.ReadContentFrom(JsonReader r)
22:39:02Error
at Newtonsoft.Json.Linq.JContainer.ReadTokenFrom(JsonReader reader)
22:39:02Error
at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader)
22:39:02Error
at Calamari.Integration.JsonVariables.JsonConfigurationVariableReplacer.LoadJson(String jsonFilePath) in Y:\work\14ffc968155e4956\source\Calamari\Integration\JsonVariables\JsonConfigurationVariableReplacer.cs:line 44
22:39:02Error
at Calamari.Integration.JsonVariables.JsonConfigurationVariableReplacer.ModifyJsonFile(String jsonFilePath, VariableDictionary variables) in Y:\work\14ffc968155e4956\source\Calamari\Integration\JsonVariables\JsonConfigurationVariableReplacer.cs:line 17
22:39:02Error
at Calamari.Deployment.Conventions.JsonConfigurationVariablesConvention.Install(RunningDeployment deployment) in Y:\work\14ffc968155e4956\source\Calamari\Deployment\Conventions\JsonConfigurationVariablesConvention.cs:line 44
22:39:02Error
at Calamari.Deployment.ConventionProcessor.RunInstallConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 60
22:39:02Error
at Calamari.Deployment.ConventionProcessor.RunConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 28
22:39:02Error
Running rollback conventions…
22:39:02Error
Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: #. Path ‘Queues’, line 4, position 16.
22:39:02Error
at Newtonsoft.Json.JsonTextReader.ParseValue()
22:39:02Error
at Newtonsoft.Json.JsonTextReader.ReadInternal()
22:39:02Error
at Newtonsoft.Json.JsonTextReader.Read()
22:39:02Error
at Newtonsoft.Json.Linq.JContainer.ReadContentFrom(JsonReader r)
22:39:02Error
at Newtonsoft.Json.Linq.JContainer.ReadTokenFrom(JsonReader reader)
22:39:02Error
at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader)
22:39:02Error
at Calamari.Integration.JsonVariables.JsonConfigurationVariableReplacer.LoadJson(String jsonFilePath) in Y:\work\14ffc968155e4956\source\Calamari\Integration\JsonVariables\JsonConfigurationVariableReplacer.cs:line 44
22:39:02Error
at Calamari.Integration.JsonVariables.JsonConfigurationVariableReplacer.ModifyJsonFile(String jsonFilePath, VariableDictionary variables) in Y:\work\14ffc968155e4956\source\Calamari\Integration\JsonVariables\JsonConfigurationVariableReplacer.cs:line 17
22:39:02Error
at Calamari.Deployment.Conventions.JsonConfigurationVariablesConvention.Install(RunningDeployment deployment) in Y:\work\14ffc968155e4956\source\Calamari\Deployment\Conventions\JsonConfigurationVariablesConvention.cs:line 44
22:39:02Error
at Calamari.Deployment.ConventionProcessor.RunInstallConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 60
22:39:02Error
at Calamari.Deployment.ConventionProcessor.RunConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 50
22:39:02Error
at Calamari.Commands.DeployPackageCommand.Execute(String[] commandLineArguments) in Y:\work\14ffc968155e4956\source\Calamari\Commands\DeployPackageCommand.cs:line 113
22:39:02Error
at Calamari.Program.Execute(String[] args) in Y:\work\14ffc968155e4956\source\Calamari\Program.cs:line 38
22:39:02Fatal
The remote script failed with exit code 100

I fixed this by handling the json files as normal text files with substitution variables…
So my problem is solved with a workaround.

However i think the current behaviour of the json value replacer should be evaluated since this makes it highly unusable.

Hi,

Sorry, I absolutely forgot to mention that the approach described on my first reply was using the “substitute variables in files” feature. I’ve edited my initial reply to reflect this for anyone else that runs into this thread.

As you correctly stated, currently the “JSON Configuration Variables” feature doesn’t handle arrays. It’s something we are aware of and that we plan on improving.

Best regards,
Dalmiro

Just as a note if you track this internally, it also does not handle any non string type of value…

Arrays,Objects,numeric values and booleans are all quoted as strings in the final file.

Adding a note for that as well. Thanks for the feedback!