Substring variable filter issue

simmotommo · 19 April 2021 14:57

Octopus Version: 2020.2.16

We use the following format for our Octopus “PackageVersion” variable:

0.1.<build_number>-<branch_name>

Occasionally a developer will create a branch with a long name and this will trip up on the restrictions on Docker secret names which we configure via Octopus variable substitution, e.g.

secrets:
  ca_certificate.pem:
    file: ./certs/ca_certificate.pem
    name: ca-certificate-pem-#{Octopus.Action.Package[our-package].PackageVersion}

If the branch name is too long this will result in an error similar to this:

failed to create secret ca-certificate-pem-0.1.51702-really-really-long-branch-name-1234: Error response from daemon: rpc error: code = InvalidArgument desc = invalid name, only 64 [a-zA-Z0-9-_.] characters allowed, and the start and end character must be [a-zA-Z0-9]
April 19th 2021 22:09:50Fatal
The remote script failed with exit code 1
April 19th 2021 22:09:50Fatal
The action scs-deploy-swarm on docker.host.local failed

We obviously ask our people to keep the branch names short but I’d like the deployments to be robust enough to handle this when it happens. I’m trying to shorten the variable using the Substring filter like so:

secrets:
  ca_certificate.pem:
    file: ./certs/ca_certificate.pem
    name: ca-certificate-pem-#{Octopus.Action.Package[sour_package].PackageVersion | Substring 33}

But this does not seem to have any effect, the resulting variable substitution is identical to the output without the Substring filter. Should this be working?

charles.hague · 19 April 2021 15:16

Hi simmotommo,

Thank you for reaching out to us with your query.

It’s possible that this is a fluke of how the Substring filter works. You might want to try using the Truncate filter instead:

If that doesn’t work could you please try one of the simple filters (e.g. ToUpper) as a test to see if that gets parsed correctly?

Best Regards,

Charles

simmotommo · 19 April 2021 16:24

Hi Charles,

I originally tried Truncate but it substitutes an ellipse (…) for the truncated text, this violates the “start and end character must be [a-zA-Z0-9]” part of the Docker Secrets naming rules.

Kenneth_Bates · 20 April 2021 00:05

Hi Simmotommo,

Thanks for keeping in touch! I gave this a test, and I’m seeing the same behavior. However I’ve noticed that by instead specifying the action name (the step that deployed the package), and not the package ID, the variable evaluates correctly and the Substring filter correctly shortens the version. Using your sample, could you try the following instead (substituting the PackageStepName with your step name)?

secrets:
  ca_certificate.pem:
    file: ./certs/ca_certificate.pem
    name: ca-certificate-pem-#{Octopus.Action[PackageStepName].Package.PackageVersion | Substring 33}

I hope this helps! Let me know how you go or if you have any further questions going forward.

Best regards,

Kenny

simmotommo · 20 April 2021 04:45

Hi Kenneth,

Thanks for chiming in. At first I your solution looked promising but having tested it a little I’ve uncovered some odd behaviour exhibited by Substring. Essentially, for me at least, it behaves as expected for some character lengths but not for others.

For example this configuration behaved as expected:

Package 'our_package' was not acquired or does not require staging 
April 20th 2021 11:09:00Info
Release Version: 0.1.51862-mission-command-150288-feature-restrict-deployment-to-releases-or-release-candidates 
April 20th 2021 11:09:00Info
Deployment Directory: /deploy/octopus 
April 20th 2021 11:09:00Info
Octopus Environment: env_01
April 20th 2021 11:09:00Info
Beginning Release-0.1.51862-mission-command-150288-feature-restrict-deployment-to-releases-or-release-candidates deployment 
April 20th 2021 11:09:00Info
Creating secret wildcard-cert-crt-0.1.51862-mission-command-150288- 
April 20th 2021 11:09:00Error
failed to create secret wildcard-fmglops-crt-0.1.51862-mission-command-150288-: Error response from daemon: rpc error: code = InvalidArgument desc = invalid name, only 64 [a-zA-Z0-9-_.] characters allowed, and the start and end character must be [a-zA-Z0-9]

This failed due to the trailing “-” so reducing the included characters should work.

But this is not the case:

Package 'our_package' was not acquired or does not require staging 
April 20th 2021 11:12:06Info
Release Version: 0.1.51866-mission-command-150288-feature-restrict-deployment-to-releases-or-release-candidates 
April 20th 2021 11:12:06Info
Deployment Directory: /deploy/octopus 
April 20th 2021 11:12:06Info
Octopus Environment:env_01 
April 20th 2021 11:12:06Info
Beginning Release-0.1.51866-mission-command-150288-feature-restrict-deployment-to-releases-or-release-candidates deployment 
April 20th 2021 11:12:06Info
Creating secret client-key-p12-0.1.51866-mission-command-150288-feature-restrict-deployment-to-releases-or-release-candidates 
April 20th 2021 11:12:06Error
failed to create secret client-key-p12-0.1.51866-mission-command-150288-feature-restrict-deployment-to-releases-or-release-candidates: Error response from daemon: rpc error: code = InvalidArgument desc = invalid name, only 64 [a-zA-Z0-9-_.] characters allowed, and the start and end character must be [a-zA-Z0-9]

So reducing the number of included characters from 33 to 32 causes Substring to have no effect.

I have done further testing with 31 characters, again with 33 characters, 34 characters and again with 33 characters. In these tests both 31 and 34 characters produced no effect, the first instance of 33 characters produced the same effect in the first example above while the second instance of 33 characters produced no effect.

This is a little odd, I’d be happy to do further testing if you have any suggestions.

charles.hague · 20 April 2021 08:22

Hi simmotommo,

Thank you for getting back to us.

The behaviour of the Substring filter in your tests is indeed a little odd! I’ll need to do some more investigation into why it isn’t working as expected. You mentioned that it worked correctly for a length of 33 in one attempt but not in the next - could you please confirm that the PackageVersion that was being fed into the filter was the same?

Due to the complexity of the restrictions on the certificate name it might also be worth considering an alternative solution with more flexibility than filters. This alternative solution would be adding a “Run A Script” step which creates a valid certificate name and stores it as an output variable:

$PackageVersion = $OctopusParameters["Octopus.Action.Package[PackageNameHere].PackageVersion"]

$SecretName = $PackageVersion.subString(0, [System.Math]::Min(33, $PackageVersion.Length))
$SecretName = $SecretName.TrimEnd("-")

Set-OctopusVariable -name "SecretName" -value $SecretName

You would then use SecretName as the variable to be replaced:

secrets:
  ca_certificate.pem:
    file: ./certs/ca_certificate.pem
    name: ca-certificate-pem-#{Octopus.Action[ScriptStepNameHere].Output.SecretName}

Would this be an option for you?

Best Regards,

Charles

simmotommo · 22 April 2021 01:10

Hi Charles,

Our Release Version format is “0.1. <build_nember>-<branch_name>”, e.g. “0.1.51866-mission-command-150288-feature-restrict-deployment-to-releases-or-release-candidates”, with the <build_number> being passed from the Azure DevOps pipeline. All of my tests were conducted on builds from the same branch so the Release Versions would only have differed in the <build_number> component and even then the number of digits in the <build_number> would have been 5 in all cases.

Your alternative solution looks interesting, I will try it out and let you know how I go.

The behaviour of Substring is definitely not as expected, I’d be interested to see what you discover. Thanks for looking at it and please let me know if you need any further information.

charles.hague · 23 April 2021 10:32

Hi simmotommo,

Thank you for providing those details.

We’ve been trying to reproduce this issue and I’m afraid we’ve not been able to do so - the Substring filter appears to work correctly in all of our tests. Our best suggestion at this point would be to try upgrading Octopus Deploy to the latest version as various improvements and fixes have been made and it’s possible one of them may resolve your issue. In the meantime I hope that the script suggestion above will turn out to be a reliable solution.

Could you please let me know how the script works out and if you’re able to try the filter after upgrading?

Best Regards,

Charles

system · 24 May 2021 10:33

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.