Variables intermittently not being replaced in runbook runs

Hi !

We are experiencing some weird new problems with variables sometimes not being properly replaced during runbook runs.

A few facts:

  • We are running the latest and greatest version 2021.2 (Build 7428)
  • We do have 3 workers running
  • We do have 2 spaces. The project in question was created when there was only 1 space.
  • The part of the variable not being properly replaced stems from a variable template in a variable set.
  • The runbook run ist tenanted.
  • The Runbook run is meant to be trigger by a scheduled trigger every couple of minutes
  • Most of the time when the runbook run was triggered by a scheduled trigger, the variable replacement fails. But sometimes it does work.
  • Most of the time when a failed runbook run is triggered again by a human (via ‘try again’), the variable replacement does work and the runbook runs to completion successfully. But sometime it doesn’t.

We are not sure when the problem started. It could have emerged this morning after the update to version 2021.2 (Build 7428), but we have no proof.

We don’t know how to tackle this one. Please help.

Thanks! David

Hey David,

Thanks for reaching out and for all of the information.

We do have another somewhat similar report from earlier today but it is based on Tenant variables: Runbooks fail to evaluate tenant variables after the first environment when scheduled to run in parallel to different environments · Issue #7042 · OctopusDeploy/Issues · GitHub. That’s not necessarily saying it’s exclusive to them, that is just what we were able to reproduce as that’s what the first case was reporting an issue with.

I assume you meant “isn’t” here?

Can you please tell me where the problem variable resides(just confirming the variable is in a library variable set and attached to the project), how it is scoped, and how it’s used within the process? Please let me know if there’s anything else that differs with the linked issue and what you’re experiencing. If you’d like to send me screenshots of everything in a DM, that would be likely help our reproduction efforts.

Looking forward to hearing back.

Best,
Jeremy

Hi!
Any news on this one?
We use this feature for many tasks which run on a daily, hourly basis or even shorter intervals.
Our current workaround is to have one powershell script with a collection of tenant variables and loop over this collection in order to process a specific task for every tenant.

But your feature to have a single trigger with a set of about 20 tenants and let octopus retrieve each tenats variables is currently useless.
The loop works well for tasks with just 1 variable like the tenant name, but we have some database tasks which require more than just one parameter to be substituted.

We need to know if we should develop our own C# solution outside OctopusDeploy (we don’t want to go this route) or if you are going to solve this issue soon.

Kind regards
Michael

Hey Michael,

I’ve added your report to the GH issue and I will reach out internally and pass along your message.

Please subscribe to the issue for any updates, but I will update you if the engineers have anything for me to pass back.

Best,
Jeremy

1 Like

Hey Michael,

I’ve heard back from our engineers on this.

Unfortunately, this isnt something they’re actively working on yet. They suggest continuing using the workaround for now. In your case, since the workaround isn’t great for you, it may be worth it to develop the C# script you referenced earlier if that’s not a huge undertaking.

I’m sorry I didn’t have better news for you but please feel free to let me know if you have any other questions or concerns.

Best,
Jeremy

Hi Jeremy,
being able to use variables in a trigger in a runbook is a key feature, don’t you think?
This features ist just broken and should be fixed.

It worked well with 2021.1.7316 and stopped with 7428.
Kind regards
Michael MĂźller

Hey Michael,

I’ll pass along your feedback to the developers.

I will let you know if they have any feedback for me to pass back.

Best,
Jeremy

Hey Michael,

I actually went to update the internal discussion on it and see that it is already going to be investigated this week.

Keep an eye on the GitHub issues for status updates or comments from our developers.

Please let me know if you have any other questions or concerns.

Best,
Jeremy

Hey Michael,

The fix for this should be going out next week if all goes to plan, so please keep an eye on that GitHub issue to see which version you can upgrade to for the fix.

Please let me know if you have any other questions or concerns.

Best,
Jeremy

Hey Michael,

Just to give you a head’s up, the fix has been merged and will be in the following versions or later:

Release stream Release
2021.2 2021.2.7783
2021.3 2021.3.8013

Please let me know if you end up upgrading and the issue doesn’t go away.

Best,
Jeremy

Hi Jeremy,
we installed the update to v2021.2 Build 7808 without problem.
We activated the original triggers which were not using the work around and they work again as intended.

Thank you for this fix and your support.

Kind regards
Michael

Hey Michael,

You’re very welcome and thank you for updating me that you’re in a good state.

I hope you have a great rest of your week.

Best,
Jeremy

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.