Updating a Step Template in a Process fails silently

I have a Step Template that is used for several steps in a number of Processes. The processes are stored in a Git repository.

I’ve updated the Step Template, and I want to use the new version in my Processes. When I go to a Step in a Process I see there is an “Update and Commit” button. When I click the button it does some work and then the button is re-enabled and no update has occurred (there is no commit to the git repository).

Looking at the browser dev tools I can see a 500 response to a request:
URL: https://howells.octopus.app/api/Spaces-1/actiontemplates/ActionTemplates-6/actionsUpdate
Method: POST
Status: 500 Internal Server Error
Response:

    {
        "ErrorMessage": "Stack empty."
    }

For Info: We’re using Octopus cloud v2022.3 (Build 5513-hotfix.7091)

Update: I’ve just manually re-created the Process step and noticed that there is a commit to migrate the Octopus Schema from 3 → 4.

I then tried the “Update and Commit” button on another step in the same process and now get a different error. The new error appears in the UI in a red banner at the top:
"An unexpected error occurred in Octopus v2022.3.5513-hotfix.7091: e is undefined

Update: There is a also a Request that returns a 400 error:
URL: https://howells.octopus.app/api/Spaces-1/actiontemplates/ActionTemplates-6/actionsUpdate
Method: POST
Status: 400 Bad Request
Response

{
  "ErrorMessage": "There was a problem with your request.",
  "Errors": [
    "Remote repository returned an error: unpacking the sent packfile failed on the remote"
  ],
  "ParsedHelpLinks": [],
  "Details": {}
}

Is this related to the Git Repository?

Hi @rik.crompton,

Thanks for reaching out and sorry to hear that you’re experiencing issues.

We’ve had a few issues recently revolving around step templates in config as code enabled projects, however, as far as I can see all of these have been resolved on the version that you’re on.

Can I ask if you have the option to update the step templates in those projects via the Library → Step Template page for that particular template, or is it forced through the deployment process?

Are you able to make manual git commits to the project repo from the Octopus Server? To rule out any networking issues at all?

Did this begin happening after an update or was there any changes made before this issue started?

Hopefully we can get to the bottom of this rather quickly and get you unblocked ASAP.

Kind Regards,
Adam

As far as I can see there is no option to do the update from the Step Template editor page (app#/Spaces-1/library/steptemplates/ActionTemplates-6/usage)

Not sure how to do that? I’m using Octopus Cloud so don’t have access to the server directly. is there something in the Octopus App UI where I can do this?

I haven’t made updates to Octopus for a while, so unfortunately there’s nothing obvious happened that I could point to and say “it started happening after X”. A colleague has suggested he had similar issues on or around 25th July 2022 if that helps?

Hi @rik.crompton,

Apologies, I didn’t realise you were using our cloud offering!

In that case, would you mind if I created a support user on your instance to have a look around, just to see if I can spot anything obvious?

As far as the upgrades are concerned, we do these automatically during your maintenance window when new builds pass testing, so I will take a look to see when the last upgrade happened on your instance and see if that correlates with any of the issues.

If it’s alright with you then once I’ve had a look around your instance I’ll get back in touch on this thread with further information, please do let me know if you’re alright with me creating a support user.

Kind Regards,
Adam

Go ahead :slight_smile:

For Info: I’ll be offline for an hour or so shortly - Thanks for looking into this so quickly :slight_smile:

1 Like

One more thing: I’ve been making changes on a branch called “17390_CheckForSqlConnectionLeak”. Each process has a separate repository behind it, but the branches have the same name

Hey @rik.crompton,

Thanks for your patience here!

Would you be able to confirm which project you’re currently testing and receiving this issue on as well as whether it would be okay for me to attempt to update the step template within that project, please?

Kind Regards,
Adam

Yep. You can see the issue on the SIP project. Look at the SQL - Ensure Login - Azure step.

happy for you to try the update.
Note: I’m working on the 17390_CheckForSqlConnectionLeak branch

Something else that may be worth noting here, I can see that in the project SHA under the branch you mentioned, ActionTemplates-6 is used in the deployment process, but inside the usage for ActionTemplates-6 there’s no reference to that branch.

I wonder if the case here is that some sort of link between that template being added and Octopus recognising the addition has gone wrong.

Would you be able to try removing the step completely from that branch, then re-adding the step template and filling in the configuration?
Perhaps that then allows you to save the commit, it’s just a theory though, of course.

I see what you mean (I think).
If I look at the usages for ActionTemplates-6[1] however, It looks like these usages are related to a release rather than a process. I’ve not created a release from my new branch yet, so I guess it won’t show up here?

[1] Octopus Deploy

I’ve actually already done this and it works (see Step 2 - SQL - Ensure database - Azure)

I’ve logged this help request as this template is used in lots of places and I was hoping the Update process would work for me. Drop/Recreate is really laborious :-/

I believe I may have to bring this up with our engineers, as I can’t see any quick way to rectify the issue as it stands and I’d rather get some information from the people behind the code who can maybe shed some more light on what’s going on.

I’ll let them know of the issue right now and hopefully have some feedback for Monday morning (they’re based in Australia so have already clocked off for the weekend).

In the meantime, I’d recommend that for any projects you need running with that new version to re-create the specific step within the branch, I really do apologise there isn’t a more convenient way at the moment but hopefully our developers will be able to find a more elegant solution.
Ultimately, it will help other users and yourself to have our engineers look over this, as it will mean if there is an underlying issue with Octopus, we can get it fixed ASAP.

Feel free to reach out in this thread if you have any more information to provide or if you have any additional queries.

Kind Regards,
Adam

Thanks for your help with this @adam.hollow !!

1 Like

Hey @rik.crompton,

Thank you for your patience here, I’ve had feedback from the developers on this one.

The error you’ve received, unpacking the sent packfile failed on the remote, seems to be from the Git provider, mostly based around permission issues.
Our devs noticed you mentioned you have a different repo for each project, could it be the case that the necessary permissions were not applied to that repo?
Are you able to make commits and change the deployment process of that project within Octopus, other than updating the step template?
Would you be able to let us know which Git provider you’re using?

Is the error above still the error you’re experiencing when trying to update step templates?
Let me know what you think of the above and I’ll pass on any more information I can.

Kind Regards,
Adam