Error updating script step template in version controller project

darryl.baker · 27 September 2022 13:55

Hi. I’m getting a 500 response when I click on the UPDATE AND COMMIT button on a project step for a project under source control.

I just updated to the latest version v2022.3 (Build 10594) just in case it has already been fixed, but to no avail. I got an error on the project screen because I’d deleted one of the branches from BitBucket that this project’s repository had - I don’t know if that’s related to this error.

2022-09-27 14:43:37.0989   2772     26 ERROR  Unhandled error on request: "Stack empty."
System.InvalidOperationException: Stack empty.
   at System.Collections.Generic.Stack`1.ThrowForEmptyStack()
   at Octopus.Core.Features.Projects.ProjectScope.GetProjectPath() in ./source/Octopus.Core/Features/Projects/ProjectScope.cs:line 37
   at Octopus.Core.Git.Schema.GitSchemaVersionProvider.LoadProject(CancellationToken cancellationToken) in ./source/Octopus.Core/Git/Schema/GitSchemaVersionProvider.cs:line 132
   at Octopus.Core.Git.Schema.GitSchemaVersionProvider.GetProjectLatestSchemaVersion(CancellationToken cancellationToken) in ./source/Octopus.Core/Git/Schema/GitSchemaVersionProvider.cs:line 126
   at Octopus.Core.Git.Schema.GitSchemaVersionProvider.LoadSchemaVersions(String verifiedFileNameOrNull, CancellationToken cancellationToken) in ./source/Octopus.Core/Git/Schema/GitSchemaVersionProvider.cs:line 102
   at Octopus.Core.Git.Schema.GitSchemaVersionProvider.LoadSchemaVersions(CancellationToken cancellationToken) in ./source/Octopus.Core/Git/Schema/GitSchemaVersionProvider.cs:line 64
   at Octopus.Core.Git.Transactions.GitDocumentTransaction.<>c__DisplayClass20_0.<<Commit>b__0>d.MoveNext() in ./source/Octopus.Core/Git/Transactions/GitDocumentTransaction.cs:line 116
--- End of stack trace from previous location ---
   at Octopus.Core.Git.Transactions.GitDocumentTransaction.MigrateBeforeExecute[T](Func`1 innerOperation, CancellationToken cancellationToken) in ./source/Octopus.Core/Git/Transactions/GitDocumentTransaction.cs:line 208
   at Octopus.Core.Git.Transactions.GitDocumentTransaction.Commit(String message, CommitAuthor author, CommitAuthor committer, CancellationToken cancellationToken) in ./source/Octopus.Core/Git/Transactions/GitDocumentTransaction.cs:line 107
   at Octopus.Core.Infrastructure.UnitsOfWork.UnitOfWork.CommitToGit(CancellationToken cancellationToken) in ./source/Octopus.Core/Infrastructure/UnitsOfWork/UnitOfWork.cs:line 177
   at Octopus.Core.Infrastructure.UnitsOfWork.UnitOfWork.Complete(CancellationToken cancellationToken) in ./source/Octopus.Core/Infrastructure/UnitsOfWork/UnitOfWork.cs:line 122
   at Octopus.NancyOwin.NancyMiddleware.RequestComplete(NancyContext context, IDictionary`2 environment, CancellationToken cancellationToken) in ./source/Octopus.Server/Owin/NancyMiddleware.cs:line 174
   at Octopus.NancyOwin.NancyMiddleware.RequestComplete(NancyContext context, IDictionary`2 environment, CancellationToken cancellationToken) in ./source/Octopus.Server/Owin/NancyMiddleware.cs:line 174
   at Octopus.NancyOwin.NancyMiddleware.<>c__DisplayClass3_0.<<UseNancy>b__1>d.MoveNext() in ./source/Octopus.Server/Owin/NancyMiddleware.cs:line 110
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Owin.WebSocketAcceptAdapter.<>c__DisplayClass6_0.<<AdaptWebSockets>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Octopus.Server.Web.UnitOfWorkMiddleware.InvokeAsync(HttpContext httpContext, IUnitOfWork unitOfWork) in ./source/Octopus.Server/Web/UnitOfWorkMiddleware.cs:line 47
   at Octopus.Server.Web.UnitOfWorkMiddleware.InvokeAsync(HttpContext httpContext, IUnitOfWork unitOfWork) in ./source/Octopus.Server/Web/UnitOfWorkMiddleware.cs:line 47
   at Octopus.Server.Web.Middleware.OctopusClientOldVersionWarningMiddleware.InvokeAsync(HttpContext context, IAutomationContext automationContext) in ./source/Octopus.Server/Web/Middleware/OctopusClientOldVersionWarningMiddleware.cs:line 53
   at Octopus.Server.Web.Middleware.DynamicContentHeadersMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/DynamicContentHeadersMiddleware.cs:line 49
   at Octopus.Server.Web.Middleware.PrivateSpaceToggleMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/PrivateSpaceToggleMiddleware.cs:line 56
   at Octopus.Server.Web.Middleware.MaintenanceModeMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/MaintenanceModeMiddleware.cs:line 61
   at Octopus.Server.Web.Middleware.OctopusAuthenticationMiddleware.InvokeAsync(HttpContext context, IUserAuthenticator userAuthenticator, IAuthCookieService authCookieService, IWebAuthCache authCache, ILogger logger) in ./source/Octopus.Server/Web/Middleware/OctopusAuthenticationMiddleware.cs:line 57
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Octopus.Server.Web.Middleware.LegacyRequestLoggerMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/LegacyRequestLoggerMiddleware.cs:line 42
   at Octopus.Server.Web.Middleware.TelemetryMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/TelemetryMiddleware.cs:line 76
   at Octopus.Server.Web.Middleware.ErrorHandlingMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/ErrorHandlingMiddleware.cs:line 98
2022-09-27 14:43:37.0989   2772     26  INFO  Request took 1256ms

On the user interface the button shows it’s trying to save then returns to the same as before I clicked it. Dev tools shows the 500 error from /api/Spaces-1/actiontemplates/ActionTemplates-801/actionsUpdate as

{
  "ErrorMessage": "Stack empty."
}

dan_close · 27 September 2022 15:01

Hi @darryl.baker,

Thanks for reaching out to Octopus Support, and I’m sorry to see you are having these issues with your version-controlled project.

It looks like you may be running into this open GitHub issue, which can occur when deleting a branch that your project was once connected to. The ticket does not explicitly reference updating step templates, but this could be another side effect.

If you have not already, can you attempt clearing the local Git cache on your server? When you browse to Configuration > Git > Clear Git Cache, you will see a button that should remove the Git cache from your server and rebuild it once you reaccess your project.

One thing to verify when doing this is that the folders on your Octopus server are removed during the cache clearing process. After pressing the button, the Git folder in the Octopus home directory (i.e., C:\Octopus\Git) should be empty. You can remove the folders manually and try the process again if the folders are not deleted.

Please let me know if this helps resolve your issue or if it continues to persist.

Thanks!
Dan

darryl.baker · 27 September 2022 21:28

Hi Dan. Thanks for that suggestion. Unfortunately after clearing the git cache the project fails to load completely.

It says:
There was an error accessing version control.

Unable to establish a connection to git.
Please check the URL, default branch name, and authentication details.

Path ‘C:\Octopus\Git\ZOHIPOVQFYL3QE437GPQ24PNZL3D5RGK\root’ doesn’t point at a valid Git repository or workdir.

The test button says it’s OK with the result “Version control settings verified successfully”

What do I need to do to get it to reload from git?

Thanks,
Darryl

darryl.baker · 27 September 2022 21:45

OK - I tried that from the Octopus web UI before remoting to the server and trying to manually empty the C:\Octopus\Git folder - I couldn’t remove the folder because it was in use. I stopped Octopus Server, deleted the folder and restarted it. My project is restored but the Update and Commit button is still having the same problem.

Unlike the linked GitHub issue, we only closed feature branches since this project was converted to source control (I used this one project as a test in case there were teething troubles). The project is still on the master branch and was able to reload all of the settings (once cleared the cache correctly).

As a workaround, I will try removing the out of date template steps and re-adding them (I only need to copy/paste the name and one parameter).

britton.riggs · 27 September 2022 22:05

Hi @darryl.baker,

I’m stepping in for Dan as he’s gone offline for the day, but I’m happy to help.

I’m sorry clearing the Git cache via the Octopus Server UI didn’t work for you, but I’m glad manually clearing the folder did the trick.

Let me know how it goes in testing the workaround you mentioned, and also if you run into any new errors and I’d be glad to take a deeper dive on this.

Best,

Britton

darryl.baker · 28 September 2022 06:08

Hi Britton.

For clarity, my workaround did the trick, clearing the git cache did not resolve the problem.

There were only two places using this step template so I just made a note of their settings, deleted the steps and re-added them. In this case it took about a minute.

If Dan hadn’t mentioned checking that the folder is empty on the Octopus server, I would have been completely stuck with a project that wouldn’t load after I cleared the cache. That extra bit of detail of was really helpful.

Thanks again,
Darryl

clare.martin · 28 September 2022 07:09

Hey @darryl.baker,

Jumping in for Dan and Britton as they are currently off shift as part of our US based team. Brilliant news you got your CaC project working again, thank you for sharing that manually clearing the Git Folder did the trick. Posting that in here will help some of our other customers who have the same issue.

If you need anything else in future please reach out as we love to help here at Octopus Support!

Kind Regards,

Clare

darryl.baker · 28 September 2022 13:49

Just an update - clearing the cache didn’t fix the 500 error, but after removing the steps based on the step template I was working on and manually re-adding them (to update their usage to the latest one) I was able to successfully update the usage automatically via the Octopus website UI.

Thanks again for the help. I’m sharing this detail in case it helps the developers in getting to the bottom of the bug.

Anyone else with the issue - workaround was just to manually re-add the step template step into the config-as-code project’s deployment steps.

system · 29 October 2022 13:49

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.