We are running version 2020.6.5163 and are experiencing an issue whenever we try to push or manually upload a new¨ package to octopus, we have no issues when uploading new versions of an already existing package. We have 2 Nodes using a NAS as shared storage.
After investigation it seems that the directory on shared storage isn’t created whenever a new package is uploaded. Manually creating this directory or creating it using the script console/Run on Octopus Server solves the issue, this last option seem to rule out permissions issues.
When uploading via the web interface we get a access denied error, as it tries to access a non existent directory:
Access to the path ‘\nas-xxx.xxx\share$\Octopus\Packages\Spaces-1\feeds-builtin\a.dummy.package’ is denied.
Thanks for reaching out, sorry to hear that you’re having an issue pushing new packages here.
This isn’t something I’ve encountered before, and I’d like to investigate further. Has this worked for you previously?
As you’re running two nodes with NAS as shared storage, I’d start by confirming everything is configured correctly with your shared folder. If you haven’t seen it already, we have some documentation on configuring HA, specifically shared storage, here.
Yes our setup previously worked fine, it only started after updating octopus (or so it seems, we don’t create new packages daily so no way of knowing for sure). At first I was thinking it was a permission issue. But everything seems fine; checked it by creating folders just fine on the NAS while impersonating the Octopus service user. It’s always possible something changed on the NAS end (I’m not managing it), but I would assume the service user (by script or script console) wouldn’t be able to create these folders if that’s the case.
Thanks for confirming you’ve checked that out already.
It’s interesting that it happened after upgrading Octopus; there aren’t any changes I’m aware of that would affect shared storage.
Could you check the Octopus server is still running as the intended service account, please? It may be this was reset during the upgrade process, which could explain the sudden issues.
Just jumping in for Stuart here, as he’s finished for the day - you mention in your first email that the referenced folder does not exist. This is generally governed by the nugetRepository config directive, which tells Octopus where to put the files for the Built-in feed. Is it possible this directory was moved or renamed, which could be leading to this happening?
Whenever a brand new package, first ever version is uploaded Octopus creates a sub folder named after the package name to store all future versions of that package. This is the folder that is missing.
Thanks for your patience while we looked into this for you.
I attempted to reproduce the error that you’re seeing, and I ran into the same issue. I created a network share and pointed my Octopus instance to use that share as the packages directory. I was able to create/delete folders manually, but it would fail with the same message as yours when attempting to upload via the UI or command line. I did this both on the version of Octopus you’re running and the most recent version; the results were the same on both machines.
Everything appeared to be set up correctly with my network share, so I then changed the service account Octopus was running as, and I was then able to push packages as expected. In my scenario, I changed the service user to run as .\Administrator.
With the above in mind, I think some permissions may have changed, which is causing your package pushes to fail when running as the current service account.
Would you mind letting me know how you get on after digging into the permissions/accounts, please?
Good to know, thank you for looking into it. I’ll raise the question with the team managing the NAS.
What I don’t understand is when I impersonate the service user I can access and create folders just fine so I always assumed it wasn’t a permission issue.
I’ll get back to you once I get a hold of someone who can look at the permissions a bit closer