I’m having trouble with re-deployment of packages and I could not find clear information on how it is supposed to work:
Octopus 2019.6.8. LTS
Problem:
a package from an external NuGet feed is no longer present on the feed (for reference: FesadEssenceTransferService release 3.0.17)
the version was deployed successfully earlier this year
the retention policy of the lifecycle is at least 10 releases (no date!) and I can see the package in question in the tentacles cache of the server I want to deploy to (also in the correct environment folder):
during redeployment Octopus states it is checking the cache but then tries to download the package from the feed, which fails.
I was under the assumption that Octopus would use a locally cached package in the tentacle cache and only download a package when it is not available in the cache. Especially with this deployment option:
Any ideas why this isn’t working? Can we force Ocotpus to prefer the tentacle cache over (re)downloading?
Edit: The more I am reading the more I suspect that the behavior I am looking for is not implemented. Can you give suggestions what we can do to avoid losing packages from ext. feeds that are currently installed?
Thanks for getting in touch. There are two download and cache steps taking place during a deployment. First, the Octopus Server will connect to the NuGet feed and download the package to the Octopus Server cache folder. Then, during deployment, this package will be downloaded to the Tentacle cache folder. At each of these steps, Octopus will check to see if the package already exists in the cache folder before downloading and skip it if it is found.
The situation you’re in sounds like the package exists in the Tentacle cache but is missing from the Octopus Server cache, which is causing it to attempt to download from the feed. This download is necessary so that the server can compare the package against what is located in the Tentacle cache to confirm whether replacement is required.
The Octopus Server will only retain cached packages for 20 days by default.
This value can be increased using the command line tool. However, I’m not sure if that would be a long term solution for you as there is no way to configure it only to keep packages attached to a release.
Another option would be to alter how your build pipeline works and have production builds pushed to the built-in Octopus repository where the retention policy will only remove packages that are not attached to a release. I do realise that this isn’t likely to be a small piece of work for you but may offer the best all-round result.
A final option would be to amend how your external repository retention policy works, possibly adding some logic that queries the Octopus API to check what packages are in use and skip deletion of them.
Not sure if any of these options are going to help you in the short term, but hopefully it provides some useful direction for you.
thank you for your detailed response, this cleared up my false assumptions.
Since we are also deploying packages from external NuGet feeds that cannot directly access our Octopus instance, using the internal feed isn’t really an option.
After a bit of discussion we are opting for increasing the server’s cache to 365 days. We are not deploying so many different products that this should be a space or quantity issue, but I guess we will find out :-).
Are there any plans to also link the server cache with retention policies that are tied to current releases? I guess this would be a nice feature, because I think it is not necessarily ideal that for getting proper retention you have to maintain a NuGet feed and also push directly to Octopus.
In any case thanks for your input, much appreciated.
Enjoy the weekend,
Chris
Having spoken with our engineers regarding how retention is handled for external feeds, they have advised that there is currently no plan to expand the retention policy to include the server cache.
The view is that this scenario would be best handled at the external repository side. It is possible to query the Octopus API to retrieve a list of packages that are currently tied to a release; this list could then be excluded from any cleanup taking place in your external repository. Example Script.
We do have a user voice page where you can raise this request though, and if it begins to gain additional support, then it would be reconsidered.
thank you for following up on this. We changed the package cache on the Octopus server to hold packages for longer. If this
doesn’t work out we will implement a logic in our external repository using the script example you provided.
sorry to re-open this but we’ve just noticed that the change in package cache isn’t properly working.
I’ve changed the package cache to 365 days (storage is plenty available) and restarted the serivce November 5th. Looking at my cache, I can clearly see that only packages from the past 20 days are kept on the filesystem of the server.
Any idea why this might be happening? The server configuration clearly shows the changed value.
Anything I did wrong potentially? Is this a bug maybe? My understanding was that changing the configuration value in Octopus Server will cache ALL packages longer, including those from external sources.
I’ve spent some time this morning testing this and there does seem to be something odd occurring at times.
On my first test, after setting the packagecache to 365 days and restarting the service, I was able to move my server date forwards in time by months and the packagecache did not clear.
After resetting the test and doing the exact same steps again, I saw packages being removed after 20 days until I performed a second service restart.
I’m going to continue looking at this to see if there is a clear cause for this, but for now I’d recommend that you perform another restart of the Octopus Server service and see if that works for you too.
I just wanted to report back. The cache still contains packages from January 8th so it seems like the second restart did the trick. I hope it stays that way after another restart. We’d appreciate if you could investigate further and let us know if this is some sort of bug or whether we have to expect deletion of the older cached packages when the service is restarted again.
Glad to hear that it looks to be working. The additional testing that I did on this didn’t show any instances of the setting rolling back after it starts working.
