I have an update from our engineers about retention on workers, they said package retention does run on workers but its not part of the retention policy.
The way it works is it will keep storing packages until the worker starts to run out of space. Once out of space an algorithm will run and it will remove the packages that are “least used” to make room for the new packages.
They mentioned your logs where you still have space for packages and that’s why no clean-up is happening:
10:34:33 Verbose | Detected enough space for new packages. (4.662 GB/9.745 GB)
I assume you have a third party program or script that runs on each server that warns you when disk space is below a certain threshold and those workers have hit that threshold, is that why you noticed this originally?
I have enquired if we can put something in our documentation which shows how retention on workers works as even I assumed we ran the retention policy on them because of the way our documentation specifies it working on tentacles (not specifically deployment targets / workers).
I ran some more tests to improve my understanding of this and on a windows worker with the same project as I was using for the Linux ones I do actually see the cache being cleared but its not where you think it would be in the logs as its not a ‘step’ that runs a policy. Its a process that runs on start of the deployment.
The spot in your logs where its showing you have enough space is where it would show the cleanup if you met the threshold criteria for disk space usage.
You can see my first deployment run below which cleans up the JhubSite package ending in BDBB:
Cleaning 222.302 GB space from the package cache.
Removing package file 'C:\Octopus\Main Computer Tentacle Worker\Files\JhubSite@S1.0.3@C4D282796B747840BA663F054B89BDBB.zip'
And if I re-run that deployment a different package gets removed (ending in B711C)
Cleaning 221.931 GB space from the package cache.
Removing package file 'C:\Octopus\Main Computer Tentacle Worker\Files\JhubSite@S1.0.3@C006024D803EB54581D1D8E01C7B711C.zip'
It looks like this is the code we run for the package retention so if it goes under 30% disk space that’s when it runs retention by the looks of the code. So, in theory, your servers can get to 1.949GB of free disk space before it runs the clean up with your 9.745GB server. I have 16GB free on my 1TB laptop which the worker is on which is why its running clean up on my deployments.
Hopefully we can get something put in our documentation regarding this so its a bit more transparent what we do for workers regarding retention but if you need your servers to be above a certain threshold you would need to manually run a script to remove those files from the worker folder.
If there are any more questions you have surrounding this let me know and I will be happy to answer them for you.