Hi,
We have a situation where we deploy our web site to a ‘pre-live’ path initially and then run automated tests on it. Our implementation is using IIS, so we just point a ‘pre-live’ virtual directory at the unpacked nuget package that is deployed (we don’t copy to a custom location).
If the tests pass, we then point the live site at the new package, if the test fail we remove the pre-live virtual directory and live site does not change.
We initially had our retention policy set to 3 releases in our test environment, so if we have 3 failed build/deploys then the live site disappears because the retention policy is removing the live package and the new packages never make it to being live.
Obviously one solution is to increases the number retained and tell the team to do a better job 
We could deploy to a custom folder, but that adds some complexity that we don’t really want, granted it is small.
Are there any plans to include the success or failure of a release into the retention policy? Currently we are retaining all releases in production to be 100% safe, but at some point this could catch us out when we run out of disk (although the machine policy will at least flag that to us).
(I was hoping to check your backlog out before posting, but I could not find it).
Hi Simon,
Thanks for getting in touch! Sorry for the delay in getting back to you I was testing a few scenarios. This is an interesting conundrum. I might ramble on a bit but this would be the tl;dr; Your process and what we would consider best practices and our retention policy won’t really play seemlessly together. That is in no way a judgement of your process (or ours) but more they are slightly incompatible. There are a couple of things that I think can be done to make it work such as finding a magic number for your process where the folders won’t be deleted, or add a workaround package step at the end of the deployment to halt the cleanup.
The main and major problem here is that our Tentacle retention policies run after all the package steps complete, but not as the final thing that happens on that Tentacle or deployment (unless they are the last steps). There are a few reasons around why this decision was made, but even if the logic is debatable, it has been that way in Octopus now for a long time, and changes in behaviors like that cannot be made lightly as they would be relied on to work as they do now for many of our customers.
Being that you deploy your packages, then do some other tests and work, the retention policies run before you have determined you have a stable release or not. Generally with Octopus our customers would deploy their previous stable release to correct the issue, but you have built that into a single process.
Finding a magic number of releases to keep inside of your process would be one way to deal with this. A number that is on the edge of successful releases and failed releases. However knowing how the retention policies work, I am going to suggest a bit of a hacky workaround. It may very well be a terrible workaround, but I have tested it and it works.
If you add a package step as the last step that your process contains, then the retention policies do not kick in for that deployment if something fails before the step runs. This means so long as you are marking the deployment as failed when your tests fail, then the final package step will not run, nor will the retention policies. Like I said, terrible workaround. You would only need a very small package for this purpose.
For your information this is where our public backlog can be found: https://github.com/OctopusDeploy/Issues/issues
It contains issues and features that we have committed to completing or investigating. We try to stay as open as we can about it.
We also have a road map but are in the process of changing where this is built from: https://octopus.com/company/roadmap
Let me know what you think!
Vanessa
Hi Vanessa,
We currently have a ‘magic number’ implemented, which is probably going to be 95% + reliable, so it is not a bad solution, I was just exploring options for a more robust solution.
The package step idea is an interesting one and not one that is obvious, so we will give that some thought.
I appreciate you getting back to us.
Simon.
Hi Simon,
Let me know how you go. It may be worthwhile checking in UserVoice or creating a new suggestion if you can think of an enhancement or feature that would work here.
http://octopusdeploy.uservoice.com/
Vanessa