We were looking at our SQL database size and growth, considering cleaning up the database.
We’ve noticed our largest table is dbo.VariableSet (23GB) and the majority of the entries are related to Runbook runs.
SQL Query:
select count(Id), OwnerType from dbo.VariableSet group by OwnerType
Output:
(No column name) OwnerType
2231 Project
283 LibraryVariableSet
10850 RunbookRun
1440 Deployment
We do have a lot of variables and scheduled Runbooks including some health check scripts that run on hourly intervals. It’s common that we might want to see the success/failure or look at the log of a previous Run, but it’s rare that we would want to refer to the Variable-snapshot. It looks like the option to set a Runbook RunRetentionPolicy was added in 2022.3, but also it seems like perhaps there was a hard-coded limit before this, I think I can see the past 1000 Runbook Runs in history.
Questions were:
(a) Is there any way to keep the Runbook Run-history but not the VariableSet information?
(b) Is there a hardcoded Runbook history limit (1000) in older versions?
(c) Is there a safe way to delete old Runbook Runs, either getting rid of just the Variable information, or getting rid of the run history entirely?
Thanks for any assistance or ideas you might have.
Thanks for getting in touch! We have made some changes to how Octopus handle runbook retention since v2020. In this version, I believe Octopus is storing 1000 runs per environments. Around v2022, we changed the default to keep only 100 per environment.
Are you able to check your runbook Settings > Retention Policy and see if you have the option to change it to a more manageable number?
Let me know if you’re not seeing an option here, you may need to upgrade your Octopus version as runbook have continued to receive constant updates and fixes since your current version.
Thanks Daniel,
I’ve looked for that option and I don’'t think it’s there in this older version.
Under a Project → Runbooks → Selected a particular Runbook → Settings tab
The only options are Name, Description, Deployment Target Status
Are you saying this older version probably only keep 1000 runs per each runbook, per environment?
Thanks for getting back! I had another look through our docs and found a reference to this which I missed with my initial response.
It looks like we added retention policies for runbooks in v2020.3.1. In your current version, I’m assuming we are just keeping everything, regardless of environment. After v2020.3, we have the option for keeping all runs or setting a value per environment for the amount to keep.
At a minimum, I’d recommend upgrading to v2020.3.9, which is the latest patch for that minor version. However, we release fixes and add new features all the time, so it’s worth considering a more recent version. Here is a comparison from 2020.1.22 to 2022.3.10594.
Hope that helps! If you have any questions at all, don’t hesitate to ask.