Hi there, I have a problem with task cap.
We have one node and task cap is set to 10.
I noticed that if we have a deployment, that starts other deployments, this task cap is not respected.
Example:
Let’s say we have three projects, first called “Create”, second called “Upgrade” and third called “Do it after upgrade”.
- “Create” as one of its steps deploys a release from an “Upgrade” project.
- “Upgrade” as one of its last steps deploys a release from an “Do it after upgrade” project.
- I have task cap set to 10.
- I start 20 “Create” tasks.
10 of them are started immediately and 10 of them are queued. So far so good.
But then at some point list of running tasks starts to grow, because every running “Create” task starts and “Upgrade” task, etc.
So at some point I am getting 30 running tasks.
These numbers are not very bad, but previously we had task cap set to 20, and at some point we’ve got ~80 running tasks, and it just killed Octopus server. It became unresponsible and we had to restart the whole server.
Is this a known problem and is there any solution?
Hi @Denis_Volovenko!
This is by design, otherwise this we’d have an issue with deadlocks with tasks staying queued forever, if the child tasks were to be added to the queue. I’m sorry for the rude shock you undoubtedly had when you encountered this.
In regards to solutions, your task cap solution is likely your best bet here to keep the maximum concurrent tasks under control.
I hope this helps explain the situation, and please don’t hesitate to reach out if you have any further questions.
Hi @Justin_Walsh,
thanks for you answer, I see your point, I guess it makes sense.
But right now I have a little bit different situation, I started 100 Create deployments against node with cap=10 and it has ~50 of Create tasks running. So not like I wrote in previous message where it has 10 parent tasks and all of the children.
Why could it be possible?
Hi @Denis_Volovenko
Thanks for the update - I’m just working on reproducing this locally, and I just want to make sure that I have my environment set up to match (as I’m not seeing these results).
Master1 Project
--- Deploy a release on Sub Project 1
--- Deploy a release on Sub Project 2
--- Deploy a release on Sub Project 3
--- Deploy a release on Master2
Master2 Project
--- Deploy a release on Sub Project 3
--- Deploy a release on Sub Project 4
--- Deploy a release on Sub Project 5
--- Deploy a release on Master3
Master3 Project
--- Deploy a release on Sub Project 6
--- Deploy a release on Sub Project 7
--- Deploy a release on Sub Project 8
You’re then doing a looped API call to deploy Master1?
