Is there a solution for dedicated task queues in octopus?
At a sort of abstract/hypothetical level it would be for instance, something like a 2 node cluster, where one node is dedicated to x project group(s) which would enable us to increase the task cap but also maintain control over how many tasks are being queued on the servers.
So lets say node one is scoped to a project (with 100 tenants) and all of the targets on that project live on a dedicated server for that project, then node 2 is scoped to everything else with all the other environments. In theory we could say, node 1 has a task cap of 3 and a full deployment for all tenants will essentially be in its own queue on the dedicated servers. Everything else then is queued in a separate node, with a higher task cap as we expect less concurrent deployments on same servers, and those deployments aren’t stuck waiting in the queue behind all of the node 1 tasks.
Initially we looked at workers for this, but as workers are for “azure deployments and deployments with no set target” i don’t think it would work.
Do you have any ideas to accomplish the idea i’ve laid out here?
Thank you for reaching out - Octopus doesn’t support that feature in the form as you’ve described it.
The closest we get is a Community Contributed Step Template entitled Re-prioritize Octopus Deploy Tasks that will give you some control over the order in which tasks are executed, I’ve attached a screenshot of it in the Community Contributed Step Template library. Be aware it’ll need to be run on a different Octopus server to the one who’s tasks you want to re-prioritize.
Depending on your setup and processes, other alternatives to consider, would be to increase Task Caps before a full deployment (may take a little experimentation to find the sweet spot due to the increased demand on the server).
Also consider keeping other node/s in a ‘warm state’ by setting their Task Cap to 0, then increasing it to the desired Cap before deployments commence. You can script the Task Cap increase; some help doing that is here: Node - Octopus Deploy. The only caveat being the script must be run on the actual server instance.
Admittedly, none of the above gets you what you want, but hopefully you’re at least part way there!
