I am auto scaling in aws and I was wondering if there was a way possible to process multiple deployments of a release at the same time instead of queuing a deployment for each machine. (trying to speed things up)
Hi Daniel,
Thanks for reaching out! Are you using Project Triggers for auto-deploy? Combining auto-scaling in AWS with auto-deploy Triggers is powerful, though some things to consider:
Octopus checks by default every 30 seconds for any new events. Auto-deploy allows you to opt-in to run the deployment task once new machines are added (and found in the 30 second intervals). However, if new machines are added while the deployment is executing, Octopus waits for the deployment task to complete before re-queueing the same task for the added machines. It can’t run the same task multiple times simultaneously.
If this looks like your scenario, I’m interested if an idea sounds helpful to you. The 30 second interval is hardcoded into Octopus. Would it be helpful to add a custom setting to increase this interval? That way, if you’re auto-scaling quite a bit, it could potentially find more added machines, thus executing the deployment to more machines at a time. However, it would mean Octopus would check less frequently. It may be a great idea to suggest on UserVoice.
If you aren’t using Project Triggers, could you explain a bit more about your process? I’ll be happy to reassess if so!
Kind regards,
Kenny
It is good to know that I can use project triggers.
The way I approached this was to setup start up scripts that registers the machine to the project and env and then triggers a deployment to that machine via the api. However I noticed a bottle neck when adding more machines. If another machine gets ready to deploy it would have to wait for the previous machine to finish. It would make sense that if the new machines just need the latest deployment that it would just pull the newest code without having to queue. since this is an on demand kind of action not a deployment to all the machines. [would be a great upgrade in the future if not]
Hi Daniel,
Thanks for the extra information about your setup. Octopus will queue any deployments for a single project to the same environment if one is already running. This is on purpose to avoid conflicts with deployment steps/scripts. To get around the behavior, similar to what we do with auto-deploy, is to deploy to as many machines as you need to create and register in a time frame.
Your point about deploying to individual machines without queuing is valid as it will not have the same issues as deploying to full environments at the same time. We are having an internal discussion about this and its need in a scaling world. I will at least get back to you with a UserVoice suggestion or a ticket for changing this behavior.
Don’t hesitate to get in touch if you have any further questions!
Kenny