I have a web application which sits behind a load balancer, hosted on 2 web servers. I deploy to these servers using the rolling deployment feature in Octopus Deploy.
Sometimes a deployment to one of the servers fails. Instead of failing the whole deployment, is there a way in Octopus to control what should happen next when the failure occurs?
By default, when a step in a deployment process fails, Octopus will fail the entire deployment.
Guided Failure mode
Sometimes, that behavior isn’t desirable, and Octopus has a feature known as Guided Failure to help deal with those situations.
Guided Failure mode provides an option to prompt a user to intervene when a deployment encounters an error so that the deployment can continue. With guided failure mode enabled, the user can fail the process, and retry or ignore any steps that failed the first time.
You can enable Guided Failure mode at the Environment level:
or at a Project level:
You can find out more about it on Redirecting to https://octopus.com/docs/releases/guided-failures.
Rolling Deployments With Guided Failures
Guided Failure also works well with Rolling Deployments. If your deployment fails to one of the targets in your rolling deployment you can decide how to proceed. Instead of failing the deployment, you might want to exclude that machine from the deployment and continue, or just retry it if the error is transient.
You can find out more about the options in a rolling deployment on Redirecting to https://octopus.com/docs/deployments/patterns/rolling-deployments
Caution with usages of:
Octopus.Action.MaxParallelism
A word of warning if you have a scenario like the following:
Octopus.Action.MaxParallelismBy setting the Octopus.Action.MaxParallelism you might find that in your rolling deployment, your Window size is no longer respected.
Here I have a rolling deployment with my Window size set to 1, but I also have set Octopus.Action.MaxParallelism to 10:
You’ll notice that Octopus is deploying to more than one server at once.
This can be confusing when you have Guided Failure mode turned on for an Environment/Project as you might expect Octopus to stop and ask for manual intervention on the failure to the first server.
This is actually expected behavior because I have explicitly overridden the defaults for running multiple processes simultaneously as described on Redirecting to https://octopus.com/docs/administration/managing-infrastructure/run-multiple-processes-on-a-target-simultaneously.
Octopus will also warn you which variable is causing you the issue:
If you don’t wish this to occur, you will need to remove the Project Variable: Octopus.Action.MaxParallelism
