I deploy to multiple machines as part of a rolling deployment in Octopus, and one of the steps creates a Scheduled Task.
If they run at the same time, they can often cause conflicts with one another. The issue I have is that I need to find a way to schedule the tasks to start 15 minutes apart to ensure that each one starts at different times to reduce the chance of a conflict.
Is this possible within a rolling deployment in Octopus?
That’s a really interesting question!
Whilst it probably wasn’t one of the primary use-cases considered when adding support for rolling deployments in Octopus, it’s possible to achieve using a combination of the following features:
The key to getting this to work is to have a counter which is incremented as each deployment target is being deployed to. With this, you can then safely increment a start time which is staggered by a different amount of time for each scheduled task.
Window Size of 1
The following solution has only been tested with a Window Size of1.
For this to work we’ll create the following script steps outside of the rolling deployment:
0.Note: These two steps could be combined.
Then we’ll add a number of script steps as child steps to the rolling deployment:
1 as each machine is deployed to.All of the relevant steps are illustrated here:
This script step which takes today’s date, and creates an Octopus Output variable called DeployStartTime. This date value will be used as the base value to be incremented for the staggered Task start time. The complete script looks like this:
$now = Get-Date -Format "s"
Set-OctopusVariable -name "DeployStartTime" -value $now
This script step simply creates an Output variable called Counter with a value of 0:
Set-OctopusVariable -name "Counter" -value "0"
The next script step increments the counter that was generated in the previous Generate Counter step, and this time, it’s part of the rolling deployment.
However, to increment the counter correctly, the step needs to take into account that this may be the first or subsequent machine to be deployed to.
It does this simply by checking for the existence of an Output variable called:
Octopus.Action[Increment Counter].Output.Counter.
Note that the name of the variable includes the name of the child step we are actually already running - Increment Counter.
This is because:
Octopus.Action[Increment Counter].Output.Counter variable as the source for the current value of the counter.Octopus.Action[Increment Counter].Output.Counter variable for the first time. We use the value from the output variable created in the Generate Counter step.In either case, we increment the value by 1.
The complete script looks like this:
$InitialCounter = $OctopusParameters["Octopus.Action[Generate Counter].Output.Counter"]
$IncrementedCounter = $OctopusParameters["Octopus.Action[Increment Counter].Output.Counter"]
Write-Host "InitialCounter is $InitialCounter"
Write-Host "IncrementedCounter is $IncrementedCounter"
if([string]::IsNullOrWhiteSpace($IncrementedCounter) -eq $false) {
$Counter = ([int]$IncrementedCounter) + 1
Write-Highlight "Incrementing counter to $Counter"
Set-OctopusVariable -name "Counter" -value $Counter
}
else {
if([string]::IsNullOrWhiteSpace($InitialCounter) -eq $false) {
$Counter = ([int]$InitialCounter) + 1
Write-Highlight "Incrementing counter to $Counter"
Set-OctopusVariable -name "Counter" -value $Counter
}
}
The next child script step takes the incremented counter, referenced by a Project variable called CounterValue and multiplies that value with a second Project variable called StaggeredDelayMinutes.
Note: This step makes use of a convenient feature within Octopus. You can create Project variables which reference values created in the deployment process itself.
The Project variable
CounterValueis actually just a reference to the output variableOctopus.Action[Increment Counter].Output.Counter.In this case, this is simply done to make the variable name simpler.
The variables can be seen here:
Once the delay in minutes has been calculated, the script then adds that value to the base start time we generated in the previous Set Global StartTime step. Finally that staggered start time value is stored as an output variable called TaskStaggeredTime.
The entire script looks like this:
Write-Host "Deploy started at: $DeployStartTime"
Write-Host "StaggeredDelayMinutes: $StaggeredDelayMinutes"
Write-Host "CounterValue: $CounterValue"
$counter = [Convert]::ToInt32($CounterValue)
$minsDelay = [Convert]::ToInt32($StaggeredDelayMinutes)
$DeployStart = [DateTime]::ParseExact($DeployStartTime, "s", $null)
$calculatedStaggerMins = $minsDelay * $counter
Write-Host "Calculated Mins Delay: $calculatedStaggerMins"
$TaskStartTime = $DeployStart.AddMinutes($calculatedStaggerMins)
$TaskStaggeredTime = $TaskStartTime.ToString("s")
$machineName = $OctopusParameters["Octopus.Machine.Name"]
Write-Highlight "${machineName}: Calculated delay of $calculatedStaggerMins mins, staggered task start: $TaskStaggeredTime"
Set-OctopusVariable -name "TaskStaggeredTime" -value $TaskStaggeredTime
Finally, you can create your scheduled task with your staggered start time. This script step just prints the staggered task time to demonstrate using the TaskStaggeredTime calculated on the previous child-step.
Write-Highlight "Simulating Scheduled Task creation with time of: $TaskStaggeredTime"
Here is the deployment of the project, showing the staggered deployment times for each task to be created across each deployment target:
Sample project:
You can see an example of this in our samples instance (log in as Guest): Octopus Deploy
