Some waiting tasks have no corresponding Interuption

reliability
server
unknown
(joacim.andersson.olsen) #1

We want to restrict deployments to our test environments to specific times during the day in order to not disturb our testers

We have a manual intervention step in all our deployment processes:
pic%201

From the log we can see that the server task ID = “ServerTasks-432273”
The raw log for this deploy:
pic%202

We use the Octopus Client API to submit the deployments:
foreach ($interupt in $repository.Interruptions.List().Items)
{
if ($interupt.TaskId -eq “ServerTasks-432273”)
{
Write-Host “This is the one”
}
if ( ($interupt.Title -eq “DeployTimeIntervention” -or $interupt.Title -eq “DevOps.DeployTimeIntervention”) -and $interupt.IsPending) #does not catch ServerTasks-432273
{
$taskId = $interupt.TaskId
$file = $date_time.ToString(“yyyy-MM-dd”)
#Write-Output “Starting server task: $taskId”
Add-Content “.$file.log” “Starting server task: $taskId”
$interupt.Form.Values[“Result”] = “Proceed”;
$octoClient.Post($interupt.Link(“Submit”),$interupt.Form.Values, $null);
}
}

This has worked well for about a year but lately some deployments are not beeing processed.

I also posted this question which might be related:

Processes awaiting a task completed 3 years ago
(Paul Calvert) #3

Hi Joacim,

Thanks for getting in touch!

When this issue occurs, would you be able to generate and attach an extract of all interruptions using http://<serverURL>/api/Spaces-1/interruptions?skip=0&take=2147483647

It would be interesting to see if the problem is that it doesn’t appear in the list at all, or if something is causing the title or state to be different to what your script expects.

Could you also provide the Task ID too when you do this?

Best regards,
Paul

(joacim.andersson.olsen) #4

The json response is too lengthy for a post so I have mailed it to you.

The server task “ServerTasks-432273” is still in the list’s of waiting tasks.

(Tina) #5

Hi Joacim,
Sorry for the delay in getting back to you.
If possible can you send over the full Raw Task Log?

Thank you,
Tina

(joacim.andersson.olsen) #6

Godmoroning Tina,

I have attached the raw log. As you can see from the log the deployment was manually resubmitted. The GUI doesn’t seem to work directly with the Interuptions object?

//Joacim
ServerTasks-432273.log.txt (52.7 KB)

(Derek Campbell) #7

Hi @joacim_andersson_olsen,

Thanks for your patience on this.

Would a script set to run for just a Test environment that checks the time against an allowed time be an approach works for you?

For instance, you could have something like the below:

$workDayStartHour = $OctopusParameters["WorkingHoursCheck.StartTime.Hour"]
$workDayStartMinute = $OctopusParameters["WorkingHoursCheck.StartTime.Minute"]
$workDayEndHour = $OctopusParameters["WorkingHoursCheck.EndTime.Hour"]
$workDayEndMinute = $OctopusParameters["WorkingHoursCheck.EndTime.Minute"]

Write-Host "Working Day Start Hour $workDayStartHour"
Write-Host "Working Day Start Minute $workDayStartMinute"
Write-Host "Working Day End Hour $workDayEndHour"
Write-host "Working Day End Minute $workDayEndMinute"

$workDayStart = Get-Date -Hour $workDayStartHour -Minute $workDayStartMinute
Write-Host "The working day is set to start at $workDayStart"

$workDayEnd = Get-Date -Hour $workDayEndHour -Minute $workDayEndMinute
Write-Host "The working day is set to end at $workDayEnd"

$currentDate = Get-Date
Write-Host "The current time on the server is $currentDate"

if ($currentDate -lt $workDayStart)
{
    Write-Host "No manual intervention required, the current time is $currentDate which is less than $workDayStart"
    $manualInterventionRequired = $false
}
elseif ($currentDate -gt $workDayEnd)
{
    Write-Host "No manual intervention required, the current time is $currentDate which is greater than $workDayEnd"
    $manualInterventionRequired = $false
}
else {
    Write-Host "Attempting to deploy during the work day, manual intervention required"
    $manualInterventionRequired = $true
}

Write-Host "Manual Intervention Is Required: $manualInterventionRequired"
Set-OctopusVariable -name "ManualInterventionRequired" -value $manualInterventionRequired

From there, you could then specify the hours allowed as a library or project variable like below:

With this is step on our demo Octopus:

Another approach would be to use Scheduled Triggers with no manual intervention to run at a set time, which is in the allowed time frame? You could set this to trigger a deployment to test at X time daily, Days per week, Days per month, and as a CRON expression as below:

Please let me know if you have any questions.

Thanks

Derek