Checking tentacle health issues

Hi Paul,

At work we currently have 12 machines across 4 different environments. When Octopus does its automatic “Check Tentacle health” task, it splits the tasks up into 2 batches (10 for the first, 2 for the second). It takes around 3 minutes to run both of those tasks. Since tasks are run in a serial manner, that only leaves about 2 minutes of idle time before the next automatic check takes place. Here are some issues I am finding because of how long those checks take:

  • All deployments are put on hold until the checks are done. Waiting 3 minutes for a deployment to start is annoying to say the least. Is there a way these “Check Tentacle health” tasks could be run in parallel to deployment tasks?
  • If I cancel an automatic check that’s in progress, the task gets marked as “Cancelled” but it seems like it is still running. Any deployments that are queued still stay “Queued” until a minute or two passes. When that time elapses, the “Cancelled” task actually gets marked as “Success!” and then the deployment task starts running.
  • Can there be a way to filter out the “Check Tentacle health” tasks? When I go to the Tasks tab, all I see are the health checks. I would instead like to see what deployments are taking place. Having an option to display/hide these health checks would be awesome.

Thanks and keep up the great work! It’s a really awesome product! :slight_smile:
Steven

Hi Steven,

Thanks for the feedback.

Is there a way these “Check Tentacle health” tasks could be run in parallel to deployment tasks?

Octopus server is actually built to run in parallel without a problem, I just need to enable it. This feedback is important because until now I wasn’t sure if other people needed it :slight_smile: I’ll try to include it in next release.

When that time elapses, the “Cancelled” task actually gets marked as “Success!” and then the deployment task starts running.

Ah yeah - cancellation is a little broken at the moment. It sets a flag and the tasks check that flag, but sometimes it is too late for them to check. I’m going to experiment with simply terminating the task thread.

Can there be a way to filter out the “Check Tentacle health” tasks? When I go to the Tasks tab, all I see are the health checks

Absolutely - I’m going to automatically delete all automatic health checks apart from the last one run.

Thanks Steven!

Paul

Hi Steven,

The build I’m sitting on (will release soon) has a few fixes:

  1. All but the last health check get automatically deleted on a scheduled basis
  2. Cancellation works for most tasks now
  3. Tasks (e.g., deploy, check health) run in parallel (up to 4 at a time)

Also, activities within a task will also run in parallel, and be a lot smarter. For example, Octopus will upload all NuGet packages to the target machines before installing any of them - that way downtime is minimised.

If you get chance, please cast your eye over this log - this is what deployment logs will start to look like in the next release. Most of the child tasks are being run in parallel now:

Paul

Hi Paul,

That looks great! The tabs and whitespace really help to keep the deployment log readable. Does the summary at the top change as the deployment is taking place? For a second I thought the individual steps were being run in parallel too, but it makes sense that steps would be done in a sequential manner.

Can’t wait to use the new features! :slight_smile:

Thanks!
Steven

Hi Steven,

Thanks! Yes, the whole log will continually update as the deployment runs. The current stage will also show [Running…] next to it, and if the install fails, the stage that failed will show [Failed]

You are right - each step is run in sequence, but the individual step is installed in parallel - so for example if you have this configuration:

Package A -> Machine 1, 2, 3
Package B -> Machine 4, 5

It will work logically equivalent to something like this:

in parallel:
    Install package A to machine 1
    Install package A to machine 2
    Install package A to machine 3
wait
in parallel:
    Install package B to machine 4
    Install package B to machine 5
wait

Paul