Do I need to upgrade the Octopus Tentacle and Calamari?

After an upgrade of the Octopus Server, I am prompted to upgrade the Tentacles from the Infrastructure page:

The health checks are showing healthy with warnings.

When I kick off an upgrade that blocks deployments to those machines. Do I really need to upgrade the tentacles and Calamari? Why am I being prompted to do so?

It is important to take a step back and understand what Tentacles and Calamari do.

Calamari

Calamari is a command-line application which does the actual work of the deployment. Calamari is the brains of the operation, it has to understand the commands the Octopus Server is sending it. As such Calamari and the Octopus Server are tied together. When doing a deployment, the Octopus Server will ensure the version it has matches the version on the deployment target. If it doesn’t match, the Octopus Server will send down the version it has. Even if the Octopus Server has an older version than what the tentacle has.

Are Upgrades Required For Calamari

The Octopus Server will ensure Calamari is up to date during deployment time. Unless you need to speed up deployments, I’d recommend waiting till the next deployment to push the latest Calamari update.

Tentacles

Tentacles are a Windows Service which facilitates the communication between the Octopus Server and the deployment target. It passes commands onto Calamari to execute. The tentacle and Octopus Server do not have to match up. If you take a look at this page you will see, for the most part, a deployment target running a 3.0 tentacle from several years ago will work with Octopus Server 2019.x.

Are Upgrades Required for the Tentacle

Upgrading the tentacle is not required, but it is recommended. New versions of the tentacles include bug fixes, performance improvements, and security enhancements.

That being said, upgrading the tentacle is a blocking task on the deployment target. No deployments can occur to the deployment target while a tentacle is being upgraded.

Upgrade Approaches

If you have a couple of dozen tentacles the upgrade process will go pretty quickly. If you have 100s or 1000s of tentacles, upgrading can take quite a bit of time and will block deployments from happening.

To prevent the Octopus Server from melting during an upgrade a rate limiter is applied. That rate limiter is based on the number of CPU cores assigned to the Octopus Server. In other words, it will upgrade a specific number of tentacles at a time based on the number of CPUs assigned to the Octopus Server.

We have found the following tends to work best when upgrading 100s or 1000s of tentacles.

  1. Ensure upgrading makes sense. Did you upgrade from 3.17 to 2019.x? Upgrading to the latest tentacle will give you extra features, so it makes sense. Are you seeing a memory leak or performance problems? Then upgrading to fix a bug makes sense. Review the change log to see what has changed since your last upgrade. Phrases to look for are Tentacle or Halibut.

  2. Plan the upgrade. Don’t upgrade all tentacles all at once. Roll it through by environment or leverage machine policies to do it for you.

Upgrading all tentacles per environment can be done by clicking the ... and selecting Upgrade all Tentacles in this Environment.

Machine policies have come along way over the past year. You can now schedule them via a Cron expression.

And you can tell the machine policy to upgrade the tentacle during the machine policy check or when it is involved in a deployment.

When you have 100s or 1000s of targets we do not recommend having a single machine policy. Break up the deployment targets into multiple machine policies. For example:

  • Machine Policy per environment
  • Machine Policy per data center
  • Machine Policy per customer region

By doing this you can control when an upgrade will occur automatically. For example, if you have a machine policy for Development and a machine policy for Testing then you can schedule the upgrade to occur at 7:00 PM when most of the devs have gone home. Or having a machine policy per customer region will allow you to schedule an automatic upgrade at 2:00 AM local time for that region.