Clustering Tentacle using active/passive windows cluster

Piers_Williams · 19 August 2013 01:03

I’m wondering what the best approach for installing a tentacle (pair) on an active/passive windows cluster is. My goal is that I should be able to install to the production environment irrespective of fail-over state, but since I am installing a database, I don’t want to just install twice (once to each node), since the two installs will fight each other.

On a 1-node cluster, I’ve had a practice, installed a tentacle, registered it into Octopus using the virtual cluster address and configured it as a cluster resource (‘generic service’). Once I brought the cluster resource online, the tentacle correctly displayed in the Octopus dashboard.

My plan was to then do the same on the passive node of the real cluster, so Octopus would only ‘see’ one tentacle at a time (depending on the cluster failover state). However for this to work the other tentacle would (presumably) need to be setup with the same thumbprint as the one on the active node, and there doesn’t appear a way to paste into the tentacle thumbprint during setup.

Failing this I guess I could make my installer cluster aware, so both tentacles are ‘active’ on the dashboard, but the install only really runs on the one which is currently the active node, but that seems like hard work.

All ideas gratefully received

Paul_Stovell · 2 September 2013 11:14

Hi Piers,

The Tentacle thumbprint information is stored in the registry under HKLM\Software\Octopus - if you export those values and import them on the other machine, they will share the same thumbprint.

Hope that helps,

Paul

bev_guest · 23 April 2014 15:34

Hi,
Running V 2.3.3 of OctopusDeploy & Tentacle.

This Certificate now doesn’t appear to be stored in the registry anymore.
I can only see this in the Tentacle.config as well as any cached User Profile Locations in Registry under Local_User.
I’ve attempted to change the Thumbprint/Certificate in this config file, and cleaned out any record in registry with old thumbprint, but find that when I start up Octopus Tentacle Mgr, it is still returning the original Thumbprint.

I need to find where this is so I can correct this to continue with the Clustered Generic Service setup.

Can anyone help?

Thx
//Bev

Paul_Stovell · 24 April 2014 00:28

Hi Bev,

In the latest version of Octopus, you can run:

tentacle.exe new-certificate -e "YourFile.pfx"

This creates a new certificate file.

Now on both machines you can run:

tentacle service --stop
tentacle.exe import-certificate -f "YourFile.pfx"
tentacle service --start

Now you can re-open the Tentacle manager on both machines and the same thumbprints should appear, and both machines will use the same certificate.

Hope that helps! Let me know if you need any other details.

Paul

bev_guest · 24 April 2014 14:57

Ho Paul, Thx.

I am waiting for approval to upgrade from 2.3.3 to 2.3.6.
Will feedback post this.

Thx
//BevG

bev_guest · 29 April 2014 13:58

Hi Paul,

OK, so we upgraded to latest 2.3.6.x and I ran the New-Certificate command.

But the following error was returned.

C:\Octopus Deploy\Tentacle>tentacle.exe new-certificate -e "MITTentacl1.pfx"
Octopus Deploy: Tentacle version 2.3.6.1385

A fatal exception occurred
System.ArgumentException: Unrecognized command line arguments: -e MITTentacl1.pf
x
at Octopus.Shared.Startup.AbstractCommand.UnrecognizedArguments(IList`1 argum ents) in c:\TeamCity\buildAgent\work\1116bd9da9e239fd\source\Octopus.Shared\Star tup\AbstractCommand.cs:line 18 at Octopus.Shared.Startup.AbstractCommand.Octopus.Shared.Startup.ICommand.Sta rt(String[] commandLineArguments, ICommandRuntime commandRuntime, OptionSet comm onOptions) in c:\TeamCity\buildAgent\work\1116bd9da9e239fd\source\Octopus.Shared \Startup\AbstractCommand.cs:line 42 at Octopus.Shared.Startup.ConsoleHost.Run(Action`1 start, Action shutdown) in
c:\TeamCity\buildAgent\work\1116bd9da9e239fd\source\Octopus.Shared\Startup\Cons
oleHost.cs:line 36

Nicholas_Blumhardt · 30 April 2014 03:58

Hi Bev;

You might try the full parameter name instead:

tentacle.exe new-certificate --export-file="MITTentacl1.pfx"

If that fails, it would be good to see what the output of:

tentacle.exe help new-certificate

…shows on this machine.

Regards,
Nick

bev_guest · 30 April 2014 08:23

Morning Nick,

Thx for that, the command --export-file still didn’t work.

OUTPUT Error:
C:\Octopus Deploy\Tentacle>tentacle.exe new-certificate --export-file="MITTentac
le1.pfx"
Octopus Deploy: Tentacle version 2.3.6.1385

A fatal exception occurred
System.ArgumentException: Unrecognized command line arguments: --export-file=MIT
Tentacle1.pfx
at Octopus.Shared.Startup.AbstractCommand.UnrecognizedArguments(IList`1 argum ents) in c:\TeamCity\buildAgent\work\1116bd9da9e239fd\source\Octopus.Shared\Star tup\AbstractCommand.cs:line 18 at Octopus.Shared.Startup.AbstractCommand.Octopus.Shared.Startup.ICommand.Sta rt(String[] commandLineArguments, ICommandRuntime commandRuntime, OptionSet comm onOptions) in c:\TeamCity\buildAgent\work\1116bd9da9e239fd\source\Octopus.Shared \Startup\AbstractCommand.cs:line 42 at Octopus.Shared.Startup.ConsoleHost.Run(Action`1 start, Action shutdown) in
c:\TeamCity\buildAgent\work\1116bd9da9e239fd\source\Octopus.Shared\Startup\Cons
oleHost.cs:line 36

This is the HELP command returned info for tentacle:

C:\Octopus Deploy\Tentacle>tentacle.exe help new-certificate
Octopus Deploy: Tentacle version 2.3.6.1385

Usage: Tentacle new-certificate []

Where [] is any of:

  --instance=VALUE       Name of the instance to use
  --if-blank             Generates a new certificate only if there is none

Or one of the common options:

  --console              Don't attempt to run as a service, even if the
                           user is non-interactive
  --nologo               Don't print title or version information

KRgds
//Bev

Nicholas_Blumhardt · 1 May 2014 00:19

Bev, I’m terribly sorry - this change did miss the 2.3.6 build after all. My mistake.

The 2.4.x builds now available in previtew definitely include it. If you’re unable to run the preview, you can still use an extracted copy of Tentacle.exe to export and import certificates with these commands.

You’d need to stop the 2.3.6 Tentacle service, perform the export/import using the 2.4 Tentacle binary (no installation required), then start the 2.3 Tentacle again.

Apologies again for the inconvenience.

Nick

bev_guest · 1 May 2014 02:52

Hi Nick,

Worked like a dream.

So what we can establish:
1.Clustering the Tentacle Service using Microsofts “Generic Service” is really straight forward.
2. Generating a New Certificate - Works
3. Importing the New Certificate into each Cluster Node Tentacle Store - Works
4. Failover Cluster Resource, OctopusDeploy continues to communicate correctly with the Identified MSCS VIP and Tentacle Thumbprint
PERFECT!

Thx for the assistance here, will make our automation to databases far less complex.

KRgds
//Bev

Piers_Williams · 1 May 2014 13:35

I should say (for posterity, but also for Bev) that whilst we did get the tentacle clustered (using the technique Paul suggested originally), and it works great, I plan to change the setup in future.

Having only one tentacle active on the cluster precludes the possibility of deploying other things to that infrastructure which may need to be deployed to the nodes, not to the cluster. In our case we do exactly that: we have some custom SSIS extensions that need to be GAC’d, and that needs to be done on both nodes (at the moment I have to remember to manually deploy to the passive node).

So instead I think I’ll go back to just having two tentacles, one per node, and do something like:

have two roles (SQL and SQL-passive), only deploy the database from one and accept that after failover occurs I need to manually swap the roles
use OD 2’s rolling deployment to sequence the install from each node, and combine with some improvements in our install process so the second node to run sees there’s no work to do

Alternatively I could (I suppose) install a second tentacle instance side-by-side on each node, and not enroll the latter into the cluster. Then I’d have a ‘SQL’ tentacle role (which the DB can target) as well as two ‘SQL-node’ role (for the other bits).

Sjors_Takes · 5 February 2016 15:20

Hey guys, just wanted to let you know I also got it working thanks to this thread. Thanks! I’ll write a blogpost about it.

robpearson · 8 February 2016 00:09

Hi Sjors,

Thanks for your feedback! Please share the blog post link with us and we’ll include it in our newsletter.

Rob

Sjors_Takes · 10 February 2016 19:14

Hi Rob,

The blog post is published. You can find it at: http://www.devopsdba.com/clustering-octopus-tentacles/.

Cheers!

Sjors

robpearson · 10 February 2016 23:42

Hi Sjors,

That’s awesome! Thanks for sharing!

Rob

Ryan_Adler · 16 March 2016 12:20

Blog post leads to a 404.

Sjors_Takes · 16 March 2016 13:22

Hi Ryan,

Thanks for letting me know. I’ve checked and it didn’t lead to a 404 (anymore). Can you try again?

Ryan_Adler · 16 March 2016 13:40

Thanks! Great post - it’s great to see that we’re not the only ones struggling with this. As you mentioned above, though, this method only works when you only need one role. Once we add another role (as is our case), we can’t install multiple tentacles. I guess I’ll just have to keep begging the Octopus team for the ability to install Tentacles multiple times on one machine.