Feature suggestion for load balancer scenarios

Legacy Infrastructure
1

Hi,

I have a small (or maybe not-so-small) suggestion for improving the deployment process for websites that use a load balancer (e.g. ARR). Apologies in advance for the length of the “Long version” below…

Short version - Add a “Run On - Specified Role(s)” option in the Execution Plan for steps to allow them to be run on all servers in a specified role in the same environment as the current Deployment Target. E.g. for a “Remove from Load Balancer” step that targets servers in the “WEB” role (say, UATWEB01, UATWEB02, UATWEB03) you could specify that it “Run On” servers in the “LOADBALANCER” role in the same environment (say, UATLB01 and UATLB02). This would remove the need for calling a script using PowerShell remoting from the Octopus Server to target the LOADBALANCER servers.

Long version - I have an internal IIS website running in PROD on 3 webheads that are load balanced behind 2 servers running ARR, which are themselves load balanced on a virtual IP address via a hardware firewall. When I deploy to a webhead I want a step to gracefully take it out of load on the 2 ARR servers first, before deploying the API, and then re-adding the webhead to load on both ARR servers.

I’m currently doing this:

  • Step 1 - [apply to WEB role, run on octopus server] - drain Deployment Target from load on LB Node 1
  • Step 2 - [apply to WEB role, run on octopus server] - drain Deployment Target from load on LB Node 2
  • Step 3 - [apply to WEB role, run on deployment target] - deploy website
  • Step 4 - [apply to WEB role, run on octopus server] - re-add Deployment Target to load on LB Node 1
  • Step 5 - [apply to WEB role, run on octopus server] - re-add Deployment Target to load on LB Node 2

where steps 1, 2, 4 and 5 use PowerShell Remoting from the Octopus Server to manage the ARR load balancers for each environment. It works fine, and it’s not a major pain to manage this, but we have different numbers of ARR servers in each environment (1 in DEV, 1 in TST, 2 in UAT, 2 in PRD) so we have to do some fiddling with environment filters in steps 2 and 4, and environment-scoped variables to make it run the right number of drains on the ARR right servers for each environment. And if we add another ARR sever to PROD I’d have to add another pair of “run on octopus server” steps for LB Node 3.

As an alternative, I was thinking it would be useful to be able to have the option to specify a different role in the “Run On” field. For example,

  • Step 1 - [apply to WEB role, run on ARR role] - drain Deployment Target from load on all LB Nodes
  • Step 2 - [apply to WEB role, run on deployment target] - deploy website
  • Step 3 - [apply to WEB role, run on ARR role] - re-add Deployment Target to load on all LB Nodes

Ideally, the “RunOn Role” option would then run Step 1 once on each server in the ARR role, then run Step 2 on the Deployment Target to deploy the site, and finally run Step 3 once on each server in the ARR role to add it back to load. This would simplify managing the process and would automatically know how many WEB and ARR servers to execute the steps on in each environment based on the roles assigned to them.

Cheers,

Mike

2

Hi Mike,

Thanks for getting in touch.

It’s possibly more common in this scenario to let the deployment target itself handle the drain/remove process, rather than running the drain/remove on the Octopus server or on a third role.

That way, you can take advantage of the WEB target’s context - i.e. using $env:computername to identify the node’s name, or using the WebAdministration module to identify the IP address to which the target website is bound.

In your specific case, you’d probably still be running a remoting block, but in Cloud scenarios such as AWS a node can discover the load balancer(s) of which it’s a member and drain/remove itself with an API call. In elastic environments this ability becomes extremely important, but in relatively static environments it can still pay dividends should you unexpectedly need to add, remove or replace a node.

As for the variable number of load balancer nodes in each environment, in lieu of a more complex discovery-based solution, I’d suggest using a scoped variable which contains the names or addresses of each load balancer node, for example

Name        Value               Scope
LBNodes     lbp01,lbp02,lbp03   Production
LBNodes     lbs04,lbs05         Staging
LBNodes     lbu01,lbu02         UAT
LBNodes     lbd01               Dev

Then in a single “Drain/remove” step, you can reference that variable and loop through the array, draining and removing as you go.

You could even put this into a variable set, allowing you to easily manage nodes across a number of different projects.

This approach would cut out some of the complexity and duplication in your current scenario, and makes it fairly simple to change your Load Balancer nodes should you need to swap one out. Also, with this approach, you can still keep running the Drain step on the Octopus server if you want to keep that work centralised.

With those three sub-steps enclosed in a Rolling Deployment, you can achieve zero-downtime deployments across a farm of potentially any size.

Does this address your scenario correctly?

regards
Jason

3

Hi Jason,

Thanks for the reply.

Our secops guys don’t really want us PsRemoting from the webheads onto the ARR nodes, so that rules out the simplest case of the webheads managing themselves on the load balancers, which is how I ended up running the drain step on the Octopus server.

Your suggestion of a CSV list in a scoped variable does simplify things a bit, so I might go with that. I also thought about hitting the Octopus API from within the server-side step to discover the tentacles with the “LOADBALANCER” role in the same environment as the deployment target - that’s a bit more complicated though so I might leave that for another day!

All in all, I’ve got a workable process so I’m happy with things as they stand. I was just hoping to be a bit lazier and have a native “RunOn -
Specified Role(s)” feature :-).

Thanks again,

Mike

4

Hey Mike,

Thanks for the update

I’m glad the scoped variable trick looks useful. Do feel free to get in touch again if you need us!

regards
Jason