I want to move my on-premise Octopus Deploy server to AWS using EC2 instances, but I also want to make Octopus highly available. Are there any recommendations on what resources to use in AWS to do this?
High availability in Octopus allows us to run multiple Octopus Deploy servers to distribute web traffic and tasks between them. To build an HA Octopus configuration we need four components;
-
Servers
-
Load balancer
-
Shared Storage
-
Database
I’ve documented some recommended resources to use in AWS to cover all the above components. The resources I’ve recommended are all AWS cloud services, but if you didn’t want to use a cloud service, you can, of course, just build some virtual machines in AWS and manage your own services.
Servers
The servers will be used for our Octopus nodes, and it’s where you will install the Octopus software.
There’s only one choice when building virtual machines in AWS, and that’s EC2 Instances. There are a number of different instance types to choose from. When selecting the size of the instance, I generally find sticking with the General purpose size(s) is the best option. Choosing the rights specs for the instance depends on how many nodes you plan to use. The more nodes you include, the fewer resources you will need in each instance, but I’d always recommend starting small and scaling as you need to.
Load Balancer
To access the Octopus web portal on each need node, you will need a load balancer to direct HTTP/HTTPS traffic to each Octopus node.
AWS provides a solution to distribute HTTP/HTTPS traffic to EC2 instances, Elastic Load Balancing is a highly available, secure, and elastic load balancer. There are three implementations of ELB;
- Application load balancer
- Network Load Balancer
- Classic load balancer.
To distribute traffic to the EC2 instances hosting Octopus, the implementation to choose would be the application load balancer, which is ideal for load balancing HTTP/HTTPS traffic.
I’d certainly recommend using AWS ELB because of how easy it is to set up and configure, plus, it’s nice to reduce windows server management!
Storage
You will need shared storage that all Octopus nodes can access as Octopus stores a number of files that are not suitable for the database.
To build a highly available Octopus server install, you’re going to need some durable Windows file storage, and in the past, AWS didn’t offer any cloud services that accommodated this. Your first option would be to spin up some more EC2 instances and build your own DFS(Distributed File System).
AWS recently introduced Amazon FSx, It’s a native Windows file system built on Windows Server. It includes full support for the SMB protocol, Windows NTFS, and Microsoft Active Directory (AD) integration and is an ideal choice for connecting to your EC2 instances hosting Octopus to house all your Octopus packages and log files.
If you choose to go with AWS FSx then taking a look at there starter guide is a good place to start and will help you connect your Octopus Server nodes to the storage.
Database
Octopus uses an MSSQL database to store all your Octopus configuration.
You’re going to need a highly available SQL Database for your Octopus instance. If you have a SQL Cluster in AWS, then use that, but if not, then AWS RDS would be a good choice for you as it will provide the high availability without all of the administrative headaches!