I am in the process of expanding the use of HashiCorp’s Terraform tool in Octopus Deploy, in particular for AWS and some Azure deployments. Terraform is basically a desired state configuration based system where you describe the layout of the infrastructure and the tool figures out how to make it happen. It fits really well with Octopus, as it handles getting the environment ready to deploy, and Octopus does the deployment (our first use case is terraform to get ECS configured, Octopus to deploy the container). The tool is ultimately an exe and produces/consumes a variety of artifacts during usage. The primary interactions with Octopus are:
- variables (same basic usage as Octopus variables)
- remote state (the unique location of the deployment metadata/state in S3 (or some other store))
- state files (the input/output of terraform, as derived from AWS/Azure before/after deployment)
I have a PowerShell step template that drives the tool, but I’m looking to do something a little bit deeper. Some of things I would like to do:
Have the ability for a user to define the set of step variables
Right now, we are using the OctopusParameters dictionary to pull in all parameters, and have code that finds all the matching terraform variables, and then set the terraform variables to the values from Octopus. This works correctly, but it isn’t very self documenting, as there is no place to see/understand what Octopus variables are being used in a given run of Terraform. In some ways, I want a template for a step template, with a base set of parameters, but the ability to add more.
Have the remote state being tied to the environment/project/deployment
Right now, I’m setting the remote state location (S3 blob) based on step variables. This works, but any errors during the step execution may result in the state file not being updated (e.g., error in get/push to S3). What I would really like is the ability to provide/consume state files that are associated with the Octopus deployment/project/release. I can add deployment artifacts, but they are really just for reporting. For example, Octopus tracks the state of the files that it sends to the tentacles, and I would love to have it also track the state files, which would then be available in the next release/deployment. I don’t know enough about Octopus internals to know if these are the correct terms.
Stop a deployment for human intervention.
One of the features of Terraform is that you can generate a plan. You can then inspect the plan, and decide if you want to continue (e.g., check to make sure that your database isn’t able to be dropped). I would love to plug this into the manual intervention mechanism, so the user can be given the choice to continue the deployment in light what what will be changed.
There are likely other interactions that I’ll need. I can continue to hack my way through them, but it doesn’t hang together well. What I want to do is extend Octopus (e.g., Docker or Azure ARM integration) that goes beyond Step Templates. Is this possible? If so, is there any documentation?