Deploy to a dynamically created Server

[cpieren]

Hello,
We’re trying to create dynamic test environments, so our process:
1.Create the machine (EC2) in AWS with Terraform,
2.Deploy the packages to the newly created instance.

How can I do that with Octopus?

[Justin_Walsh]

Hi @cpieren!

Thanks for reaching out, and for the great question. The most common way that this is done is through the use of a deployment target trigger on your project, configured to automatically deploy one or more projects to the machine when it is registered as a deployment target in Octopus.

We also have a great set of resources on deploying to elastic and transient environments here:

Elastic and transient environments - Octopus Deploy

I hope this helps, and please don’t hesitate to reach out if you have any further questions!

[cpieren]

Hey, thanks for the quick reply!
Our idea is to have multiple versions deployed to N number of AWS EC2 instances. So, our QA team can deploy a version to an instance and run tests, after some inactive time that machine would be terminated.
If I understood the “Elastic and transient environments - Octopus Deploy”, this would deploy the last version to every registered machine. Am I right?

[Bob_Walker]

Hey there, Justin asked me to take this over. I work for our customer solutions team and I hope I can help get you up and running.

You mention wanting to have multiple versions deployed to N number of EC2 instances. Essentially the QA team can quickly spin up a new testing lab with a specific version and run some tests. Then eventually have it all get torn down.

I have a few clarification questions if you don’t mind.

1. Will the release already exist in Octopus Deploy when QA wants to spin up a mini-test lab they need to select a release from a project and kick it off? For example, in this screenshot, if I wanted to test out 2021.1.1.2, I’d click on that release and tell it to deploy to the test environment (as part of the deployment it would spin up the necessary infrastructure)?

Deploying 2021.1.1.3 would spin up different testing infrastructure specific for that version.

2. Or, would QA create the release as part of this process and they’d choose a specific version to deploy? They’d add some sort of unique identifier to the release to indicate it is for them?
3. Would you have a situation where multiple EC2 instances are running the same package version? For example:

• t-myapp-01: 2021.1.1.1 -> test instance for User A
• t-myapp-02: 2021.1.1.1 -> test instance for User B
• t-myapp-03: 2021.2.1.1 -> test instance for User C

Etc.

[cpieren]

Hello, thanks for the help!
Answers:

1. Yes! The releases are created every time a developer pushes a change to the git repository.
2. No
3. It is possible for them to have more than environment with the same version.

[Bob_Walker]

Hi Chris,

Awesome, thank you for providing that extra context. The tricky part here is when you register a server you register it to specific environments for specific roles.

For example, if I had the following targets in the test environment:

• t-target-01: hello-world-api role
• t-target-02: hello-world-api role
• t-target-03: hello-world-api role

If I were to spin up a new EC2 instance, t-target-04, and give it the role hello-world-api, then by default when I deploy a release to the test environment all 4 servers would be chosen.

It is possible to limit which machines are used in a deployment via both the API and the UI.

But this is confusing, as anyone who looked at the Octopus Deploy project overview wouldn’t know what machines have what versions. And anyone could overwrite the versions on any machines at anytime.

My suggestion is to leverage the multi-tenancy tenants in Octopus Deploy. Tenants add an extra layer in which you can pick a version

And then deploy that version to a specific tenant

Once you add tenants you get the ability to restrict deployment targets to specific tenants.

TL;DR; tenants solve both problems of restricting targets to specific “testing environments” and provide a nice visualization as to what “testing environment” has what version.

To do this:

1. Create tenants representing your testing environments. I’d recommend either QA people’s names or you can use fun names like “Jedi” or “007” to represent these environments.
2. In your project configure multi-tenancy to allow for both with or without a tenant.
   
   

What this will also do is allow you to have your normal deployment process except for “test” which has your various testing tenants.

3. Assign your projects to your tenants for your test environments.
4. Create a couple of runbooks to spin up and tear down machines. Use the run a runbook step template to call them from your deployment process when doing a tenanted deployment (use a variable run condition).

The run a runbook step accepts tenant name as a parameter

You can pass in #{Octopus.Deployment.Tenant.Name}.

The runbooks themselves will do as you described, except when it comes time to register the target you’d specify the tenant name.

I hope that helps!

[cpieren]

Hey, thanks for the help!
A project has multiple clients ( tenants ), and the way they want this to work is like this:
yesterday I had 10 test env, today 5 and tomorrow 20. That is why I cannot create a tenant for each one.
So, is my understanding that we cannot do this with Octopus, that is fine. My proposed solution is using octopus to handle the ec2 deploy and do the package deploys with PowerShell. This raises some new questions:

1. Is it possible to pass a list of all variables in octopus to the TerraForm script ( without having to write one by one )?
2. Is it possible to get the package with the .config files already transformed ( values that match variable replaced by the corresponded variables values )?

Again, thank you. I really appreciate the help

[Bob_Walker]

Another option is to have a runbook that creates a tenant on the fly, spins up the machines in EC2, and triggers a deployment to that tenant. You could have another runbook query the API and see the last time a tenant was deployed to. If it is more than a certain timeframe you could remove it.

It’d be a bit more API work but completely possible.

To answer your specific questions:

1. Not easily. OctopusParameters is a dictionary, you could have a script loop through the variables and write them out to a file you could share in a specified location.
2. Using the transfer a package step you can extract a package to a location and perform all the necessary transforms. You would need to write a script to package that folder back up.

[cpieren]

Thank you! I’ll discuss these ideas with the team.

[Bob_Walker]

You are welcome. If anything new comes up please let me know.