I notice that some errors of terraform are not shown in the octopus console
Like if i forget to declare a variable it will not show in the octopus, i guess its not part of the terraform.exe output command.
Also, there is a nice terraform log, except its never attached to octopus (if ($false -and (Test-Path $logFile)) )
please show more terraform information in octopus like validation errors so its easier to debug problems.
Hi Marcel, thanks for getting in touch,
I believe we have a current issue where output in task logs is being truncated which may be somewhat responsible for this. There is also an associated issue for this which has been waiting on some other changes to be implemented first. Since those changes have been made, we can now look at improving the general behavior of our terraform steps.
Regards,
Shaun
also, i see there are 2 options to run a terraform task, on the octopus server, or on the octopus server on behalf of each deployment target. Not really sure what that second option is but i just want to run the task directly on the deployment target (the octopus server is kinda isolated and and cannot access aws, while the tentacles can). Will this be an option in the future?
Hi Marcel,
That is potentially an option and did consider the same for our AWS steps, however workers may be a more suitable since it represents where something should run as opposed to a target which is being deployed to. Workers would guarantee that the action would execute once on a given machine as opposed to an arbitrary number of targets which may cause some problems for something like terraform. Workers do currently have some limitations such as not being able to scope them nor being able to bind the worker pool, however this will be changing in the near future.
In terms of run on behalf of each target, means that we will execute the step for each associated target at the given execution location, which is either the server or a worker, while providing the properties for the given target at deployment time. As an example, you could run a script step on behalf of a an azure web app target which runs on the server/worker which gives you access to properties such as web app name, resource group etc.
Regards,
Shaun
workers seems indeed the way to go, but would be handy if you can assign task to a specific worker pool, sounds now the workers need to do all. The octopus has also a builtin worker, how do i make sure that the terraform task is run on the external workers and not on the builtin worker?
Any idea when binding/roles of a worker will be implemented?
Also a nice feature would be to spin up automatic workers in Azure/AWS through VM’s or containers, then you can have a dynamic worker fleet.
Hi Marcel,
If you add an external worker to the default worker pool then Octopus will by default use that worker instead of the built-in worker. You could also add some external workers to a different worker pool and change your terraform steps to execute using that worker pool instead.
Progress has been made on the binding for worker pools along with scoping, however it’s currently on our backlog as we have some other more urgent priorities. I would love to provide an exact time frame, but would be unable to do so at this point in time.
We do have another feature called dynamic workers which isn’t too far off, that will be available to hosted customers. There could be a number of directions we could possibly take this, so would definitely be keen to hear if you have any ideas of what features you would expect from something like this and what would be useful features for you to have.
Regards,
Shaun
Thanks for all the help. I’m working on a poc where we can use octopus for deploying aws infrastructure and containers. The first part is now working (terraform), but i got still some improvements:
Next task is retrieving an container image from ECR and Putting it in ECS.
Hi Marcel,
No problem at all and thanks for providing that feedback. It definitely helps to make sure that we are heading in the right direction.
Workers support for git and ssh is not something we have originally considered. In regards to ability to run custom bash/powershell commands on the workers, we currently have the ability to run ad-hoc scripts using the script console on an individual worker or all workers in a given worker pool. Would that suit your needs or did you have something else in mind?
Regards,
Shaun
Well, modules are one of the powerfull features of terraform, and in our case those modules can be retrieved from a GIT repository with authentication.
To work around the above we can configure the worker in the right way (install GIT, configure ssh credentials) by executing a powershell/bash command. But then there should be an option in the deployment to run this.
Also, it would be handy if you can assign roles/environments to the workers. This makes it easier for us to restrict certain workers, instead of giving them full access.
Hi Marcel,
I thought you may be interested to know that since 2019.4.4 you can add the Octopus.Action.Terraform.AttachLogFile variable to your project with a value of True which will attach the terraform log file as an artifact. This should hopefully help when running into issues when using the terraform steps.
In addition to the above we also made it possible for the terraform steps to run on Linux worker machines. The only caveat here is that we don’t bundle the linux version of terraform. This means that terraform will have to be manually installed and added to the path on the worker machines. We are also working on a better story around tooling, so hopefully that would be another thing to look forward to.
Regards,
Shaun
Thanks, that would be helpful indeed.
Currently the workers are all identical and you cannot assign roles/environments to them. I understand the reason why. Except in our case we don’t have any trust between accounts and cannot use the asume function and need a worker in each different account that we own.
Any idea of assigning roles/environments to workers is something you are going to support and when?
Hi Marcel,
Sorry that we missed this reply when it first came in!
While roles/environments aren’t available for workers, what we do have the the concept of Worker Pools. You could create a pool per environment and have all of the workers in that pool running under the required account.
Any questions about this approach please let us know!
Regards,
Alex