We have a deployment target called “Web Server”. It has multiple load balanced servers defined.
We have another deployment target called “Test Machine” that has one Test VM defined.
We deploy the web apps to the web server and the selenium tests to the Test VM.
The deployment to the web server is a rolling deployment and contains multiple steps. It will run once for each server defined in the “Web Server” .
Here is my problem. I would like to execute the selenium test on the Test VM inside the deployment for the web applications. i.e. i want to test if everything is OK before i add the server back to the load balancer.
When i move the test step into the web deployment i get a warning
“This step previously targeted a different set of roles than the step you are moving it to. This means the step is now likely to run on a different set of machines. Are you sure you wish to move the step?”
When i go ahead of course my test fails because my tests cannot be found on the web servers.
Any ideas on how i can mix deployment targets within one deployment ?
So far we do not want to install software on our production web servers to run the tests. Also the octopus tentacle has to run as an interactive process and not as a service in order for the selenium tests to succeed. it also seems i have to be logged on the the test server as the user who is executing the tests. All no-nos for a production web server.
Thanks for getting in touch! I have run this past the team and may have an idea as to how you can achieve the results you want here.
Currently child steps are only able to inherit the Machine Role from the parent step. It looks like this is causing you some issues with running your tests but I believe we may have some options for you to automate this process in Octopus.
The idea we had was to see if you could possibly implement a Deploy Release Step in the middle of your rolling deployment. The Deploy a Release step will let you deploy the tests as a separate project whilst the rolling deployment is put on hold pending its completion.
This approach does mean you will need to maintain a separate project for your tests but the end result should be full automation of the tests on a separate role from your Web Server targets.
As for running the Tentacle as an interactive process, I believe this is required for the Selenium tests to work. Tentacle by default does not have a desktop or UI when running as a non-interactive service. Selenium requires a web browser to work so running the Tentacle as an interactive service is currently the only method I know of getting this to work.
Does the above help, or work for you?
I am happy to continue working with you to get your deployment running as smoothly as we can. If you have any thoughts or questions here please feel free to send them through.
I thought of this solution as well but i got stuck because the deploy release only deploys an existing release. however i need a new release because the tests are changing sometimes. I could not find a step template that creates a release and then deploys it. also because this would be running inside of a rolling deployment that means i would have to deploy the tests at least twice to the same test VM (we only have one) even though the test has not changed during the deployment.
The other downside to this approach is we are doubling the number of projects . However i’m willing to try this and then run it by our team to see if they would be ok with doubling the number of projects.
Could you let me know if there is a way to create a release of the test project from within the web project?
Thanks for the update. I think I can provide you with some ideas on overcoming this hurdle.
The first option relies on you using the Built-in feed. By enabling Automatic Release Creation on the project with the test you can automatically create a release whenever a package is pushed to the built-in repository and matches the package ID you select. This option means a new release will be created for this project every time you have a new package to use.
The other option is to add a script step in the rolling deployment which kicks off Octo.exe create-release which will create a release for whatever project you specify. As this option is technically just a script line, it can be placed anywhere during your deployment that you can use Custom Scripts. (Before the deploy a release step)
If you use a second project to hold the package for the testing, you can automate the release creation process and have the deploy a release step do what it needs to in the middle of the rolling deployment.
Do these options help you out here? I have tried to address all of your concerns, let me know if you have further questions about any of this.
I tried to use your suggestion to use octo.exe to create a release. This runs and creates a release. However the deploy release then fails with this error.
The step failed: Activity Deploy and run Selenium Test on WebTest01 failed with error ‘Unable to cast object of type ‘Octopus.Server.Orchestration.ServerTasks.Deploy.Steps.DeployRelease.DeployReleaseActionHandler’ to type ‘Octopus.Core.Orchestration.ServerTasks.Deploy.ITargetActionHandler’.’.
Another problem would be that when i run the deployment i am asked to pick a version of the test project. However i don’t have a version yet because it will be created during the deployment. So i just picked the newest version. Is there such a thing as telling the deploy release step to just use the latest version always?
i also tried to put the creation and deployment of the test project into one script like this
but the problem there is that octopus doesn’t wait for the test to finish on my other machine. it just continues directly to the next step not waiting for the tests on the test vm to finish.
maybe this is because it’s not running on the same machine as the web deployment?
Hi, I was reading this and thought I might offer a slightly different suggestion. I think your testing needs to be abstracted away to the CI/CD level. What I mean is that Octopus is a great deployment tool and you should just let it do what its best at which is deploying.
So you should have a CI/CD process in place. We will typically use Azure DevOps for building the code, creating packages, and then automatically kicking off a release to push the package out to a dev environment.
Then when ready we typically promote that to the next environment (QA, staging, production, etc.).
Part of your release process would be to deploy the code to your Web Server farm (use Octopus Deploy), then follow with deployment of your Test VM, initiating testing, and then (if testing was successful) adding that VM back to the load balancer. Again all of this would be done as part of your release process in your CI/CD pipeline.
Colin Pear
Principal Software Architect
Clear Measure
Hi Colin
Thank you for your feedback. We use Azure DevOps on premises for CI/CD. As part of the pipeline we do execute tests after DEV before we go to UAT and again after UAT before we go to prod. So this prevents us to promote the changes to another environment if it has errors. However during the prod deployment when we deploy to several load balanced servers we want to execute a test again just before we bring a server back online and continue deploying to the other prod servers.
I don’t think we can do that outside of Octopus . In octopus we have a rolling deployment setup so it’s looping through our prod servers. So during the production deployment we want to run the tests again on every prod server before we add the server back to the load balancer.
You dont necessarily need to do them one at a time.
Again consider this. Allow the flow to be controlled at the Azure Devops level.
Take half of the prod servers offline.
Deploy updates to them with Octo
Test them.
Turn them back on.
Repeat the process for the other half.
You can tag half the servers as green and the other half as blue. This would allow you to deploy to half at a time with Octo.
I dont now how many servers you have but you could also do a true blue green deployment. You could have a set of prod (blue) and pre-prod (green) VM’s. You would deploy to whichever group was pre-prod which in this instance would be green. Test them. Then if successful swap blue and green so that green is now production and blue is now pre-prod. This also gives you a quick opportunity to roll back if for some reason the new deployment has issues.
You can take this even one step further and use terraform or ARM templates (IaaS). Stand up a new clean VM. Deploy to it. Test it. Then repeat. When ready swap the new VMs in and the old ones out.
Maybe you misunderstood our environment. Are you using Devops Release? We are only using Devops build . All release work is done on Octopus. I don’t think we should take servers off line during a build. I wouldn’t even know how to do the things you describe for all servers in TFS build.
