Running smoke tests after deploy

Legacy Uncategorised
1

Is there a any story around running smoke tests (integration tests) after deployment, and having that reflected somehow in the Octopus UI? Maybe running nunit.exe or something from the PostDeploy.ps1?

2

Hi Rob,

It’s a feature I’m planning to support soon. For now, you could manually invoke Nunit.exe from PostDeploy using a special-purpose package, and then echo the NUnit results to the PowerShell script output.

Going forward I imagine you’ll be able to ‘attach’ the NUnit results to a deployment and visualise a summary of the test run in the UI on the release/deployment details pages.

Paul

3

We use mstest.exe to execute our selenium ui tests as the last part of deploy.ps1, we then fail the deployment if the tests fail. Finaly we hoover up the log file so you can see what went wrong in the octopus ui.

4

May I ask, but isn’t that beyond the scope of octopus? Building and testing would probably be in the remit of a CI system whereas octopus is for deployment no?

Or have I misunderstood the remit of the app?

Thanks

Richard

5

Richard,

Build and test are generally best handled by the CI tool, yes. But some tests (smoke tests, automated UI tests) might be better handled by Octopus. Currently we don’t do much to support this (Matt’s example is the only way to do it at the moment) but it’s something we plan to improve support for in future.

Paul

6

Any progress on launching tests, consuming results?

Thanks

7

Hi Gavin,

We’re planning to add this in Octopus 2.0. If you’d like to share more about how you’d like the feature to work, please share on this thread and we’ll make sure to consider it.

Paul

8

Since the last response was in July and was hopefully going to be available in 2.0, and anything else related was closed (or still listed as an idea to vote on)
http://help.octopusdeploy.com/discussions/problems/12323-implementing-full-ci-with-test-automation-based-on-builds
and here:


and this:

In the second link it mentions Octopus attaching test results as a step as an artifact. That sounded like it might be automatic, but no further discussion I’ve seen indicates that happened. Also I know that specifically one of the things our development team would want is the ability to see it trend over time (which would be problematic if it’s attached to an artifact to a particular release.)

9

Hi - you can attach results to a deployment with the New-OctopusArtifact PowerShell command like so:

New-OctopusArtifact .\TestResults.xml

Including the machine name in the filename is currently the only visible way to track the origin of each result file - we’d like to improve this in the future.

10

Has there been any progress in this area?

It would be really useful for us.

11

Hi Brian,

Your question seems very related to what you asked on this other thread. I’m gonna close this 3-year-old thread and lets continue our conversation on that other ticket, ok?

Cheers,
Dalmiro

12

:slight_smile:

I’m not Brian, but if I click on that other link I can’t see it.

Mike Peterson
DevOps Engineer
Preventice Solutions Group

3033 41st Street NW
Suite 330
Rochester, MN 55901
Office: 507-218-3053
Mobile: 507-251-2315
mpeterson@preventice.com

Preventice Services, LLC and Preventice Technologies, Inc. are part of the Preventice Solutions Group of companies and are independent, wholly owned subsidiaries of Preventice Solutions, Inc.

This message contains confidential information and is intended only for the sole use of the addressed recipient(s). If you are not one of these recipients you should not disseminate, distribute or copy this e-mail. Please notify the designated recipients immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmissions cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender, therefore, does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version.

From: Dalmiro Grañas tender2+de7e5b393a2500da67aa02744a3e44d4922c7d4c2@tenderapp.com
Date: Monday, October 24, 2016 at 11:56 AM
To: Mike Peterson mpeterson@preventice.com
Subject: Re: Running smoke tests after deploy [Questions #214]

13

That’s because that link is private and only Brian and our staff can see it.

Lets keep this thread closed and if anyone has any questions about a related subject, they can just create a new one with fresh info.