Change a step to be child of another - is it possible

vsaireddy · 14 October 2016 17:47

Hello,
We have a deploy database step and a bunch of different package deploy steps. Currently they are all setup to run in parallel. We would like the database step to succeed and then run all the packages in parallel. Is it possible to achieve this by setting up the package deployments as children of the database deploy steps. Also, I am only able to add a child step to existing step but cannot seem to find a way to make an existing step a child of another step. Is this possible.
So, to summarize:

Will setting up the steps as child of database deployment allow them to be run only when the database deployment is successful.
Is it possible to move the existing steps to be child of the database step ?

Thanks,
Vishal

Dalmiro · 19 October 2016 13:25

Hi Vishal,

Thanks for reaching out!

1. Will setting up the steps as child of database deployment allow them to be run only when the database deployment is successful.

Not necesarily. Is this database step running on a single database, and then the package steps are running on multiple machines? or is the database step running on multiple locations which are the same as the targets for the package steps?

2. Is it possible to move the existing steps to be child of the database step ?

This is not possible at the moment. The only way is to re-create the step as a child. There’s a uservoice suggestion open to add support for this if you want to drop some votes on it: https://octopusdeploy.uservoice.com/forums/170787-general/suggestions/7948041-move-steps-as-child-steps-into-another-step

Best regards,
Dalmiro

vsaireddy · 1 May 2017 14:44

Hello,

Just wanted to followup on this.

The database step is running on a single database and then the other steps are running on mutliple targets each.

Is there a way to setup the database step in such a way that if this step fails , the subsequent steps will not get executed until this is resolved ?

Thanks
Vishal

Daniel_Fischer · 2 May 2017 01:45

Hi Vishal,

Thanks for getting in touch! We have just the feature to do what you need here. You are able to enable Guided Failure mode in the advanced options after you have created a release, just before you deploy. The Guided Failure option will prompt you with a manual intervention when a step fails, pausing the deployment and giving you the option to either Fail, Retry, or Ignore.

We have some great documentation with instructions on how you can use the Guided Failure feature.

Let me know if this helps you or if you have any further questions here.

Best regards,
Daniel

vsaireddy · 2 May 2017 11:47

I do have guided failure enable on this project. But I am not sure if that will resolve the issue. Let me clarify what we are looking for.

I would like the first step (an email to be sent out after all packages are acquired) to go through.
Then I want the database deployment to run. (This is a common database that is used by about 20 or so different applications).
If there is no error in the database deployment step, I would like those 20 applications (to about 100 servers) to run in parallel without a failure in one application preventing others from being deployed. We do have a very tight downtime.
At the same time, if one failed, I would like to be able to manually correct the problem and retry that step (which is what we are doing right now).
Even if a failure in one would stop the rest of the applications from getting deployed, that is probably acceptable.

I think it helps to think of this process like building a visual studio solution having multiple projects with every project having a set of dependencies. The build sequence and parallelization is figured out based on all these dependencies. Ideally, we should be able to specify any number of steps as prerequisites for each step and the ordering and parallel execution should be determined based on this.

To summarize, our rules are:

 No application can get deployed until the database step completed successfully (with ability to manually intervene).

 Once database is done, rest of the applications should run in parallel .

Please let me know if you have any suggestions on setting up such a scenario. Your help is much appreciated.

Thanks,
Vishal

Daniel_Fischer · 3 May 2017 06:30

Hi Vishal,

Thanks for getting back! I understand what you are after now. This is entirely possible. So running your deployment in guided failure mode is the way to do the manual intervention.

To run the subsequent steps in parallel, you will need to configure those steps into a rolling deployment. A rolling deployment will allow you to deploy to all of your targets in parallel and can be applied to only the steps you require.

We have some fantastic documentation that will help you achieve this: https://octopus.com/docs/patterns/rolling-deployments

To enable your rolling deployment you need to select the Configure a rolling deployment option on your step. From there, you can configure the Window Size which will determine how many targets it can deploy onto simultaneously. Then you can add a child step to the step configured to run in parallel. After adding the child step, the deployment process will now show the step as containing multiple actions, these will all run in parallel as per your window size.

Please let me know if you run into any issues here or if I have missed something.

Best regards,
Daniel

vsaireddy · 3 May 2017 22:10

Hi Daniel,

Thanks for your response.

I could not quite figure out how to set this up.

Let me give an example and you could show me how the steps can be configured with the rolling deployments and child steps.

Please see the below hierarchy (I attached an image of the same in case the text displays incorrectly in your viewer. Sorry I could not provide a better image of the hierarchy.) .

Each row represents one application and has a separate step. The first number represents the current step number and the number in brackets at the end shows the prerequisite steps to be completed before starting this step. [Prequisite Steps: ] . Please note that row 13 needs both 8 and 11 to be completed.
The names in parenthesis () are list of servers in each role.

-1. Database (DatabaseServer1) [Prerequisite Steps: None]
|_2. WindowsApplication1 (WindowsApplicationServer1, WindowsApplicationServer2, WindowsApplicationServer3, WindowsApplicationServer4) [Prerequisite Steps: 1]
|_3. WindowsApplication2 (WindowsApplicationServer5, WindowsApplicationServer6) [Prerequisite Steps: 1]
|_4. WindowsApplication3 (WindowsApplicationServer7, WindowsApplicationServer8) [Prerequisite Steps: 1]
|_5. Webservice1 (WebServiceServer1, WebServiceServer2) [Prerequisite Steps: 1]
|_6. WebSite11 (WebSiteServer1, WebSiteServer2) [Prerequisite Steps: 5]
|_7. WebSite12 (WebSiteServer1, WebSiteServer2) [Prerequisite Steps: 5]
|_8. Webservice2 (WebServiceServer1, WebServiceServer2) [Prerequisite Steps: 1]
|_9. WebSite21 (WebSiteServer1, WebSiteServer2) [Prerequisite Steps: 8]
|_10. WebSite22 (WebSiteServer1, WebSiteServer2) [Prerequisite Steps: 9]
|_11. Webservice3 (WebServiceServer3, WebServiceServer4) [Prerequisite Steps: 1]
|_12. WebSite31 (WebSiteServer3, WebSiteServer4) [Prerequisite Steps: 11]
|_13. WebSite32 (WebSiteServer3, WebSiteServer4) [Prerequisite Steps: 8, 11]

Could you please guide me through the following questions.

 Which steps should be configured as child steps.

 Which ones should be configured as rolling build.

 What is the batch size I need to use on each step ( note that we do need all the servers on each line to be deployed to simultaneously for each application as much as possible.)

Thanks,
Vishal Saireddy

Daniel_Fischer · 8 May 2017 04:45

Hi Vishal,

Thanks for getting back! I’m sorry for the delay in responding to you.

It looks like I have miss-communicated in my initial response and created some confusion here. By default, Octopus will deploy each to all targets simultaneously, in parallel, the rolling deployment is for controlling how many targets are deployed to simultaneously. For the most part, this is used for when you want to remove a target from a load balancer, run the step, then enter it back into the load balancer.

Running your steps in parallel will cause them to deploy based on the window size, so if you have step 3 and 4 all as a rolling deployment, with a window size of 1, they will deploy to one target at a time. Your deployment will still deploy step by step, the window size will just affect the rate in which it deploys to the targets in an environment.

If you would like each of your steps to run in parallel then you do not need to configure a rolling deployment. Octopus will, by default, run each step completely and simultaneously in the order you have them. However, Octopus can not deploy more than one step at a time.

-1. Database (DatabaseServer1) [Prerequisite Steps: None]
|_2. WindowsApplication1 (WindowsApplicationServer1, WindowsApplicationServer2, WindowsApplicationServer3, WindowsApplicationServer4) [Prerequisite Steps: 1]
|_3. WindowsApplication2 (WindowsApplicationServer5, WindowsApplicationServer6) [Prerequisite Steps: 1]
|_4. WindowsApplication3 (WindowsApplicationServer7, WindowsApplicationServer8) [Prerequisite Steps: 1]
|_5. Webservice1 (WebServiceServer1, WebServiceServer2) [Prerequisite Steps: 1]
    |_6. WebSite11 (WebSiteServer1, WebSiteServer2) [Prerequisite Steps: 5]
    |_7. WebSite12 (WebSiteServer1, WebSiteServer2) [Prerequisite Steps: 5]
|_8. Webservice2 (WebServiceServer1, WebServiceServer2) [Prerequisite Steps: 1]
    |_9. WebSite21 (WebSiteServer1, WebSiteServer2) [Prerequisite Steps: 8]
    |_10. WebSite22 (WebSiteServer1, WebSiteServer2) [Prerequisite Steps: 9]
|_11. Webservice3 (WebServiceServer3, WebServiceServer4) [Prerequisite Steps: 1]
    |_12. WebSite31 (WebSiteServer3, WebSiteServer4) [Prerequisite Steps: 11]
    |_13. WebSite32 (WebSiteServer3, WebSiteServer4) [Prerequisite Steps: 8, 11]

It looks like the order you have given me will work fine, if you are receiving any errors here, please attach a deployment log, I would be happy to investigate.

Instructions on retrieving the deployment log:

Sorry if I caused any confusion here, please let me know if you have any further questions here, or if I have something wrong.

Best regards,
Daniel

vsaireddy · 8 May 2017 05:21

Hi Daniel,

Thanks for taking the time to review this. No problem at all. I understand that communicating this issue is complicated over email. Maybe it will be easier to just have an online meeting if possible. Let me know if this can be arranged. I can send you a skype link if we arrange for a time ( I work in US Central Time Zone, but I can get on the call at any prearranged time even outside of normal business hours as I know you are in Australia).

Meanwhile, continuing on with the email:

As you requested, please see the attached log. If you look at the time stamps for the start of each step, you will notice that step2 (deploy database) ended at 21:59:38. A bunch of the subsequent steps have started right after that.
This indicates that steps 3 onwards

```
 waited for step 2 to complete.
```
```
 ran in parallel.
```

In my step configuration, I have step 3 setup as follows:
Run Condition = Run Always,
Start Trigger = Wait for previous step to complete, then start
Step 4 onwards are setup
Run Condition = Run Always,
Start Trigger = Run in parallel with previous step.

Based on the above configuration, it seems logical that step 3 waited for 2 to complete. Once step 3 started the rest of them started too.

So far so good.

In this case there was no error on step 2.

However, in cases where there was an error on step 2, I would like step 3 to not start at all. So to achieve this, I had tried setting step 3 like below:
Run Condition = Success, run only when the previous steps are successful,
Start Trigger = Wait for previous step to complete, then start

The subsequent steps were not changed.

What ended up happening was that when there was an error in step 2, step3 did not run (which is correct) , but steps 4 onwards still went ahead. This is incorrect. Unfortunately I do not have a log at this point with this setting as my current setting is what I described first (step 3 set to always run). I am changing this now to “Success, run only when the previous steps are successful”. I will send you the log as soon as I run into a failure situation on step 2.

Thanks again for your efforts in helping with this.

Thanks,
Vishal

Octopus_ServerTasks-35831.log.txt (396 KB)

Daniel_Fischer · 9 May 2017 03:02

Hi Vishal,

Thanks for getting back with that! I think I understand now, thanks for the extra explanation. From what I can see, the reason it is continuing with step 4, even though step 3 is set to only run on success of step 2 is due to the guided failure mode being enabled.

For you to be able to do what you are after here, you will need to set the rest of your steps, 4onwards to Run Condition = Success, run only when the previous steps are successful otherwise you will get this behavior.

I don’t think there are any other options available here. With guided failure mode, the deployment will simply keep going unless that run condition is set on all the subsequent steps.

If you find this is still not a solution to your issue, I am more than happy to do a call/screen share with you.

However, I do hope the above does help you resolve this. Let me know!

Best regards,
Daniel