Run steps in parallel + logic

ruf-f
(Clayton Taylor) #1

I have a few deployments setup where there are several steps in the process. I currently have them ordered in the most efficient way as far as what needs to wait on something else and what can run in parallel. One block I seem to consistently run into is having steps that could run in parallel with certain steps, but not others. The problem being that steps can only be run in parallel with the previous step so everything is somewhat forced to be linear. One way to solve this would be to add a “Run in parallel after…” that takes a step name/number.

Are there any current plans for something similar or perhaps an existing solution that I haven’t thought of?

(Michael Richardson) #3

Hi Clayton,

There are no current plans for that.

The only work-around (if that is the right word?) is to partition your steps into the groups which can run in parallel with each other (I’m sure there is a fancy set-theory term for this). For example, if steps A, B, and C can run in parallel, and steps D,E, and F can run in parallel, your process would be

A
==B
==C

D
==E
==F

But of course this gets tricky to maintain because then you add step G which can run in parallel with steps A, B and F, but not C, D, or E. Etc…

I’m not even sure your “Run in parallel after” proposal would solve this? If you have an example that you could explain we would be interested to read?

(Clayton Taylor) #4

Thanks for the reply.

As an example, we have services that get deployed to multiple clusters related to the same application because we are in the process of changing architecture (which is a multiple year goal considering the scope). Our current process is something like:

  1. Check Health
    a. Target 1
    b. Target 2
  2. App 1 Target 1
  3. ==App 1 Target 2
  4. ==App X Target 2
  5. App 2 Target 1
  6. App X Target 1

This allows us to deploy all apps in one architecture in sequence and another in parallel (as one is supported and one isn’t). I would like to be able to run these in the same process because they are the same system, but have two simultaneous “paths” if you will. Not sure whether this gets requested often, but this would also be especially nice for setting up testing flows inline rather than towards the end of processes.

Thanks