Hi Clayton,
Hmm, I can see your conundrum. In Octopus we treat each deployment as a separate event, it doesn’t really know about other projects. One thought I had was leveraging the run a script step and API to find if there are active deployments for your specific projects. The script would wait until all projects are completed before kicking off your tests. For example, in this article, I was doing something similar, but waiting for all deployments to a specific machine to wait: How can I force a deployment to wait in Octopus Deploy until another deployment finishes?
That approach has its own pros and cons. Putting that step in the right spot could be tricky, and I could see instances where you would have that step multiple times in the same project. Another question that comes to mind is how often are N of the 12 services being built and deployed at the same time? If it is once every couple of days, would it be easier include retry logic in the script which kicks off the script? Or would the effort be much greater?
Finally, how long do those tests run? Should they always run after each build? The reason I ask that is because I worked on another application a few years ago that typically had between 20-40 builds a day. A test suite which took 20-30 minutes to finish caused a lot of grumbling from the developers. Our compromise was to run the test suite at set intervals throughout the day, at 9 during our stand-up, over lunch, and at the end of the day.
I hope that helps trigger something that can drive towards a solution!