Good morning @manikantanmca,
Thank you for that information, I did see you had set that variable in one of your other forum posts but thank you for confirming that is set on your instance.
I have got some information back from our engineers which I have posted below:
IIS doesn’t like being modified by two different processes in parallel. All sorts of things happen from config file corruption to hangs. We go to some lengths to put locks around our steps that modify IIS so they run in sequence when required. However, if they have other steps from the library or self written that will cause problems.The `Cannot start this IIS website related task yet’ message is printed when we are waiting on that lock mentioned above. It’s possible if a step terminates that the lock is left hanging (and why a reboot helps).
The engineers have looked at your dumps and unfortunately none of those point the engineers in the right direction, they have asked if you would not mind providing us with the dumps for this specific powershell (the one running inside Calamari)?
That way they can check what command is actually being run in each of the deployments.
The engineers do agree it’s environmental - IIS deployments will run sequentially regardless of whether they’re set up as parallel or not, so it’s pointless to run them parallel. All the IIS operations would just get stuck behind each other. So we’d expect to see one deployment doing something IIS-related and everything else waiting on a mutex.
The Tentacle mutex is more robust than the IIS mutex so the engineers have said customers should not be running these deployments in parallel, it would only cause more harm than good. We do allow turning this off, but we do warn it is at their own risk due to this kind of problem.
I hope this helps, if you are happy to provide us with those dumps we can look into them for you, but I am afraid the above information does coincide with what you are experiencing (deployments getting stuck as they are waiting on others). And so this would suggest you may need to separate your projects and deploy say 10 web applications at once (which you have suggested works) instead of 30 to ensure you are consistently deploying properly and not having to potentially kill a process if one of the 30 deployments gets stuck.
I hope this helps, please let me know how you would like to proceed, if you would like us to look at the dumps for you I can create you another secure link for you to upload them to. If you are happy trialling separating those applications out into different projects and see if that works better for you we are happy to help advise.
I look forward to hearing from you,