We have an application that we’ve been running for a while. Recently (within the last month) we added some new steps for smoke testing our recently deployed application which required the deployment of some tools. When testing our latest release candidate we realized we forgot to do some deploy testing (besides the smoke testing, that is) and so we decided to revert to the latest release build, which is from before the summer.
Deployment of said release failed because there was no smoke test application package with that version number.
The workaround for our particular problem is easy enough (just exclude the smoke test steps when deploying) but the mere fact that a deployment can fail because the release process was changed after the creation of the release you’re trying to deploy makes me afraid. Our current change is possible to work around because it’s just packages missing but what if we modify existing build steps to rely on behaviour that exists in our new but not our old packages? Maybe we’ve removed applications that the old version relied upon? Then all of a sudden, we can’t revert to an old version without recreating the old way of deploying things from memory or by setting up a mirror system from a backup.
Is it possible to ensure that releases remember the build steps as well as the variables and package versions? It seems to me that not doing so is inherently dangerous and insecure. That the only way to change the deployment steps is to modify things one step at a time directly in the system used to deploy to production with nothing to stop CloudOps from starting such a deployment when you’re in the middle of reconfiguring things (save for walking around telling people not to deploy anything until you’re done) just exacerbates that feeling.