We are trying to do a simple rolling deployment and at the end of the bash script issue a reboot of the server. We continue to get a failed process due to “failed with error ‘Connection to the target has been lost while attempting to read’”
Any suggestions on how we can accomplish this task?
Thanks for getting in touch!
There is a way to mark a past deployment as successful however, by design this is a manual process. This can be done on the Deployment task screen by selecting the overflow menu in the top right corner and choosing
Edit State :
You can then change the state of the task to
Success and input a reason why. I wouldn’t recommend this approach in your case though.
My recommendation would be to configure this project to allow for a restart. This is fairly straightforward, but does require a couple of changes.
Step 1: Allow for the machine to go offline during the deployment
- Go to the
Settings page for the project and change the option to
Skip deployment targets if they are or become unavailable
- Once that is done you can use your existing step to trigger the restart, however you will need to add two more steps to your project after the reboot step.
Step 2: Add a wait Step
- Add a script step which only needs to contain
start-sleep in it along with enough time to allow for the server to reboot. This step must run on the server otherwise it will fail if you try to run it on the deployment target that is restarting.
Step 3: Re-add to the Deployment Manifest
- Once that step completes and the server is [hopefully] back online we need to re-add it to the deployment manifest as it may have been removed during the restart. This is done by adding a
Health Check step which will re-add the machine to the available targets. Once you have added the step I would recommend changing two of the options:
- Health check type -> Perform a connection-only test [much faster]
- New Deployment Targets -> Include new deployment targets in the deployment [to re-add the machine if it had been removed]
This should allow your deployment to complete successfully with string of nice green ticks:
Hopefully this helps, please let me know if you have any questions!
Thanks for the article, I had reviewed that a couple months ago and it did not work ro rolling deployments. I think that is the key rolling deployments.
What I have found that seems to work today is:
screen -d -m /root/reboot.sh &
Thank you for taking the time to get back to us with the solution that you have found.
That is a definitely an acceptable way to handle restarting rolling deployments. At the moment, it seems to be one of the better solutions.
In the future, I will be looking to update our documentation with similar details to what you have provided, as well as exploring other solutions to handle this exact situation.