We recently added some machines running windows server 2019, and the latest windows updates appear to have broken the way calamari logs in to private docker image registries.
From what I’ve been able to figure out, it seems like the docker command is writing to stderr, which is now causing powershell to throw an exception, and the deploy step fails.
In attempt to reproduce this in the simplest form, I’ve run the following script on 2 different windows server 2019 machines, one with a slightly newer version of powershell (and I don’t believe there are any other significant differences). You can see the result in the attached image.
(Get-CimInstance Win32_OperatingSystem).version;
$PSVersionTable.PSVersion;
echo "my_secret" | docker login --username octo_user --password-stdin ***.azurecr.io
This problem is much worse with scripts that attempt to run any docker-compose commands, since it writes to stderr even for saying that services are up to date. I’ve been having a hard time trying to find a good work around for this issue, beyond running docker commands via a segregated command block with the error action set to silently continue (which of course will hide any actual errors that might happen). Even attempting to pipe stderr to stdout doesn’t work because of its own unrelated quirk.
Any suggestions?
Hi Michael,
Thanks for those simple steps to reproduce. I think you got pretty close here! Could you please try this tweak to your minimal script?
(Get-CimInstance Win32_OperatingSystem).version;
$PSVersionTable.PSVersion;
echo "my_secret" | cmd /c "docker login --username octo_user --password-stdin ***.azurecr.io 2>&1"
The PowerShell quirk that you mentioned is annoying, but hopefully the tweak above is enough to get that output sent over to stdout and get you back up and running.
Please let us know how you go with that, or if there’s anything else we can help with.
Thanks,
Roy
Yes, using cmd /c does infact work to prevent the error. I ended up going with something similar, but trying to keep it all as powershell:
Powershell -NoLogo -NonInteractive -Command {
param([string] $composeFolder)
$ErrorActionPreference = "SilentlyContinue";
Write-Host "Starting containers";
docker-compose -f $composeFolder\docker-compose.yaml up -d 2>&1
exit 0;
} -args $composeFolder
I went with writing it as a docker compose file so it would be easier to maintain this way. Attempting to resolve the problem with these work-arounds makes it so we can’t use the actual steps built in to Octopus, which I would prefer over creating my own scripts.
Hi Michael,
I wanted to let you know that a fix for this issue when using Octopus Docker steps has since shipped with Octopus 2020.2.8, and you can find the latest version of Octopus available for download here: https://octopus.com/downloads.
Kind regards,
Roy
1 Like