Azure CLI logging errors from progress

I’m really struggling with this one & I’m sure there’s some simple fix I’m missing.

I have an Azure Powershell step which runs a script that includes the az cli command az storage blob upload-batch to upload a bunch of files to a storage account. For reasons I can’t understand, this command writes its progress as errors. You can add the --no-progress command but this just stops it writing pretty progress bars, which are useless from a log file. In a more recent az cli (2.3.1) there is the --only-show-errors switch.

The trouble with trying to reproduce this, is when run interactively there is no indication it’s logging errors. It’s only in Octopus that they get logged as errors.

I’ve tried redirecting the stream using various combinations of 2>&1. Despite what the docs say, this is the only one that sort of worked, in that it failed after the command finished because it didn’t understand the log messages generated by Octopus.

Not working at all:

ParserError: At D:\Octopus\Work\20200722123152-76992-877\Script.ps1:18 char:112 
July 22nd 2020 13:32:05
Error
+ ... ss" -d $targetUrl --destination-path "$($path)css" --no-progress 2>&4 
July 22nd 2020 13:32:05
Error
+                                                                        ~ 
July 22nd 2020 13:32:05
Error
Missing file specification after redirection operator. 
July 22nd 2020 13:32:05
Error
At D:\Octopus\Work\20200722123152-76992-877\Script.ps1:18 char:112 
July 22nd 2020 13:32:05
Error
+ ... ss" -d $targetUrl --destination-path "$($path)css" --no-progress 2>&4 
July 22nd 2020 13:32:05
Error
+                                                                        ~ 
July 22nd 2020 13:32:05
Error
The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it as part of a string. 
July 22nd 2020 13:32:05
Error

Not understanding errors:

NotSpecified: WARNING: uploading D:\Octopus\Work\20200722122547-76990-873\assets\css\animate.css

13:26:06   Error    |       At D:\Octopus\Work\20200722122547-76990-873\Script.ps1:18 char:1

13:26:06   Error    |       + az storage blob upload-batch -s "$basepath\css" -d $targetUrl --desti ...

Hi Simon,

Thanks for reaching out.

This does sound like an interesting one.

Could you please run az --version within a dummy azure CLI step, and also in your interactive powershell window to see if we’re running the same versions? I’m curious as to why we’re seeing different results between the two. Do you currently have bundled tools selected within the azure step, or pre-installed?

Could you also please send the full command you are running including parameters?

Which version of Octopus Server are you currently running?

Thanks,
Jeremy

Octopus Server is 2020.2.16

I’m using the bundled Azure CLI in Octopus, which is 2.0.50
On my machine I’m using a much more recent version 2.3.1.

I’ve also tried using the bundled az cli tool on the Octopus server, but again there’s no indication when you do, that the progress is sent to stdErr.

If you send the output from the az command into a variable, you get the result of the command in there, which is a piece of json describing what you uploaded, but the progress items are not there - they always appear in the console which seems to line up with it using stdErr.

It just seems to be how they wrote az cli. But I’m wondering how I stop Octopus logging it all as errors, even though it’s technically doing the right thing.

I’m loath to upgrade the az cli across all our workers if it’s not going to stop this happening.

I’d also point out that 2.0.50 is quite old - released in November 2018 (https://docs.microsoft.com/en-us/cli/azure/release-notes-azure-cli?view=azure-cli-latest&tabs=azure-cli#november-6-2018)

The current version is 2.9.1

The same goes for the Azure Powershell tools - they’re quite old as well, released at the same time. They won’t be supported any longer after December 2020, having been replaced with the new Az cmdlets which are at version 4.4 (https://docs.microsoft.com/en-us/powershell/azure/new-azureps-module-az?view=azps-4.4.0#continued-support-for-azurerm)

The answer maybe to upgrade all the Azure tooling anyway

Hey Simon,

A colleague and I also tried without success to redirect the warning output from that az.exe command.

Unfortunately, I don’t think the bundled tools will be getting updated in the near future, so if you want to get rid of the warnings, you will probably have to update the AZ tools locally on your workers and switch to using pre-installed tools. You should be able to do this with minimal effort using the Script Console as you can target all workers.

I’m sorry I don’t have better news for you.

Please let me know if you have any other questions or concerns.

Sincerely,
Jeremy

I’ve updates the version of the cli & it works perfectly. I had to tweak the IAM on the account because that reacts differently on newer versions.

I hadn’t realised the Azure tooling was so old in Octopus. Thanks for the tip about using the script console - I hadn’t realised it was this useful.

1 Like

Hey Simon,

Thanks for updating me, I’m really glad to hear it worked well for you. I agree, the script console is definitely a powerful tool for updating or targeting workers with a script.

Have a great rest of your week!

Thanks,
Jeremy