Azure deployments failing since upgrade to 3.11.4

Integration
1

Azure service fabric deployments are failing since upgrade to 3.11.4
The log shows:
13:51:24 Error | Invoke-Expression : Unable to find type [System.Fabric.FabricObjectClosedException].
13:51:24 Error | At C:\Octopus\Work\20170302135107-21\tools\Octopus.AzureContext.ps1:99 char:5
13:51:24 Error | + Invoke-Expression ". $OctopusAzureTargetScript $OctopusAzureTarge …
13:51:24 Error | + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13:51:24 Error | + CategoryInfo : InvalidOperation: (System.Fabric.F…ClosedException:TypeName) [Invoke-Expression], Runt
13:51:24 Error | imeException
13:51:24 Error | + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.InvokeExpressionCommand
13:51:24 Verbose | Updating manifest with output variables
13:51:24 Fatal | The remote script failed with exit code 1
13:51:24 Verbose | at Octopus.Worker.Scripting.ScriptResult.EnsureSuccessful() in Z:\buildAgent\workDir\eec88466c176b607\source\Octopus.Worker\Scripting\ScriptResult.cs:line 87
| at Octopus.Server.Orchestration.Deploy.DeploymentTaskController.<>c__DisplayClass28_0.b__0() in Z:\buildAgent\workDir\eec88466c176b607\source\Octopus.Server\Orchestration\Deploy\DeploymentTaskController.cs:line 320
| at Octopus.Server.Orchestration.Deploy.DeploymentTaskController.ExecuteWithTransientErrorDetection(Action action, Machine machine) in Z:\buildAgent\workDir\eec88466c176b607\source\Octopus.Server\Orchestration\Deploy\DeploymentTaskController.cs:line 424
| at Octopus.Server.Orchestration.Deploy.DeploymentTaskController.ExecuteActionAndInitLoggingContext(PlannedStep step, Machine machine, PlannedAction action) in Z:\buildAgent\workDir\eec88466c176b607\source\Octopus.Server\Orchestration\Deploy\DeploymentTaskController.cs:line 318
13:51:24 Fatal | Get existing application version on the Octopus Server

There is the possibility the nuget packages are not unpacking correctly in the working folder.

2

Hi,

Thanks for getting in touch.

Could you please confirm whether this error occurs only when you deploy using Octopus, or whether the same error occurs when you call the Service Fabric PowerShell cmdlets manually from the machine that Octopus is installed on?

Can you confirm whether the “Microsoft Azure Service Fabric SDK” is installed on the server? Could the SDK have been uninstalled recently?

We’ve seen this exception occur when we’ve uninstalled the Service Fabric SDK and then tried to call the Service Fabric PowerShell cmdlets.

Let me know how you go.

Cheers
Mark

3

Hi,
The only change on the server was the upgrade of octopus from 3.7.7 to 3.11.4
The SF SDK and related powershell cmdlets are still available on the box.
Also when trying to revert back to the previous version of octopus the octopus service throws exceptions and fails to start.

Cheers
Dave

4

Hi,

Has there been a change that affects how octopus handles $psscriptroot?
As part of an azure powershell step a parameter is passed into the script containing $psscriptroot\PublishProfiles\cloud.xml.
Previously this would evaluate to the correct place in the working folder however it no longer appears to resolve. From the raw output:
Previously:
Invoking target script “C:\Octopus\Work\20170210084232-631\tools\GetExistingAppVersion.ps1” with -PublishProfileFile C:\Octopus\Work\20170210084232-631\PublishProfiles\Cloud.xml
Now
Invoking target script “C:\Octopus\Work\20170306153415-16\tools\GetExistingAppVersion.ps1” with -PublishProfileFile $psscriptroot\PublishProfiles\Cloud.xml

Cheers
Dave

5

Hi Dave,

Nice find. We’re not aware of any changes regarding $psscriptroot. We reviewed the list of changes between 3.7.7 and 3.11.4 and can’t see any issues that would have caused this PowerShell variable to no longer exist.

We use this variable in one of our Calamari build scripts and have a fallback in place in case this is missing, which may help in your case:

if(!$PSScriptRoot){
    $PSScriptRoot = Split-Path $MyInvocation.MyCommand.Path -Parent
}

Re. Octopus rollback. If you’re trying to revert back to the 3.7.7 version, you would need to also restore the database, as the minor version change indicates potential database changes that won’t be backwards compatible (more information can be found in the “Downgrading or Rolling Back an Upgrade” section of the upgrading docs).

Could you add that fallback code above to your script and see if it then unpacks correctly?

Cheers
Mark

6

Hi.
I’ll try your suggestion and feedback.
I have tried replacing $psscriptroot with the octopus system variable however this returns access denied on the working folder.

Cheers
Dave

7

Hi.

The suggestion of adding the code into all our script would have been far too much work
However after trial and error and a a lot of digging around i realised $psscriptroot can be swapped out for #{OctopusOriginalPackageDirectoryPath}.
By substituting this variable the deployments are now functioning again.

Cheers
Dave

8

Hi Dave,

Glad you got a solution. Thanks for letting us know.

Cheers
Mark

9

Hi,
Will the issue with psscriptroot no longer functioning be I investigated further as I am wondering if there is likely to be any other similar issues occur with other powershell variables.
Cheers
Dave

10

Hi Dave,

It’s possible. We were unable to reproduce the issue with $PSScriptRoot. If we make a simple script step and include this:

Write-Host "Root is $($psscriptroot)"

We get the expected work directory shown in the log (eg. “Root is D:\Octopus\Work\20170309020857-1162”).

Are you able to confirm if this variable only fails to work when Octopus is running PowerShell, or could this be a common problem on the machine? (ie. If you launch PowerShell ISE on the server and run a script with that variable, does it work there ok?)

Cheers
Mark

11

Hi,

I have tried it directly on the server and there is no issue there.
If its not replicating at your end ill assume its something in the upgrade that wasnt quite right but at least i have got a simple fix so ill leave it at that.
If i come across anything else ill update again.

Cheers
Dave