Deploy azure cloud service: Forbidden error

We’re trying to create a new cloud service through an Azure Powershell step. The following error is shown in the task log:

Task ID:       ServerTasks-813
Task status:   Failed
Task queued:   donderdag 24 september 2015 16:05
Task started:  donderdag 24 september 2015 16:05
Task duration: 29 seconds

                    | == Running: Deploy Controller Service release 3.0.52.25 to Dev ==
16:05:22   Verbose  |   Guided failure is not enabled for this task
16:05:51   Error    |   The deployment failed because one or more steps failed. Please see the deployment log for details.
                    | 
                    |   == Failed: Step 1: Create cloud service ==
16:05:22   Verbose  |     Using account ID 'azuresubscription-cross-point-dev'
16:05:24   Verbose  |     Loading certificate with thumbprint: ===================
16:05:24   Verbose  |     Certificate was found in store
16:05:25   Verbose  |     Azure context parameters:
16:05:25   Verbose  |     Subscription ID:       ===================
16:05:25   Verbose  |     Subscription name:     ===================
16:05:25   Verbose  |     Importing Windows Azure modules
16:05:27   Verbose  |     Invoking target script C:\Data\Octopus\Work\20150924140522-51\Script.ps1
16:05:27   Info     |     Checking if the cloud service =================== already exists.
16:05:40   Info     |     Creating the cloud service =================== in affinity group westeu-services
16:05:50   Error    |     New-AzureService : ForbiddenError: The server failed to authenticate the reques
16:05:50   Error    |     t. Verify that the certificate is valid and is associated with this subscriptio
16:05:50   Error    |     n.
16:05:50   Error    |     At C:\Data\Octopus\Work\20150924140522-51\Bootstrap.Octopus.AzureContext.ps1:18
16:05:50   Error    |     7 char:9
16:05:50   Error    |     +         New-AzureService -ServiceName $name -AffinityGroup $affinityGroup -La
16:05:50   Error    |     bel ...
16:05:50   Error    |     +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
16:05:50   Error    |     ~~~
16:05:50   Error    |     + CategoryInfo          : CloseError: (:) [New-AzureService], ComputeCloud
16:05:50   Error    |     Exception
16:05:50   Error    |     + FullyQualifiedErrorId : Microsoft.WindowsAzure.Commands.ServiceManagemen
16:05:50   Error    |     t.HostedServices.NewAzureServiceCommand
16:05:50   Fatal    |     The step failed: The remote script failed with exit code 1
16:05:50   Verbose  |     Create cloud service completed
                    |   
                    |   Cancelled: Acquire packages
16:05:50   Verbose  |     Step "Acquire Packages" runs only when all previous steps succeeded; skipping
                    |   
                    |   Cancelled: Step 2: Deploy to EU
16:05:50   Verbose  |     Step "Deploy to EU" runs only when all previous steps succeeded; skipping

Somehow authentication to Azure fails. What we have tried:
Re-genereate the certificate and imported it in the portal again
Run the Octopus service under different account
Generate a certificate on the command line and imported it into octopus
Re-create the azure account in Octopus
Runned the command by hand in powershell which works fine

We’re using Octopus 3.1.1.

Hi Erwin,

Thanks for getting in touch. It’s really cool that you’re making good use of the Azure PowerShell Step in combination with the Cloud Service Step, I see that being a really common approach to deploying feature branches etc.

Thanks for going into depth with your troubleshooting. In this case the best suggestion I can make is that you either:

  1. Haven’t uploaded the “Management Certificate” for that Account into the Azure Portal: https://manage.windowsazure.com/#Workspaces/AdminTasks/ListManagementCertificates
  2. Have uploaded the “Management Certificate” to the wrong Subscription - easy to do if you have multiple Subscriptions - speaking from experience!

Personally I just use the “Management Certificate” that Octopus generates for you on the Account page.

If you can confirm that the Account, Subscription, Thumbprints etc all align, and it still isn’t working I’d be happy to investigate this further with you.

Hope that helps!
Mike

Hi Michael,

Thanks for the response. I’ve read these topics and double (triple :wink: ) checked if I was using the right certificate and if I’ve uploaded it to the right subscription. I’ve disabled the custom script step now and added a ‘Deploy Azure Cloud Service’ step instead. I’ve added the log below.

The strange thing is that uploading the package to the blob succeeds, but executing the New-AzureDeployment call gives the forbidden error.

09:18:35   Verbose  |     Using account ID '=======================-dev'
09:18:59   Info     |     Deploying package:    C:\Data\Octopus\OctopusServer\PackageCache\feeds-teamcity-packages\=======================.nupkg
09:19:03   Verbose  |     Extracting package to: C:\Data\Octopus\Work\20150928071835-64
09:19:04   Verbose  |     Extracted 6 files
09:19:04   Verbose  |     Ensuring cloud-service-package is V20120315 format.
09:19:05   Verbose  |     Package is Legacy format. Converting to V20120315 format.
09:19:13   Verbose  |     Extracting Cloud Service package: 'C:\Data\Octopus\Work\20150928071835-64\=======================.cspkg'
09:19:16   Verbose  |     Azure Cloud Service Configuration file (*.cscfg) not found: C:\Data\Octopus\Work\20150928071835-64\ServiceConfiguration.Dev.cscfg
09:19:16   Verbose  |     Found Azure Cloud Service Configuration file: C:\Data\Octopus\Work\20150928071835-64\ServiceConfiguration.Cloud.cscfg
09:19:16   Verbose  |     Updating configuration settings...
09:19:16   Info     |     Updating setting for role =======================: service.maxInitializationWaitTime = 00:05:00
09:19:16   Verbose  |     Looking for appSettings and connectionStrings in any .config files
09:19:17   Info     |     Updating appSettings and connectionStrings in: C:\Data\Octopus\Work\20150928071835-64\LocalContent\=======================\approot\=======================.dll.config
09:19:17   Verbose  |     Setting 'ServiceUsername' = '======================='
09:19:17   Verbose  |     Setting 'ServicePassword' = '********'
09:19:17   Info     |     No matching setting or connection string names were found in: C:\Data\Octopus\Work\20150928071835-64\LocalContent\============================\base\x64\WaHostBootstrapper.exe.config
09:19:17   Info     |     No matching setting or connection string names were found in: C:\Data\Octopus\Work\20150928071835-64\LocalContent\============================\base\x64\WaWorkerHost.exe.config
09:19:17   Info     |     No matching setting or connection string names were found in: C:\Data\Octopus\Work\20150928071835-64\LocalContent\============================\base\x86\WaHostBootstrapper.exe.config
09:19:17   Info     |     No matching setting or connection string names were found in: C:\Data\Octopus\Work\20150928071835-64\LocalContent\============================\plugins\RemoteAccess\RemoteAccessAgent.exe.config
09:19:17   Verbose  |     Re-packaging cspkg.
09:19:20   Info     |     Uploading package to Azure blob storage: 'C:\Data\Octopus\Work\20150928071835-64\=======================.cspkg'
09:19:22   Verbose  |     Loading certificate with thumbprint: =======================
09:19:22   Verbose  |     Certificate was found in store
09:19:48   Verbose  |     Uploading, response received: 202 Accepted
09:19:48   Verbose  |     Uploading the package to blob storage. The package file is 6 MB.
09:19:51   Verbose  |     Uploading package to blob storage: 1024 KB of 6 MB
09:25:03   Verbose  |     Uploading package to blob storage: 2 MB of 6 MB
09:25:04   Verbose  |     Uploading package to blob storage: 3 MB of 6 MB
09:25:04   Verbose  |     Uploading package to blob storage: 4 MB of 6 MB
09:25:04   Verbose  |     Uploading package to blob storage: 5 MB of 6 MB
09:25:05   Verbose  |     Uploading package to blob storage: 6 MB of 6 MB
09:25:05   Verbose  |     Uploading package to blob storage: 6 MB of 6 MB
09:25:06   Verbose  |     Upload complete
09:25:06   Info     |     Package upload complete
09:25:06   Info     |     Package uploaded to https://==============.blob.core.windows.net/octopuspackages/=======================.3.0.53_22c85fbab08db4a26dc49dc407286814f56dafc6.cspkg
09:25:06   Info     |     Config file: C:\Data\Octopus\Work\20150928071835-64\ServiceConfiguration.Cloud.cscfg
09:25:07   Verbose  |     Loading certificate with thumbprint: ============================
09:25:07   Verbose  |     Certificate was found in store
09:25:21   Verbose  |     Azure context parameters:
09:25:21   Verbose  |     Subscription ID:       =======================
09:25:21   Verbose  |     Subscription name:    =======================
09:25:21   Verbose  |     Importing Windows Azure modules
09:25:30   Verbose  |     Invoking target script C:\Data\Octopus\Work\20150928071835-64\DeployToAzure.ps1
09:25:48   Verbose  |     Creating a new deployment...
09:26:00   Error    |     New-AzureDeployment : ForbiddenError: The server failed to authenticate the req
09:26:00   Error    |     uest. Verify that the certificate is valid and is associated with this subscrip
09:26:00   Error    |     tion.
09:26:00   Error    |     At C:\Data\Octopus\Work\20150928071835-64\DeployToAzure.ps1:75 char:5
09:26:00   Error    |     +     New-AzureDeployment -Slot $OctopusAzureSlot -Package $OctopusAzurePackage
09:26:00   Error    |     Uri ...
09:26:00   Error    |     + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
09:26:00   Error    |     ~~~
09:26:00   Error    |     + CategoryInfo          : CloseError: (:) [New-AzureDeployment], CloudExce
09:26:00   Error    |     ption
09:26:00   Error    |     + FullyQualifiedErrorId : Microsoft.WindowsAzure.Commands.ServiceManagemen
09:26:00   Error    |     t.HostedServices.NewAzureDeploymentCommand
09:26:00   Error    |     Script 'C:\Data\Octopus\Work\20150928071835-64\DeployToAzure.ps1' returned non-zero exit code: 1
09:26:00   Error    |     Running rollback conventions...
09:26:00   Error    |     Script 'C:\Data\Octopus\Work\20150928071835-64\DeployToAzure.ps1' returned non-zero exit code: 1
09:26:01   Fatal    |     The step failed: The remote script failed with exit code 1
09:26:01   Verbose  |     Deploy to EU completed

I’ve tried this on our other subscription but I get another error. The strange thing is that it says it is loading certificate starting with DF4430E0 but the response on the API call says it cannot find the certificate starting with 0D0D4ABC.

Package uploaded to https://octopusacccrosspoint.blob.core.windows.net/octopuspackages/===================.3.0.53_d469f7a4f059995b59e8f97a22e60b221b5d7614.cspkg
10:12:41   Info     |     Config file: C:\Data\Octopus\Work\20150928081204-70\ServiceConfiguration.Cloud.cscfg
10:12:41   Verbose  |     Loading certificate with thumbprint: DF4430E0===================
10:12:41   Verbose  |     Certificate was found in store
10:12:42   Verbose  |     Azure context parameters:
10:12:42   Verbose  |     Subscription ID:       ===================
10:12:42   Verbose  |     Subscription name:     =================== Acc
10:12:42   Verbose  |     Importing Windows Azure modules
10:12:44   Verbose  |     Invoking target script C:\Data\Octopus\Work\20150928081204-70\DeployToAzure.ps1
10:12:58   Verbose  |     Creating a new deployment...
10:13:09   Error    |     New-AzureDeployment : BadRequest: The certificate with thumbprint 0d0d4abc=====
10:13:09   Error    |     =================== was not found.
10:13:09   Error    |     OperationID : '55a9dca9cf2e1440b72451d9370068b2'

The problem with the different certificate was mentioned in my last reply was related to the certificate in the Azure package file which was not available in the cloud service.

I’ve reinstalled Octopus server (also updated to v3.1.2) and the problem seems to be solved as I can access the api now.

Thanks, Erwin

Hi Erwin,

Thanks for keeping me in the loop. I’ve been away on vacation, hence my lack of replies until now.

So to confirm I understand what went wrong: your CSCFG file referenced a Certificate that wasn’t available to the Cloud Service? That makes sense.

So it seems like you’re up and running happily, and I’m glad that you were persistent enough to figure it out independently!

Happy Deployments!
Mike