Calling the REST API from an Octopus process step results in 403 forbidden

Octopus Server
1

Hello,
First, I’m sorry if the issue has already been mentioned in the past in another thread, after a quick search I couldn’t find a topic related to the problem I’m facing.

I have an issue when trying to call the Octopus REST API from a project step using PowerShell.

Here is an example of the simple query I’m trying to run :

$header = @{ “X-Octopus-ApiKey” = “my API key” }
$OctopusURL = “http://my-octopus-url”
Invoke-RestMethod -Method Get -Uri “$OctopusUrl/api/Spaces-X/projects/Projects-X/releases” -Headers $header

When the step is executed during a release, here is the error message I get :

InvalidOperation: The remote server returned an error: (403) Forbidden.
(…)
ERROR: The requested URL could not be retrieved
ERROR
The requested URL could not be retrieved
The following error was encountered while trying to retrieve the URL: http://my-octopus-url/api/Spaces-X/projects/Projects-X/releases
Access Denied.

Here are some important details I need to share :

  • The step is run on the Octopus server (defined as the Execution Location in the process’ step)
  • The tentacle on the server is running under the local system account
  • The API key used in the script is associated to an Octopus service account which has sufficient permissions to query the releases of the project (I also tried to put the account as an Octopus administrator during my troubleshooting)
  • The same PowerShell script is returning correct information when ran directly on the server and also from my local machine (access to my Octopus API is done via a corporate VPN)
  • I also tried to use Octopus.Client (from a PowerShell session on the server and from an Octopus release) and ran into the same output (working fine from the server, not from the release process)

I couldn’t find any information so far on the Internet regarding this issue…

Any guidance would be appreciated.

Thank you very much !

3

Hi Johan,
Thanks for reaching out to Support.

I reproduced your process step on our test server and was able to run it successfully. I used this script with only slight modifications:

$header = @{ "X-Octopus-ApiKey" = "API-XXXXXXXXXXXXX" }
$OctopusURL = "https://octopus-server.domain"
Invoke-RestMethod -method Get  -Uri "$OctopusUrl/api/Spaces-xxx/projects/Projects-xxx/releases/"  -Headers $header -UseBasicParsing

I added the -UseBasicParsing (IE compatibility) but that’s just for good practice - it should work without it.

Possibly the URL is where it might be falling down. The structure of the API URL needs to have the correct Project ID. You can get this from this URL (in a browser) https://octopus.server.domain/api/Spaces-XXX/projects . This will output lots of info on each project in your Space including the Project ID. From there you can get helpful example URLs for releases and other info under the Links section in the json output.

Failing that the 403 does indicate permissions issues for the API user for the Space you are querying. You can test permissions for users at https://octopus.server.domain/app#/Spaces-1/configuration/testpermissions as a sanity check.

Let me know if this works for you or if we need to investigate further.

Kind Regards,
Paraic

4

Hello Paraic,

I made sure that the project ID was correct, and as you suggested I tested the permissions for the service
account and visibly I get the permissions I expect for the Space I’m targeting.

One thing that I find odd that I mentioned in my initial post is that, the same PowerShell script ran on the Octopus’ server itself is returning the JSON objects I expect. I’m using the same URL and the same API key.
When the “Execution Location” is set to “Octopus Server”, is it using the tentacle’s account? Could it be related to the tentacle itself?

Thank you in advance for your guidance !

5

Hi Johan
Thanks for confirming all those items. This is an unusual one for sure.

Would you be able to send through the verbose task log for the process you are running? There might be a clue there as to why the steps are failing when run on the Octopus server rather than the cli or in Powershell.

You should be able to attach it directly here.

Looking forward to getting to the bottom of this one.

Kind Regards,
Paraic

6

Hi Johan,
Just a quick question, are you using any Proxy setup on the Octopus server? We have seen 403 forbidden returned when there are issues there.

Kind Regards,
Paraic

7

Paraic,

Yes, I see that we have some proxy settings defined in our Octopus. I’ll contact our network team to check on this, but first I’m wondering : is there any way to bypass those proxy settings when making Octopus API calls from the Octopus server (round trip that doesn’t need the proxy in the end)?

8

Hi Johan,
We do have an open issue being fixed which is being rolled into the next release which addresses the default proxy not being picked up correctly. This generates a 403 forbidden error.

GH issue: Default Proxy settings not being propagated to Workers with upgrade to 2021.1.7198 or later · Issue #6941 · OctopusDeploy/Issues · GitHub

This has come about in recent releases so if you upgraded recently it will show up.

We have a partial workaround:

A partial workaround is to set the Custom proxy var using the Tentacle Manager on the worker. This is impractical in most cases though either due to the number of targets or dynamic workers etc

We generally do a minor release with certain fixes and this fix will come out in the next few weeks for sure.

Kind Regards,
Paraic

9

Hello Paraic,

The proxy was my problem. While waiting for the fix you are mentioning, I’m using this line in my PowerShell script :
[System.Net.WebRequest]::DefaultWebProxy = [System.Net.GlobalProxySelection]::GetEmptyWebProxy()

And that did the trick. I’m happy with this workaround for now.

Thank you very much for your help!

10

Hi Johan
Delighted to see you got to the bottom of that issue.

Its always good to see our customers post the solutions as it helps others who might have the same issue. Thanks for that!

If you need anything else in future do let us know.

Kind Regards,
Paraic

11

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.