Nuget push to external Octopus server fails with 403 Forbidden

Hi,

We have the Octopus server outside of our network (at our operating supplier’s location), using our TFS build agents to build nuget packages and push them to the Octopus server. There are three build agents in our setup, and all of them should be able to push the packages.

Our problem is that only one of the build agents is able to push all 5 nuget packages to the Octopus server. TFS builds 5 different nuget packages, but the two other build agents are only able to push 2 of them (for no apparent reason). When I try to manually enter the command in Command Prompt on the two build agents, I get this error message:

System.InvalidOperationException: Failed to process request. ‘Forbidden’.
The remote server returned an error: (403) Forbidden… —> System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.GetResponse()
at NuGet.RequestHelper.GetResponse()
at NuGet.HttpClient.GetResponse()
at NuGet.PackageServer.EnsureSuccessfulResponse(HttpClient client, Nullable1 expectedStatusCode) --- End of inner exception stack trace --- at NuGet.PackageServer.EnsureSuccessfulResponse(HttpClient client, Nullable1 expectedStatusCode)
at NuGet.PackageServer.PushPackageToServer(String apiKey, Func`1 packageStreamFactory, Int64 packageSize, Int32 timeout, Boolean disableBuffering)
at NuGet.PackageServer.PushPackage(String apiKey, IPackage package, Int64 packageSize, Int32 timeout, Boolean disableBuffering)
at NuGet.Commands.PushCommand.PushPackageCore(String source, String apiKey, PackageServer packageServer, String packageToPush, TimeSpan timeout)
at NuGet.Commands.PushCommand.PushPackage(String packagePath, String source, String apiKey, TimeSpan timeout)
at NuGet.Commands.PushCommand.ExecuteCommand()
at NuGet.Commands.Command.Execute()
at NuGet.Program.Main(String[] args)

I can’t see why this should happen, when the same command works on the first build agent, not the other two.
Any help is much appreciated!

Kind regards,
Trond

Hi Trond,

Thanks for getting in touch! Are you able to please provide us with your build log. Are your packages all for one project or across multiple projects?
Is there a proxy between you and the Octopus Server?

All of this should help us get to the bottom of this for you.

Thanks!
Vanessa

Hi Vanessa,

Thanks for your quick reply!

(1)The build log in Visual Studio shows the same error as I get in the command line:
“Failed to process request. ‘Forbidden’.”
“The remote server returned an error: (403) Forbidden…”

(2) The packages are all the one project.

(3) There is not a proxy between me and the Octopus Server.

Some additional information that came up today is that the Octopus Server gives this message when I try to push one of the ‘Forbidden’ packages:

2015-04-07 09:53:41.8102 92 WARN You do not have permission to perform this action. Please contact your Octopus administrator. Missing permission: BuiltInFeedPush (USERNAME requesting NUGETSTORE_URL)

The user does indeed have the BuiltInFeedPush permission, with no constraints on projects or environments etc.

/Trond

Hi Trond,

I guess that confirms that it is to do with the API key and user so, it might be worth a restart of the service as permissions may need to be rebuilt for that user.
Also double checking the API keys are identical across all builds.

Let me know if that helps
Vanessa

Hi Vanessa,

Still no luck after restarting the Octopus service. I’m pushing two packages to the same project from Powershell on the build agent, and only one of the packages gets pushed. The only difference in the command is the package name. It has nothing to do with the size of the packages as well, because it’s the smallest one (57 KB vs 7 682 KB) that doesn’t get pushed.

I’ll try setting up a new user with a new API key just to try it out. Thought I’d just keep you posted on the progress :slight_smile:

/Trond

Hi Trond,

There is only one more thing I can think of to try here, and that would be to repair the Raven DB.
You can do this from the Octopus Server manager.

Vanessa

Hi again Vanessa,

We tried repairing the Raven DB with no apparent success.
However, when one of the administrators removed the Projects constraint on the Service Account’s Team, the issue was resolved!

I still don’t understand why this should cause any trouble, because all the nuget packages are tagged to the exact project that was written down in the constraint on the Team.

But I’m going to let it slide, because I’ve spent way too much time on this already :slight_smile: Just thought I’d share our solution even though it may not be the best. Maybe someone else can make use of it too.

Thanks for being to helpful and responsive, Vanessa!

Kind regards,
Trond