User has BuiltInFeedPush, but Octopus says it doesn't

jake_kerber · 30 October 2015 21:17

We’re trying to secure things a bit - so we don’t have API Keys for octopus administrators out in our build files for pushing nuget packages to octopus.

I created a Team that has just the built-in Package Publisher role. I added several (about 15) projects to that Team.

I created an Octopus service account user and added it to the team. I got an api key for that user.

I updated the nuget push step in 3 of our builds (in TFS) to use the API Key for the new “pusher” user.

This works for 2 of the 3 projects, but fails for the third. If I look in the Audit trail I can see that the new pusher user is indeed the one who published the 2 packages. I get a 403 error when it tries to push the package for the third project. In the log on the server it says that the pusher user is missing BuiltInFeedPush.

When I Test Permissions for the user, I can see that the user does have BuildInFeedPush for the necessary projects.

For the project of interest, there is only one package ID (which is the one being pushed when we get the 403 error), so there should be no confusion there.

The exact same thing happened with another team of developers in our organization, too. They have their own “pusher” Team (with Package Publisher) for their specific projects. They have a service account user that is a member of that team. Two of their projects work; one does not.

This is very similar (perhaps the same issue) to this post: http://help.octopusdeploy.com/discussions/problems/32512-nuget-push-to-external-octopus-server-fails-with-403-forbidden

Things I’ve tried:

restarting the server service
adding environments to the Team
removing the problematic project from the Team, saving, and re-adding the project

If I remove all projects from the team, so that it should be able to publish packages for any project, it then works (what the user from the above referenced support issue eventually resorted to).

However, for security reasons, we want our “pusher” users to only be able to push packages for specific projects, and have separate “pusher” users for the different project teams in our organization.

Michael_Noonan · 3 November 2015 01:49

Hi Jake,

Thanks for getting in touch and going to this length with the troubleshooting process. Can you clarify a couple of things for me?

Which version of Octopus Deploy are you running? The rest of my investigation applies to 3.x.

Is the NuGet Package in question used in any other Projects?

The logic we apply goes a bit like this:

Apply Deny permissions, and exit early.
Find all Projects where the Package is used, and make sure the current User has the right permission for ALL of those Projects.

The permission we require changes based on what you are doing:

If the package doesn’t exist, you need BuiltInFeedPush
If the package does exist and is being overwritten, you need BuiltInFeedAdminister

Take a look and see if this helps. If not please get back to me with the specific details of the Octopus Server version, the Package ID, and any relevant screenshots and/or deployment logs.

Hope that helps!
Mike

jake_kerber · 3 November 2015 16:05

We are using version 3.1.2.

The nuget packages with the problem are each only used in a single project.

How do you determine which projects the package is used in – do you look in the JSON of the Projects table?

I don’t understand what you mean by “If the package doesn’t exist”
Does that mean if we’ve never published a package with that ID before? If so, that’s not the case for us. We’ve pushed that package to the repository before.

The file we push is named like this:
App.database..nupkg
Where is something like: 1.0.111687

In Library->Packages, it correctly shows app.database in the list of IDs.

Should the user have any other permissions – like ReleaseCreate (since the project is setup to create a new release when a new package arrives)?
However, that’s how our other projects are also configured and it works for them.

Michael_Noonan · 6 November 2015 00:04

Hi Jake,

I apologise for not getting back to you sooner.

Getting dirty in the SQL Server
We determine which Packages are used by a Project by inspecting each Step in the Deployment Process that references a PackageId. If you want to dive into the SQL Database you can look in the DeploymentProcess table at the JSON column for the Property Octopus.Action.Package.NuGetPackageId.

For example, this will find all current Deployment Processes where the OctoFX.Database package is referenced, and the OwnerId will tell you the Project:
SELECT * FROM DeploymentProcess WHERE IsFrozen = 0 AND JSON LIKE '%OctoFX.Database%'

If the package doesn’t exist?

What I mean there is “if you have already uploaded the exact same package before (by name+version)” in which case you would be overwriting the existing package. To do this you need BuiltInFeedAdministrator.

Auto Release Creation

Yes, I would make sure that User has ReleaseCreate if you’re going to use Auto Release Creation, but the absence of ReleaseCreate shouldn’t stop the package being pushed in the first place.

Hope that helps!
Mike

jake_kerber · 6 November 2015 19:39

That helped a ton!

For the other team having this problem, we were able to find another project that was erroneously referencing one of their packages. We fixed that and their problem is solved.

However, we’re still having a problem with my team, but we’re getting closer.
It looks like another project was cloned from ours – referencing our app.database package. In the DeploymentProcess table, there are a bunch of records for Projects-41 (the valid project), but also one record for Projects-60 that was originally cloned from Projects-41.

The Projects-60 project no longer references our nuget package. But I think what happened is that after it was created a new release was created while it was still referencing our app.database package. Projects-60 has never yet been deployed (not even to dev), so I just deleted all of its Releases (via octo delete-releases). That record referencing our package is still in the DeploymentProcess table in the database though. In case it means anything, the Id = deploymentprocess-Projects-60-s-1-K8XSU, OwnerId = Projects-60, IsFrozen = 1, and Version = 1.

If I look at all the DeploymentProcess records for Projects-60, there is only one other (the new one with Json referencing the correct nuget package id), with Id = deploymentprocess-Projects-60, OwnerId = Projects-60, IsFrozen = 0, and Version = 3.

It’s interesting that there is no version 2. I assume that got cleaned up by some process.

Is it safe to simply delete the bad record (version 1, isFrozen = 1)? Or is there some process that should delete that automatically?

Michael_Noonan · 18 November 2015 08:24

Hi Jake,

I’m sorry for the delay getting back to you. When you create a Release, we snapshot the DeploymentProcess and Variables, which are the IsFrozen=1 records. If you delete the Release it should take its frozen DeploymentProcess and Variables with it.

So, could you delete that Release and confirm that the frozen DeploymentProcess is deleted also? Otherwise we will fix that as a bug. Once the Release has been deleted there won’t be anything else referencing the frozen DeploymentProcess nor Variables.

Hope that helps.
Mike

jake_kerber · 18 November 2015 15:04

I did delete the release (I deleted all of that project’s releases on 11/6). But the DeploymentProcess record remains.

Is it safe to just delete it from the database?

Michael_Noonan · 19 November 2015 00:13

Hi Jake,

Yes, you can delete the DeploymentProcess which should fix the security problem. I’m going to investigate whether we should cascade delete these with Releases, and what flow-on effect that will have.

Hope that helps.
Mike