Octopus works fine, but have repeating errors in the logs. "An error (1301) occurred while enumerating the groups"

lars_oyvind_bodahl · 30 July 2015 08:21

The followin error keeps on occuring in the log file. Any suggestions on how to troubleshoot this error?

FYI: We upgraded from 2.6 -> 3.0 last week.

2015-07-30 10:02:29 Error
An error (1301) occurred while enumerating the groups.  The group's SID could not be resolved.

DETAILS:
An error (1301) occurred while enumerating the groups.  The group's SID could not be resolved.
System.DirectoryServices.AccountManagement.PrincipalOperationException: An error (1301) occurred while enumerating the groups.  The group's SID could not be resolved.
   at System.DirectoryServices.AccountManagement.SidList.TranslateSids(String target, IntPtr[] pSids)
   at System.DirectoryServices.AccountManagement.SidList..ctor(SID_AND_ATTR[] sidAndAttr)
   at System.DirectoryServices.AccountManagement.AuthZSet..ctor(Byte[] userSid, NetCred credentials, ContextOptions contextOptions, String flatUserAuthority, StoreCtx userStoreCtx, Object userCtxBase)
   at System.DirectoryServices.AccountManagement.ADStoreCtx.GetGroupsMemberOfAZ(Principal p)
   at System.DirectoryServices.AccountManagement.UserPrincipal.GetAuthorizationGroupsHelper()
   at Octopus.Server.Web.Infrastructure.Authentication.ActiveDirectoryMembership.GetMemberExternalSecurityGroupIds(String username) in y:\work\refs\heads\master\source\Octopus.Server\Web\Infrastructure\Authentication\ActiveDirectoryMembership.cs:line 124

Dalmiro · 30 July 2015 16:12

Hi Lars,

Thanks for reaching out! That error indicates that the Octopus Server could not resolve the SID of one of your Active Directory groups. Is it possible that a team was deleted from AD, but it is still on an Octopus Team?

Thanks,

Dalmiro

lars_oyvind_bodahl · 31 July 2015 07:12

Not that I can see… Everything seems correct in the GUI. I checked the JSON field in the Team table in the database, and all the ExternalSecurityGroups matches the objectSid on the group in AD.

The only thing that have been deleted are some of the users that are listed in the Everyone group. They are set as inactive in the Octopus database. (IsActive = 0) But, they are listed in MemberUserIds in the Team table. Does that have anything to do with this?

–
Lars Øyvind

lars_oyvind_bodahl · 31 July 2015 13:37

I deleted all the users from Octopus as well. It did not help… So there has to be something with the groups. Any way to make it log the SID it is having trouble with?

lars_oyvind_bodahl · 31 July 2015 13:37

Regarind my previous post: I deleted all the inactive users, not all the users

Dalmiro · 3 August 2015 15:53

Hi Lars,

It’s definitely something with the groups according to the error message

An error (1301) occurred while enumerating the groups. **The group's SID** could not be resolved

We haven’t received any reports of similar issues, which leads me to believe its an environmental issue. Do you have many AD groups on your Octopus instance to remove them all and add them back again from the web UI? I know it sounds like an ugly workaround, but it will be helpful to know if this is an issue within Octopus or with your specific AD groups.

Thanks,

Dalmiro

James_R · 3 August 2015 20:46

I can report seeing the same issue, everything works great but I get this error in the logs every minute.

2015-08-04 08:37:53.3824    174 ERROR  An error (1301) occurred while enumerating the groups.  The group's SID could not be resolved.
System.DirectoryServices.AccountManagement.PrincipalOperationException: An error (1301) occurred while enumerating the groups.  The group's SID could not be resolved.
   at System.DirectoryServices.AccountManagement.SidList.TranslateSids(String target, IntPtr[] pSids)
   at System.DirectoryServices.AccountManagement.SidList..ctor(SID_AND_ATTR[] sidAndAttr)
   at System.DirectoryServices.AccountManagement.AuthZSet..ctor(Byte[] userSid, NetCred credentials, ContextOptions contextOptions, String flatUserAuthority, StoreCtx userStoreCtx, Object userCtxBase)
   at System.DirectoryServices.AccountManagement.ADStoreCtx.GetGroupsMemberOfAZ(Principal p)
   at System.DirectoryServices.AccountManagement.UserPrincipal.GetAuthorizationGroupsHelper()
   at Octopus.Server.Web.Infrastructure.Authentication.ActiveDirectoryMembership.GetMemberExternalSecurityGroupIds(String username) in y:\work\refs\heads\master\source\Octopus.Server\Web\Infrastructure\Authentication\ActiveDirectoryMembership.cs:line 124

I don’t have any AD groups added to any teams in Octopus Deploy but do use AD for authentication.

Michael_Noonan · 5 August 2015 23:30

Hi Lars and James,

My name is Mike and I’m going to offer whatever help I can to resolve this issue. AD integration for most people just works, and for the few where it fails there can be several root causes that lead to the same error message.

Take a look at this old help ticket: http://help.octopusdeploy.com/discussions/problems/20757-after-a-fresh-install-of-24546-ad-users-cannot-login

Also this StackOverflow conversation may provide some more context as to all the different root causes: http://stackoverflow.com/questions/17027781/userprincipals-getauthorizationgroups-an-error-1301-occurred-while-enumerating

Here’s the next step where you can help: We only call this method when authenticating a User, so it’s interesting that you’re seeing the message occur periodically. We actually log a Verbose message Finding external security groups for '{Username}'... and I think this would help understand which user(s) are failing, and why it seems to happen periodically.

Please follow http://docs.octopusdeploy.com/display/OD/Log+files to change the minLevel to Verbose and take a look at the log files to make sure you can see which user(s) are causing the error. Don’t forget to reset the minLevel afterwards or you’ll have performance issues!

Take a look at the log yourself, and I’d appreciate if you can post a reply if you’re able to resolve the issue.

Hope this helps!
Mike

James_R · 6 August 2015 01:12

Hi Mike,

I enabled Trace logging levels which as you mentioned gave us some more information. Here is a redacted sample:

2015-08-06 12:49:13.7995    120 TRACE  Loading permissions for user: jamesr@redacted.net
2015-08-06 12:49:13.8151    120 DEBUG  Finding external security groups for 'jamesr@redacted.net'...
2015-08-06 12:49:13.9245    120 ERROR  An error (1301) occurred while enumerating the groups.  The group's SID could not be resolved.
System.DirectoryServices.AccountManagement.PrincipalOperationException: An error (1301) occurred while enumerating the groups.  The group's SID could not be resolved.
   at System.DirectoryServices.AccountManagement.SidList.TranslateSids(String target, IntPtr[] pSids)
   at System.DirectoryServices.AccountManagement.SidList..ctor(SID_AND_ATTR[] sidAndAttr)
   at System.DirectoryServices.AccountManagement.AuthZSet..ctor(Byte[] userSid, NetCred credentials, ContextOptions contextOptions, String flatUserAuthority, StoreCtx userStoreCtx, Object userCtxBase)
   at System.DirectoryServices.AccountManagement.ADStoreCtx.GetGroupsMemberOfAZ(Principal p)
   at System.DirectoryServices.AccountManagement.UserPrincipal.GetAuthorizationGroupsHelper()
   at Octopus.Server.Web.Infrastructure.Authentication.ActiveDirectoryMembership.GetMemberExternalSecurityGroupIds(String username) in y:\work\refs\heads\master\source\Octopus.Server\Web\Infrastructure\Authentication\ActiveDirectoryMembership.cs:line 126

Based on the logs the error occurs for every currently logged in user (including my own account). So the above is repeated several times in the logs over the period of a minute, just with a different user name that it is attempting to get permissions for.

The weird thing is that everything still works fine, I can login and do everything i need. It’s just not ideal to have the logs filled with such errors and I’d like to resolve it out of interest more than anything else.

Michael_Noonan · 6 August 2015 01:26

Hi James,

Thanks for getting back to me! I’m glad the Verbose logs provide more context for us to diagnose what’s going on. I’ve had a good look over the code: we try two different methods of getting the groups from AD because either method can fail in certain cases. Based on your report that everything is working fine I have raised an Issue to fix the logging: https://github.com/OctopusDeploy/Issues/issues/1850

You can use that Issue to track the progress of the bug fix.

Hope that helps!
Mike

James_R · 6 August 2015 04:34

Thanks Mike, that makes sense! Good to see it isn’t anything serious or environmental on my side.

Michael_Noonan · 20 August 2015 23:32

Hi James,

Hopefully everything is working fine for you and you aren’t getting any more false-positive warnings.

Happy Deployments!
Mike

lars_oyvind_bodahl · 3 September 2015 06:02

I finally got around to upgrade our Octopus server. Upgrade only takes 4 minutes, so there’s no need to postpone it for so long

After one hour, there are no log entries regarding this “error” Case closed!

Michael_Noonan · 3 September 2015 06:03

Hi Lars,

Thanks for getting back to me. I’m glad to hear it’s working better for you now.

Out of interest’s sake, how are you getting on with Octopus 3.0?

Happy Deployments!
Mike

lars_oyvind_bodahl · 3 September 2015 06:14

Thumbs up! It is working great.

Michael_Noonan · 3 September 2015 06:16

Hi Lars,

That’s fantastic news.

Happy Deployments!
Mike

James_R · 6 September 2015 22:13

Agreed, it is fixed

Michael_Noonan · 6 September 2015 22:51

Hi James,

Thanks for the positive report!

Happy Deployments!
Mike