Had an interesting issue this morning with the external security groups sync job. We use role-based group membership controls in Active Directory to control who can do what within the Octopus system, along with many other systems.
Had an issue where one of the two nodes running Octopus Deploy ran out of memory shortly before the external security groups sync job ran which caused COM exceptions for each call to AD asking for group membership.
Although this obviously isn’t an Octopus-caused issue in itself, the way Octopus handled it made for an interesting bit of troubleshooting. When the calls to AD failed, Octopus removed the group memberships from each user leaving everyone unable to do anything in the system!
Waiting an hour for the job to run again would have obviously solved the issue but I ended up updating the DB so it thought a run of the job was overdue so I was able to resolve fairly quickly.
Just wondering if it might be worth considering not dropping the user group membership if the call to AD fails?