When creating a new release for our projects, we often find that loading the newly created release takes a very long time: it can take everything from 1 to 10 minutes. And sometimes everything works as expected, loading the release immediately.
We’re running version 2.5.12.666, and this occationally happens for all our projects, not just some. Taking a look at the Diagnostics page, I only find the folowing error (and I suspect that is a completely different issue):
The specified forest does not exist or cannot be contacted.
System.DirectoryServices.ActiveDirectory.ActiveDirectoryObjectNotFoundException: The specified forest does not exist or cannot be contacted.
at System.DirectoryServices.ActiveDirectory.Forest.GetForest(DirectoryContext context)
at System.DirectoryServices.AccountManagement.ADStoreCtx.GetGroupsMemberOf(Principal p)
at System.DirectoryServices.AccountManagement.Principal.GetGroupsHelper()
at Octopus.Server.Web.Infrastructure.Authentication.ActiveDirectoryMembership.ReadUserGroups(Principal principal, ICollection`1 groups) in y:\work\refs\heads\master\source\Octopus.Server\Web\Infrastructure\Authentication\ActiveDirectoryMembership.cs:line 141
at Octopus.Server.Web.Infrastructure.Authentication.ActiveDirectoryMembership.GetMemberExternalSecurityGroupIds(String username) in y:\work\refs\heads\master\source\Octopus.Server\Web\Infrastructure\Authentication\ActiveDirectoryMembership.cs:line 120
Has anyone else had the same issue? Can you supply some information as to what happens when you click the “Create release” button so that I can begin troubleshooting this issue?
Thanks for getting in touch! Unless you get an error, the diagnostics page or logs aren’t going to tell us much. So we agree that the error you found is not related.
As for creating releases. When the release page loads, it needs to find packages from feeds, and the last release.
So to try to get to the bottom of the exact cause of the slowness I’ll ask some questions.
How many package steps are in the project/s that you see this issue for?
Are they using internal or external feeds?
how many packages are stored in your NuGet feed that it would have to iterate through?
Do you ever find you have network issue between your feed and Octopus, even during deployments (such as speed to find the package and then download it)?
How many releases have been completed for that project?
I would also suggest that the next time you see anything longer than 1 minute, you try to browse to your NuGet feed (like from Visual Studio).
All of this information, while allowing us to help you, will also give us more scenarios for testing these kinds of load/slowness issues in future!
P.S. We are all fans of your twitter feed
Vanessa
Thank you for your reply! And I’m glad you like my Twitter feed! I’ll try to answer your questions as well as I can, given that this happens for almost all our projects occationally.
Number of steps: 4-7 steps per project
We’re using an external feed, but as we know the feed has some performance issues we will be trying out the internal feed shortly.
Between 200-400 packages stores in the NuGet feed, depending on the amount of activity since the last cleanup
Not that I know of. During deployments, I find that downloading the packages works as expected.
No more than 20 releases.
Just to be clear, the 1-10 minutes loading time occurs after we’ve specified the release number and selected the nuget versions to use for the release. It happens when we click the “Create release” button.
I’ve also seen the same load time if I go to the “Releases” page of a project and click on one of the previous releases.
Thanks for answering the questions. From your description below, this all appears to be related to the release page, as after you create a new one that is the page it takes you to.
I will get back to you tomorrow if I have further questions, but I will get all the different queries looked at to see where the bottleneck could be on this page.
I if could ask for your help here. If you could use Developer tools in your browser to watch the network when that page loads, and give us an idea of which API call/s is/are taking the longest to process.
Let me know what you find. We had a look and have determined there could be a few pain points, but none really match the numbers you gave, so this should definitely point us in the right direction.
I’m very sorry for the late reply, I’ve been out of the office for almost a week now. However, I have taken a closer look at the API calls that are being done, and these three seem to be the ones that take time (see attachment).
Not a problem at all. I have created a GitHub issue which you can track here: https://github.com/OctopusDeploy/Issues/issues/1394
Thanks for doing the leg work for us. If we get a patch build out is it something you would be able to test in a safe environment?
If you wouldn’t mind scheduling a time for a screen share after the new year (from the 5th onward) https://octopusdeploy.acuityscheduling.com/schedule.php
It doesn’t make a whole lot of sense why these queries are taking 4 minutes, so understanding your environment and database will really help us figure this out.
