Octopus.Client: repository.Tasks.FindMany() runs too slowly on big number of tasks

Legacy Uncategorised
1

Hi guys,

I want to send metrics to graphite about deployments:

  • Success deployments count
  • Failed deployments count
  • Deployment duration

I faced an issue, that methods FindAll() and FindMany() run too slow on our installation. We have 21739 tasks from the beginning, 5606 of those are deploys .

For example, this query in DataBase lasts for less than second:
SELECT ProjectName, EnvironmentName, DurationSeconds, StartTime, CompletedTime, TaskState FROM [dbo].[DeploymentHistory] where TaskState = 'success' and CompletedTime >= '{startTime}' and CompletedTime < '{endTime}'

Then I wrote this code:

        private static Dictionary<string, int> GetDeploymentsCount()
        {
            var deploysCount = new Dictionary<string, int> {{"Success", 0}, {"Failed", 0}};
            var sw = new Stopwatch();
            sw.Start();
            var successDeploysCount = _repository.Tasks.FindMany(IsSuccessAndRecent);
            sw.Stop();
            Console.WriteLine(sw.Elapsed);

            return deploysCount;
        }

        private static bool IsSuccessAndRecent(TaskResource task)
        {
            return task.Name == "Deploy" 
                && task.State == TaskState.Success 
                && task.IsCompleted 
                && task.CompletedTime >= _startTime 
                && task.CompletedTime < _endTime;
        }

And this code execution lasted for 00:19:27.8643567.

I’ve also tried the Henrik’s suggestion from this issue.

        private static Dictionary<string, int> GetDeploymentsCount()
        {
            var deploysCount = new Dictionary<string, int> {{"Success", 0}, {"Failed", 0}};
            var sw = new Stopwatch();
            sw.Start();
            var successDeploysCount = _repository.Deployments.FindMany(IsSuccessAndRecent);
            sw.Stop();
            Console.WriteLine(sw.Elapsed);

            return deploysCount;
        }

        private static bool IsSuccessAndRecent(DeploymentResource deploymentResource)
        {
            var task = _repository.Tasks.Get(deploymentResource.TaskId);
            return task.Name == "Deploy"
                && task.State == TaskState.Success
                && task.IsCompleted
                && task.CompletedTime >= _startTime
                && task.CompletedTime < _endTime;
        }

But the result is almost the same.

Is there a way to improve execution speed of that code?

Thank you in advance!

Kind regards,
Denis Titusov

2

Hi Denis,

Thanks for your question. The slowness is due to the Find*() methods using pagination, retrieving 30 records at a time, resulting in a large number of http requests. The alternative is to use the following code:

var tasks = repository.Client.Get<ResourceCollection<TaskResource>>(repository.Client.RootDocument.Links["Tasks"] + "?take=1000000");

You can explore the available APIs by going to /api.

Hope that helps!

Robert W

3

Hi Robert,

Thank you for this suggestion!
I will check it out and let you know how it works.

Kind regards,
Denis Titusov

4

Robert,

Thank you very much, this code is much faster.

Kind regards,
Denis Titusov