Powershell Script for Octopus API to list all deployments

puk.it.scm · 30 November 2021 08:19

Hi Team ,

we are using a script to fetch the list of deployments in an environment and report few details from it

Add-Type -Path ‘.\Octopus.Client.dll’

$apikey = ‘API-XXX’
$OctopusURI = ‘http://octopusserver/’
$CheckEnvironment=“env”

$endpoint = new-object Octopus.Client.OctopusServerEndpoint $OctopusURI,$apikey
$repository = new-object Octopus.Client.OctopusRepository $endpoint

#get environment and deployments
$environment = $repository.Environments.FindByName($CheckEnvironment)
$deployments=$repository.Deployments.FindAll()| Where-Object {$_.EnvironmentId -eq $environment.Id}

This solution has been working without issue few weeks back but now this is not working as fetching deployments<repository.Deployments.FindAll()> got stuck and not getting any result even after waiting for more than hours
The same script works fine when it ran against different instance(TestInstance)of Octopus(which contains 372 projects)
Is there any threshold for fetching details as we more projects in Prodinstance currently we have (564 projects )?
we are using Octopus of Version- 2021.2.7727
Can someone suggest on any solutions or alternatives please as we rely of the report for our deployments ?

mark.harrison · 30 November 2021 09:58

Hi @puk.it.scm

Thanks for your question.

Whilst I can’t speculate on the reason that the FindAll method isn’t working for you specifically, I would recommend against using that method at all. That will return all results from the Server and then filter on the client-side, which is not the most efficient way of retrieving the results you want.

The Octopus Client has supported paged calls since Octopus 3.0 so I think this method would work out best. Here is some example code that you can try out. It also prints out to the console for every page it returns, so it’s useful to see if it’s not returning any results:

 Load octopus.client assembly
Add-Type -Path "Octopus.Client.dll"
$octopusURL = "http://your.octopus.app"
$octopusAPIKey = "API-KEY"

$spaceName = "default"
$environmentName = "Development"

$endpoint = New-Object Octopus.Client.OctopusServerEndpoint $octopusURL, $octopusAPIKey
$repository = New-Object Octopus.Client.OctopusRepository $endpoint

# Get space id
$space = $repository.Spaces.FindByName($spaceName)
Write-Host "Using Space named $($space.Name) with id $($space.Id)"

# Create space specific repository
$repositoryForSpace = [Octopus.Client.OctopusRepositoryExtensions]::ForSpace($repository, $space)

# Get environment
$environment = $repositoryForSpace.Environments.FindByName($environmentName)

# Get deployments to environment
$projects = @()
$environments = @($environment.Id)
$deployments = New-Object System.Collections.Generic.List[System.Object] 
    
$repositoryForSpace.Deployments.Paginate($projects, $environments, {param($page) 
    Write-Host "Found $($page.Items.Count) deployments.";
    $deployments.AddRange($page.Items); 
    return $True; 
})

Write-Output "Retrieved $($deployments.Count) deployments to environment $($environmentName)"

puk.it.scm · 30 November 2021 11:10

Thanks @mark.harrison for the response.
will try to look into your recommendation if it solves our purpose

Adding few more information from octopus server log for findAll() is re-attempting to fetch details

2021-11-30 09:56:02.5718 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:06.8038 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:06.8365 6488 2021-11-30 09:56:11.4439 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:15.7453 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:15.7744 6488 2021-11-30 09:56:20.3257 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:24.9061 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:24.9470 6488 2021-11-30 09:56:29.5206 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:33.7223 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:33.7548 6488 2021-11-30 09:56:38.0498 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:42.5548 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:42.6829 6488 2021-11-30 09:56:47.4870 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:51.5661 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:51.6082 6488 2021-11-30 09:56:55.7841 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:59.6877 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:56:59.7192 6488 2021-11-30 09:57:03.7422 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:57:08.0243 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC
2021-11-30 09:57:08.0616 6488 2021-11-30 09:57:12.4778 6488 FROM [dbo].[Deployment]
WHERE ((([SpaceId] = ‘Spaces-1’)))
ORDER BY [Created] DESC 235 INFO Process reader took 3803ms in transaction ‘GetDeployments.Get|800034a8-0000-eb00-b63f-84710c7967bb|T235’: SELECT *
235 INFO Process reader took 4098ms in transaction ‘GetDeployments.Get|800034a8-0000-eb00-b63f-84710c7967bb|T235’: SELECT *
235 INFO Request took 8214ms: GET http://octopusserver/api/Spaces-1/deployments 800034a8-0000-eb00-b63f-84710c7967bb by User
235 INFO Process reader took 4364ms in transaction ‘GetDeployments.Get|800034a9-0000-eb00-b63f-84710c7967bb|T235’: SELECT *
235 INFO Process reader took 4171ms in transaction ‘GetDeployments.Get|800034a9-0000-eb00-b63f-84710c7967bb|T235’: SELECT *
235 INFO Request took 8857ms: GET http://octopusserver/api/Spaces-1/deployments?skip=30&take=30 800034a9-0000-eb00-b63f-84710c7967bb by User
243 INFO Process reader took 4157ms in transaction ‘GetDeployments.Get|800034aa-0000-eb00-b63f-84710c7967bb|T243’: SELECT *
243 INFO Process reader took 4451ms in transaction ‘GetDeployments.Get|800034aa-0000-eb00-b63f-84710c7967bb|T243’: SELECT *
243 INFO Request took 9037ms: GET http://octopusserver/api/Spaces-1/deployments?skip=60&take=30 800034aa-0000-eb00-b63f-84710c7967bb by User
243 INFO Process reader took 4397ms in transaction ‘GetDeployments.Get|800034ab-0000-eb00-b63f-84710c7967bb|T243’: SELECT *
243 INFO Process reader took 4071ms in transaction ‘GetDeployments.Get|800034ab-0000-eb00-b63f-84710c7967bb|T243’: SELECT *
243 INFO Request took 8797ms: GET http://octopusserver/api/Spaces-1/deployments?skip=90&take=30 800034ab-0000-eb00-b63f-84710c7967bb by User
198 INFO Process reader took 4118ms in transaction ‘GetDeployments.Get|800034ac-0000-eb00-b63f-84710c7967bb|T198’: SELECT *
198 INFO Process reader took 4373ms in transaction ‘GetDeployments.Get|800034ac-0000-eb00-b63f-84710c7967bb|T198’: SELECT *
198 INFO Request took 8910ms: GET http://octopusserver/api/Spaces-1/deployments?skip=120&take=30 800034ac-0000-eb00-b63f-84710c7967bb by User
236 INFO Process reader took 4444ms in transaction ‘GetDeployments.Get|800034ad-0000-eb00-b63f-84710c7967bb|T236’: SELECT *
236 INFO Process reader took 3949ms in transaction ‘GetDeployments.Get|800034ad-0000-eb00-b63f-84710c7967bb|T236’: SELECT *
236 INFO Request took 8840ms: GET http://octopusserver/api/Spaces-1/deployments?skip=150&take=30 800034ad-0000-eb00-b63f-84710c7967bb by User
236 INFO Process reader took 3989ms in transaction ‘GetDeployments.Get|800034ae-0000-eb00-b63f-84710c7967bb|T236’: SELECT *
236 INFO Process reader took 3774ms in transaction ‘GetDeployments.Get|800034ae-0000-eb00-b63f-84710c7967bb|T236’: SELECT *
236 INFO Request took 8090ms: GET http://octopusserver/api/Spaces-1/deployments?skip=180&take=30 800034ae-0000-eb00-b63f-84710c7967bb by User
238 INFO Process reader took 3841ms in transaction ‘GetDeployments.Get|800034af-0000-eb00-b63f-84710c7967bb|T238’: SELECT *
238 INFO Process reader took 4150ms in transaction ‘GetDeployments.Get|800034af-0000-eb00-b63f-84710c7967bb|T238’: SELECT *
238 INFO Request took 8327ms: GET http://octopusserver/api/Spaces-1/deployments?skip=210&take=30 800034af-0000-eb00-b63f-84710c7967bb by User
243 INFO Process reader took 4129ms in transaction ‘GetDeployments.Get|800034b0-0000-eb00-b63f-84710c7967bb|T243’: SELECT *

We were able to execute the above query and it took around 1second for displaying result in the DB …Does api has any timeout for the response from DB to retry the query again

mark.harrison · 30 November 2021 11:35

Hi @puk.it.scm

The API has a default HttpClient timeout which is typically 100 seconds.
I’m not sure if it’s possible to change that timeout, but that timeout only controls the connection between the client and the server, not the database query itself.

As I mentioned previously using FindAll is not recommended so I’d recommend moving away from that usage and trying my alternative.

If you still experience issues, then you can try profiling your requests using a Network tool like Fiddler (Fiddler | Web Debugging Proxy and Troubleshooting Solutions) or WireShark. This will help to show you if your requests are being made and returned successfully to the client where your script is running

I hope that helps!

puk.it.scm · 3 December 2021 09:09

Hi @mark.harrison ,

Thanks for the above script, we tried to fit in our requirement and it seems to be working but unfortunately it returns only items from a single page(Self/Current) i.e… only 30 items and we tried to traverse across but no luck.
Can you guide us if we can pass on the number of items to return say latest 300 items?

mark.harrison · 3 December 2021 09:25

Hi @puk.it.scm,

I tested the script against 2021.3 and it returned multiple pages.

Are you able to provide more information about the output the script ran with?

If the API returned multiple pages you should get a line for each page like this:

Best regards

puk.it.scm · 3 December 2021 09:35

Using Space named Default with id Spaces-1
Found 30 deployments.
Found 0 deployments.
Retrieved 30 deployments to environment Dev

Octopus version - 2021.2.7727

mark.harrison · 3 December 2021 09:51

Unfortunately, I can’t think of any reason why that wouldn’t work for you.

Are you able to provide the complete script you used (removing any sensitive values)?

In addition, can you confirm what version of the Octopus Client you are using?

Kind regards

puk.it.scm · 3 December 2021 10:53

Seems to be odd - have just tried using the above script by replacing the url and api keys - I am observing same behavior of returning only 30 items

we are using - octopus.client.11.3.3425

Have also tried using the REST call method getting same output returning only 30 items

mark.harrison · 3 December 2021 11:10

Hi @puk.it.scm

That is strange indeed. I’m not aware of any behavior in Octopus that would limit the results you should get back to only 30 results.

I’m going to ask my colleagues in Support to take a look with you, as they have more expertise in troubleshooting issues like this.

Best regards

dane.falvo · 3 December 2021 13:42

Hi @puk.it.scm,

I’ve just been reading up on this ticket and attempting to work out what might be going wrong.

I’m wondering if there is actually just 30 deployments in that space that the particular user/api key your using, has access to.

Can you please try navigating to the following api endpoint?

https://<octopusurl>/api/deployments?environments=<Environment ID>

so for example, it might be
https://octopusinstance.com/api/deployments?environments=Environments-1

If you are unsure of what the Environment id will be, you get the ID by going to the Environment Edit page within your Octopus Instance.

You will find the ID in the URL that you get redirected to.

Also, make sure the API key entered into your Powershell Script is owned by the same user you are logged in as, when you navigate to the API endpoint. I would like to see what the “Total Results” is.

I look forward to hearing from you.

Thank you.

puk.it.scm · 6 December 2021 04:29

Hi @dane.falvo,
Please find the below result of the API endpoint

This is the output for the same user whose api key used in Power Shell script

sergei.dubovsky · 6 December 2021 17:42

Hi @puk.it.scm,

@dane.falvo is not online at the moment; I will step up in the meantime. If I understand correctly, you are getting the correct number of deployments using the REST API, but it only gives you 30 deployments with the Client API.

I wonder if that might be caused by the outdated Client API DLL, which does not support the pagination. Will it be possible for you to update the client to the latest version, published here: NuGet Gallery | Octopus.Client 11.3.3453?

Thank you.
Sergei

system · 6 January 2022 17:43

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.