Acquire Packages Step - Long execution time

Octopus Cloud
1

Hello. I have an octopus project that has a few steps that deploy to Azure App Service. The acquire packages step is taking about 23 mins to complete. This seems like a really long time to download the packages (less than 200mb). The log is below. Any ideas on how to fix this? Our deploy times are taking much longer then needed because of this. Thanks.

Logs:

Acquiring packages June 10th 2023 04:30:33 Info
Making a list of packages to acquire June 10th 2023 04:30:33 Info
Leasing WindowsDefault dynamic worker... June 10th 2023 04:30:34 Info
Worker not ready, trying again in 10 seconds June 10th 2023 04:30:44 Info
Obtained WindowsDefault worker lease successfully. June 10th 2023 4:30:44 Info
Leasing WindowsDefault dynamic worker... June 10th 2023 04:30:44 Info
Obtained WindowsDefault worker lease successfully. June 10th 2023  4:30:44 Info
Leasing WindowsDefault dynamic worker... June 10th 2023 04:30:44 Info
Obtained WindowsDefault worker lease successfully. June 10th 2023  4:30:44 Info
Leasing WindowsDefault dynamic worker... June 10th 2023 04:30:44 Info
Obtained WindowsDefault worker lease successfully. June 10th 2023 04:30:57 Info
Package xxx v5.2.1 was found in cache. No need to download from feed. June 10th 2023 04:30:57 Info
Package yyy v5.2.1 was found in cache. No need to download from feed. June 10th 2023 04:30:57 Info
Package zzz v5.2.1 was found in cache. No need to download from feed. June 10th 2023 04:53:33 Info
All packages have been acquired
3

Hi Doug,
Thanks for reaching out with worker question.

There are a few unknowns here if you don’t mind me asking:

  • is this repeatable? Does package zzz v5.2.1 always take a long time? This may be due to a leased worker which is either broken or somehow resource constrained. If its using the same worker the same problem will arise each time.
  • The delay seems to be only on the package zzz - is this the case? The other two packages seem to be found quite quickly.
  • Can you try using the Linux worker instead? It may not have the same issue and could get you unblocked
  • Is the package zzz much larger than the other two? This still shouldn’t cause this large a delay though.

Let us know how you get on with the above and we can dig deeper if the issue persists

Kind regards,
Paraic

4

Hi Paraic. Thanks for the reply.

  • Yes, the acquire packages step always takes a long time
  • The delay seems to always be the last package which varies between releases.
  • I can try the linux worker
  • One of the packages (zzz in this case) is bigger, but less than 200mb.
5

Little more info. Investigating further, it looks like it’s this task that is taking long:

image
image1257×887 108 KB

6

I’ve tracked down the issue. I had a step in my deploy process that depended on the following image from Docker Hub

octopusdeploy/worker-tools:3.3.2-windows.ltsc2019

This image was taking over 20 mins to download and run on the default windows octopus worker. I’m going to have to find an alternate method for executing my azure script.

Regards.

2 Likes
7

Hi Doug,
Great to see you tracked down the issue.

I would just add that the docker image you are using is quite outdated and the reason its taking so long is that the image is not cached, due to its age. Its large so not being cached it will download the full amount from the Docker repo.

To help with this, you can try using the latest:
https://hub.docker.com/r/octopusdeploy/worker-tools/tags

I posted the list of tags here as you may actually want to try a Linux Ubuntu image which is considerably smaller but for the most part contains the same worker-tools. Naturally it doesn’t have any Windows commands so if you use those this may not work for you.

These latest images are cached and should download much faster as they are constantly cached.

Let me know how you get on with the above.

Kind regards,
Paraic

8

Thanks Paraic !

2 Likes
9

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.