We’re using a Dynamic Windows 2019 worker, and until recently no problems, but similar to the issue posted a couple of days ago about the Ubuntu workers, suddenly we can’t pull docker images? This has happened a few times in the last week, but usually resolves quickly, but today it’s persistent.
Relevant section from the log:
02:58:07 Info | Leasing Windows2019 dynamic worker...
02:58:08 Info | Obtained worker lease successfully.
02:58:08 Verbose | Leased worker 21-12-22-1949-lb1s8 from pool Latest Windows Worker (lease Leases-1733916).
02:58:08 Info | Leasing Windows2019 dynamic worker...
02:58:08 Info | Obtained worker lease successfully.
02:58:08 Verbose | Leased worker 21-12-22-1949-lb1s8 from pool Latest Windows Worker (lease Leases-1733917).
02:58:08 Info | Leasing Windows2019 dynamic worker...
02:58:08 Info | Obtained worker lease successfully.
02:58:08 Verbose | Leased worker 21-12-22-1949-lb1s8 from pool Latest Windows Worker (lease Leases-1733918).
02:58:08 Info | Leasing Windows2019 dynamic worker...
02:58:08 Info | Obtained worker lease successfully.
02:58:08 Verbose | Leased worker 21-12-22-1949-lb1s8 from pool Latest Windows Worker (lease Leases-1733919).
.
.
.
02:58:26 Verbose | CurrentUser: NT AUTHORITY\SYSTEM
02:58:26 Verbose | MachineName: WIN-CR3QPFU5EJU
02:58:26 Verbose | ProcessorCount: 2
02:58:26 Verbose | CurrentDirectory: C:\Octopus\Tentacle\Work\20211223025814-3591-39
02:58:26 Verbose | CurrentLocation: C:\Octopus\Tentacle\Work\20211223025814-3591-39
02:58:26 Verbose | TempDirectory: C:\Windows\TEMP\
02:58:27 Verbose | HostProcess: powershell (4944)
02:58:27 Verbose | TotalPhysicalMemory: 4193844 KB
02:58:27 Verbose | AvailablePhysicalMemory: 2646640 KB
02:58:28 Info | Unable to connect to docker service
02:58:28 Error | NotSpecified: You will need docker installed and running to pull docker images
Thanks for getting in touch, though I’m very sorry to see you’re hitting this issue! It looks likely to be a case of a bad worker, unfortunately. Could you let me know your instance URL for me to have a look at? It might just require deactivating that worker so that a new one will be provisioned on the next deployment, which I can try to confirm and fix as soon as possible.
Sorry for the delay in response, but haven’t been in the office over Christmas. Initially it looked like waiting a while had finally resolved the issue, as I was able to deploy later that night (after the 6 hour timeout?).
But the issue is happening again today. This is quite frustrating as our Test team is working the week and really need to be able to move deployments around, and currently they are not able to.
Our instance is minnovare.octopus.app - let me know if you need any further information.
Just stepping in for Kenny while he’s offline for the Holidays.
I’ll go ahead and deactivate that worker so another one is provisioned which should have Docker installed this time.
Sorry that you keep hitting this issue, it’s normally quite rare and might indicate that there is a deeper problem. I’d be happy to investigate it further if it happens again!
Our engineers are currently looking into this; in the meantime, we’ve found the best workaround is to manually run a process step at the start of your deployment (or in a separate project/runbook) to restart the docker service on the worker.
Configure a Run a Script step to run on the worker, with the in-line source code of: Restart-Service docker
This seems to be allowing docker deployments to go ahead - can you try this out and let me know if this is successful for you, please?