Octopus cloud hung on deployment


(Frank Heljebrandt) #1

I have a deployment that has hung on acquire packages since yesterday. Have tried to restart the tentacle buit with no success. I also tried to cancel the deployment.

Including log below:

Task ID: ServerTasks-3608
Task status: Cancelling
Task queued: Tuesday, November 6, 2018 1:13:41 PM
Task started: Tuesday, November 6, 2018 1:13:41 PM
Task duration: 19 hours
Server version: 2018.9.0+Branch.origin-master.Sha.46395c8392a50ac9abfa78f35004f0fc6bf494b7
Server node: EC2AMAZ-J1D75PT

                | == Running: Deploy FileImporter release 0.0.160 to Test for VSPerfTest ==

13:13:41 Verbose | Guided failure is not enabled for this task
13:21:52 Info | Requesting cancellation…
|
| == Success: Step 1: Stop services with 245 ==
13:13:45 Verbose | Stop services with 245 completed
|
| Success: VSPERFTEST
13:13:41 Verbose | Octopus Server version: 2018.9.0+Branch.origin-master.Sha.46395c8392a50ac9abfa78f35004f0fc6bf494b7
13:13:41 Verbose | Environment Information:
| OperatingSystem: Microsoft Windows NT 10.0.14393.0
| OsBitVersion: x64
| Is64BitProcess: True
| CurrentUser: EC2AMAZ-J1D75PT\svcOctopusServer
| MachineName: EC2AMAZ-J1D75PT
| ProcessorCount: 2
| CurrentDirectory: C:\Windows\system32
| TempDirectory: C:\Users\svcOctopusServer\AppData\Local\Temp
| HostProcessName: Octopus.Server
| PID: 1824

                |   == Running: Acquire packages ==

13:13:46 Info | Acquiring packages
13:13:46 Info | Making a list of packages to acquire
13:13:46 Verbose | No packages are required on the Octopus Server
13:13:46 Verbose | Delta compression is enabled for package transfers from the Octopus Server to deployment targets
13:13:46 Verbose | Upload FSSystem_Processor_Availability_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_Bid_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_Detail_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_FileReader_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_Hierarchy_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_Link_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_Marketing_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_PostCalculation_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_Price_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_ProductOption_Service.5.0.1-13514.win-x64 to VSPERFTEST
| Upload FSSystem_Processor_Specification_Service.5.0.1-13514.win-x64 to VSPERFTEST

13:13:46 Verbose | Checking package cache for package FSSystem_Processor_PostCalculation_Service v5.0.1-13514.win-x64
13:13:46 Info | Package FSSystem_Processor_PostCalculation_Service v5.0.1-13514.win-x64 was found in cache. No need to download.
13:13:46 Verbose | Using file: E:\Octopus\Packages\FSSystem_Processor_PostCalculation_Service\FSSystem_Processor_PostCalculation_Service.5.0.1-13514.win-x64.zip
13:13:46 Verbose | Checking package cache for package FSSystem_Processor_Link_Service v5.0.1-13514.win-x64

13:14:54 Info | Applying delta: 0%
13:14:54 Info | Applying delta: 10%
13:14:54 Info | Applying delta: 20%
13:14:54 Info | Applying delta: 30%
13:14:54 Info | Applying delta: 40%
13:14:54 Info | Applying delta: 50%
13:14:54 Info | Applying delta: 60%
13:14:54 Info | Applying delta: 70%
13:14:54 Info | Applying delta: 80%
13:14:54 Info | Applying delta: 100%
13:14:54 Verbose | Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in D:\FSSystem\OctopusDeploy\Work\20181106131451-3608-1597 exited with code 0
13:15:00 Info | Delta for package FSSystem_Processor_Marketing_Service v5.0.1-13514.win-x64 successfully uploaded and applied.
|
| Running: Upload package FSSystem_Processor_PostCalculation_Service v5.0.1-13514.win-x64
13:13:57 Verbose | Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory ‘D:\FSSystem\OctopusDeploy\Work\20181106131350-3608-1583’ using ‘OEM United States’ encoding running as ‘NT AUTHORITY\SYSTEM’ with the same environment variables as the launching process
13:13:57 Verbose | Octopus Deploy: Calamari version 4.9.15
13:13:57 Verbose | Environment Information:
13:13:57 Verbose | OperatingSystem: Microsoft Windows NT 10.0.14393.0
13:13:57 Verbose | OsBitVersion: x64
13:13:57 Verbose | Is64BitProcess: True
13:13:57 Verbose | CurrentUser: NT AUTHORITY\SYSTEM
13:13:57 Verbose | MachineName: VSPERFTEST
13:13:57 Verbose | ProcessorCount: 6
13:13:57 Verbose | CurrentDirectory: D:\FSSystem\OctopusDeploy\Work\20181106131350-3608-1583
13:13:57 Verbose | TempDirectory: C:\Windows\TEMP
13:13:57 Verbose | HostProcessName: Calamari
13:13:57 Verbose | Package FSSystem_Processor_PostCalculation_Service version 5.0.1-13514.win-x64 hash c1dcabe22640b2820dda8403c403fe036ce8111a has not been uploaded.
13:13:57 Verbose | Finding earlier packages that have been uploaded to this Tentacle.
13:13:57 Verbose | Found 3 earlier versions of FSSystem_Processor_PostCalculation_Service on this Tentacle
13:13:57 Verbose | - 5.0.1-13504.win-x64: D:\FSSystem\OctopusDeploy\Files\FSSystem_Processor_PostCalculation_Service@S5.0.1-13504.win-x64@C3C1022A59A6984390F7A21177760955.zip
13:13:57 Verbose | - 5.0.1-13503.win-x64: D:\FSSystem\OctopusDeploy\Files\FSSystem_Processor_PostCalculation_Service@S5.0.1-13503.win-x64@3074EF6BBDE22C4A8FE2F4CF58BD14F5.zip
13:13:57 Verbose | - 5.0.0-13293.win-x64: D:\FSSystem\OctopusDeploy\Files\FSSystem_Processor_PostCalculation_Service@S5.0.0-13293.win-x64@C6ACC446D03B6E4A965F28A3AE417129.zip
13:13:57 Verbose | Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in D:\FSSystem\OctopusDeploy\Work\20181106131350-3608-1583 exited with code 0
13:14:18 Verbose | Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory ‘D:\FSSystem\OctopusDeploy\Work\20181106131411-3608-1593’ using ‘OEM United States’ encoding running as ‘NT AUTHORITY\SYSTEM’ with the same environment variables as the launching process
13:14:18 Verbose | Octopus Deploy: Calamari version 4.9.15
13:14:18 Verbose | Environment Information:
13:14:18 Verbose | OperatingSystem: Microsoft Windows NT 10.0.14393.0
13:14:18 Verbose | OsBitVersion: x64
13:14:18 Verbose | Is64BitProcess: True
13:14:18 Verbose | CurrentUser: NT AUTHORITY\SYSTEM
13:14:18 Verbose | MachineName: VSPERFTEST
13:14:18 Verbose | ProcessorCount: 6
13:14:18 Verbose | CurrentDirectory: D:\FSSystem\OctopusDeploy\Work\20181106131411-3608-1593
13:14:18 Verbose | TempDirectory: C:\Windows\TEMP
13:14:18 Verbose | HostProcessName: Calamari
13:14:18 Verbose | Package FSSystem_Processor_PostCalculation_Service version 5.0.1-13514.win-x64 hash c1dcabe22640b2820dda8403c403fe036ce8111a has not been uploaded.
13:14:18 Verbose | Finding earlier packages that have been uploaded to this Tentacle.
13:14:18 Verbose | Found 3 earlier versions of FSSystem_Processor_PostCalculation_Service on this Tentacle
13:14:18 Verbose | - 5.0.1-13504.win-x64: D:\FSSystem\OctopusDeploy\Files\FSSystem_Processor_PostCalculation_Service@S5.0.1-13504.win-x64@C3C1022A59A6984390F7A21177760955.zip
13:14:18 Verbose | - 5.0.1-13503.win-x64: D:\FSSystem\OctopusDeploy\Files\FSSystem_Processor_PostCalculation_Service@S5.0.1-13503.win-x64@3074EF6BBDE22C4A8FE2F4CF58BD14F5.zip
13:14:18 Verbose | - 5.0.0-13293.win-x64: D:\FSSystem\OctopusDeploy\Files\FSSystem_Processor_PostCalculation_Service@S5.0.0-13293.win-x64@C6ACC446D03B6E4A965F28A3AE417129.zip
13:14:18 Verbose | Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in D:\FSSystem\OctopusDeploy\Work\20181106131411-3608-1593 exited with code 0
13:14:24 Info | Found matching version 5.0.1-13504.win-x64: E:\Octopus\Packages\FSSystem_Processor_PostCalculation_Service\FSSystem_Processor_PostCalculation_Service.5.0.1-13504.win-x64.zip
13:14:24 Info | Building delta for FSSystem_Processor_PostCalculation_Service v5.0.1-13514.win-x64 (34.361 MB)
13:14:24 Info | Using package E:\Octopus\Packages\FSSystem_Processor_PostCalculation_Service\FSSystem_Processor_PostCalculation_Service.5.0.1-13504.win-x64.zip with hash a6d7e8099b7517d29a544832c3d99f5686743da5 for creating delta.
13:14:24 Verbose | Building signature file: C:\Octopus\OctopusServer\PackageCache\FSSystem_Processor_PostCalculation_Service.5.0.1-13504.win-x64.zip.octosig
13:14:24 Verbose | - Using nearest package: E:\Octopus\Packages\FSSystem_Processor_PostCalculation_Service\FSSystem_Processor_PostCalculation_Service.5.0.1-13504.win-x64.zip
|
| Success: Upload package FSSystem_Processor_Link_Service v5.0.1-13514.win-x64

13:15:20 Info | Delta for package FSSystem_Processor_Specification_Service v5.0.1-13514.win-x64 successfully uploaded and applied.
|
| == Pending: Step 2: FileReader ==
|
| == Pending: Step 3: Hierarchy ==
|
| == Pending: Step 4: Availability ==
|
| == Pending: Step 5: Detail ==
|
| == Pending: Step 6: Link ==
|
| == Pending: Step 7: Marketing ==
|
| == Pending: Step 8: ProductOption ==
|
| == Pending: Step 9: Specification ==
|
| == Pending: Step 10: Bid ==
|
| == Pending: Step 11: Price ==
|
| == Pending: Step 12: PostCalculation ==
|
| == Pending: Step 13: Start services ==


(Pawel Pabich) #2

Hi Frank,

Thanks for getting in touch! We are in the process of rolling out a fix for this problem. Once your instance is upgraded to 2018.9.4 your problem should be solved.

Please let me know if I can help you in any other way.

Regards,

Pawel