We build one ZIP package in TeamCity using octo.exe. The package ends up being 750 Mb and it takes 2.5 minutes to create it. So far so good. Next TeamCity step is to push the package to Octopus internal store using “Push packages” Octopus build step. Below I pasted excerpt from TeamCity build log:
[19:23:01]Running command: octo.exe push --server http://TUKC-DEPLOY-01 --apikey SECRET --package D:\BuildAgent\work\cc8e4a0f4383b619\GeneratedPackages\PublishedWebsites.1.4.01202.14770.zip --replace-existing
[19:23:01]Pushing packages to Octopus server
[19:23:01]Octopus Deploy Command Line Tool, version 3.4.2+Branch.master.Sha.869d64eaf5d2657a5daadda886a2dd697e2a024c
[19:23:01]
[19:23:01]Handshaking with Octopus server: http://TUKC-DEPLOY-01
[19:23:01]Handshake successful. Octopus version: 3.6.0; API version: 3.0.0
[19:23:01]Authenticated as: “TeamCity” “(a service account)”
[19:23:01]Pushing package: “D:\BuildAgent\work\cc8e4a0f4383b619\GeneratedPackages\PublishedWebsites.1.4.01202.14770.zip”…
[19:25:21]Push successful
[19:25:21]Octo.exe exit code: 0
It is successful, but what rises my suspicion is that it takes 2 min and 20 sec between “Pushing package” and “Push successful”, i.e. approximately the same time it takes to create the package. I have tried to copy the package from build machine to Octopus machine manually and it merely took 5 seconds.
WHY “Pushing package” takes that long?? I have strong feeling that it does some unnecessary work. Could you please help me straighten it out?
For experiment I tried to combine packaging and pushing into one step (using TeamCity runner) and got the same result - packaging takes 2 min 20 sec and then pushing takes 2 min 11 sec. What does it do during the push which takes so long??
Could you try and use Octo.exe to push the package from your TeamCity server to Octopus and see if that too takes a very long time? See our docs for usage.
This is what happens when you push a package to Octopus:
We save the package to a temporary folder
We check if the package (package id and version) already exists in the feed
If the package exists and --replace-existing is specified
3a. We check the user is allowed to replace an existing package
3b. If they are we remove the existing package
If the package doesn’t already exist
4a. We check the user is allowed to push packages to the built-in feed
We add the package that was pushed
We do some necessary administrative logic, such as adding an event
If you have lots of large packages in the built-in repo, then step 2 could take some time and it’s not much we can do about that as it does IO related tasks.
We have experienced something similar with TeamCity and Pushing to Octopus. It looks like the same step as reported and jumped by 20 minutes in October (See Image).There is a lot of pushes under this step.
Individually these pushes increased to double what it was taking previously.
[16:26:02]push: Publish package output\Packages\Access.WebService.1.0.0.610.nupkg (22s)
[16:26:02]NuGet command: C:\Win32App\BuildAgent\plugins\nuget-agent\bin\JetBrains.TeamCity.NuGetRunner.exe C:\Win32App\BuildAgent\tools\NuGet.CommandLine.DEFAULT\tools\NuGet.exe push C:\Win32App\BuildAgent\work\37bddbc3712d3470\output\Packages\Access.WebService.1.0.0.610.nupkg %%teamcity_nuget_api_key_1481088362929%% -Source http://ampbnkftgp02:8098/nuget/packages?replace=true
[16:26:02]Starting: C:\Win32App\BuildAgent\temp\agentTmp\custom_script3048823404423753156.cmd
[16:26:02]in directory: C:\Win32App\BuildAgent\work\37bddbc3712d3470\output\Packages
[16:26:02]JetBrains TeamCity NuGet Runner 8.0.37377.9
[16:26:02]Registered additional extensions from paths: C:\Win32App\BuildAgent\plugins\nuget-agent\bin\plugins-2.8
[16:26:02]Starting NuGet.exe 2.8.60717.93 from C:\Win32App\BuildAgent\tools\NuGet.CommandLine.DEFAULT\tools\NuGet.exe
[16:26:03]Pushing Access.WebService 1.0.0.610 to ‘http://ampbnkftgp02:8098/nuget/packages?replace=true’…
[16:26:25]Your package was pushed.
[16:26:25]Process exited with code 0