OctoPack Push Fails on first Build of Day

During the build task we are getting the following errors on our nupkg. I have checked to ensure that version is not a duplicate . nothing is shown in the Octopus logs for the event or in the Windows logs.
it does not always happen on the first build for a build definition every day but seems out of our 9 build defs, 6 will fail, after that all builds work.

FeatureB\Products\packages\OctoPack.3.6.3\build\OctoPack.targets(109):FeatureB\Products\packages\OctoPack.3.6.3\build\OctoPack.targets(109,5): Error MSB3073: The command "“C:\agent_work\17\s\FeatureB\Products\packages\OctoPack.3.6.3\build\nuget.exe” push “C:\agent_work\17\s\FeatureB\Services\Document Conversion Service\Merchants.DocumentConversionWS\obj\octopacked\Merchants.DocumentConversionWS.2018.2.27.1-featureB.nupkg” ******** -Source http://octopus.merchantsfleet.com/nuget/packages " exited with code 1.

Hi Richard,

Thanks for getting in touch. It seems that this is unfortunately not the first time we’ve heard of NuGet.exe behaving like this, see Nuget Push Errors, How to Troubleshoot?.

As was mentioned in that other thread, there can sometime be more information further back in the log file, would you be able to check if there are any other error entries further back on the build log?

I’m wondering if it’s related to a timeout or something like that during the push, e.g. if there was a high level of network traffic or something at the time then maybe Nuget.exe times out and returns us that error. Is the package large enough that a timeout could be possible/likely?

Regards
Shannon

I checked early in the build logs and nothing for other error also this occurs at any time of the the day and only seems to happen for the first build of a build definition of that day but not always every day. I have had some of this builds fail in the middle of the night when the load on the network is very low and these server are not busy. I even tried different times for the nightly builds and still get those build failures if it is the first build of the day.

It’s probably a long shot, but would it be possible for you to try using OctoPack to just create the package and then have a second step that uses Octo.exe push to push the package to Octopus?

The subtle difference is that OctoPack uses NuGet.exe to do the push and Octo.exe uses HttpClient. I suspect the issue is actually network related, so like I said it’s probably a long shot, but it may reveal something.

Regards
Shannon

Trying some different thing I get this now on builds x.1
2018-03-12T15:01:40.9223814Z OctoPack: Successfully created package ‘C:\agent_work\4\s\Main\Services\Security\Merchants.SecurityWS\obj\octopacked\Merchants.SecurityWS.2018.3.12.1-main.nupkg’.
2018-03-12T15:01:40.9223814Z OctoPack: Copy file: C:\agent_work\4\s\Main\Services\Security\Merchants.SecurityWS\obj\octopacked\Merchants.SecurityWS.2018.3.12.1-main.nupkg
2018-03-12T15:01:40.9692616Z OctoPack: Packaging finished successfully
2018-03-12T15:01:40.9692616Z Publish to repository: http://octopus.merchantsfleet.com/nuget/packages
2018-03-12T15:01:40.9692616Z “C:\agent_work\4\s\Main\Products\packages\OctoPack.3.6.3\build\nuget.exe” push “C:\agent_work\4\s\Main\Services\Security\Merchants.SecurityWS\obj\octopacked\Merchants.SecurityWS.2018.3.12.1-main.nupkg” ******** -Source http://octopus.merchantsfleet.com/nuget/packages
2018-03-12T15:01:41.4380248Z NuGet Version: 3.5.0.38733 (Custom build for OctoPack. See http://g.octopushq.com/VersioningInOctopusDeploy)
2018-03-12T15:01:41.5005383Z Pushing Merchants.SecurityWS.2018.3.12.1-main.nupkg to ‘http://octopus.merchantsfleet.com/nuget/packages’…
2018-03-12T15:01:41.5319464Z PUT http://octopus.merchantsfleet.com/nuget/packages/
2018-03-12T15:01:42.4693609Z Conflict http://octopus.merchantsfleet.com/nuget/packages/ 923ms
2018-03-12T15:01:42.4693609Z Response status code does not indicate success: 409 (Conflict).
2018-03-12T15:01:42.4849613Z ##[error]Main\Products\packages\OctoPack.3.6.3\build\OctoPack.targets(109,5): Error MSB3073: The command "“C:\agent_work\4\s\Main\Products\packages\OctoPack.3.6.3\build\nuget.exe” push “C:\agent_work\4\s\Main\Services\Security\Merchants.SecurityWS\obj\octopacked\Merchants.SecurityWS.2018.3.12.1-main.nupkg” ******** -Source http://octopus.merchantsfleet.com/nuget/packages " exited with code 1.
2018-03-12T15:01:42.4849613Z C:\agent_work\4\s\Main\Products\packages\OctoPack.3.6.3\build\OctoPack.targets(109,5): error MSB3073: The command "“C:\agent_work\4\s\Main\Products\packages\OctoPack.3.6.3\build\nuget.exe” push “C:\agent_work\4\s\Main\Services\Security\Merchants.SecurityWS\obj\octopacked\Merchants.SecurityWS.2018.3.12.1-main.nupkg” ******** -Source http://octopus.merchantsfleet.com/nuget/packages " exited with code 1. [C:\agent_work\4\s\Main\Services\Security\Merchants.SecurityWS\Merchants.SecurityWS.csproj]
2018-03-12T15:01:42.4849613Z Done Building Project “C:\agent_work\4\s\Main\Services\Security\Merchants.SecurityWS\Merchants.SecurityWS.csproj” (default targets) – FAILED.

Could I be doing something wrong in passing the MSBuild arguments for the Visual Studio Build Task

/p:DeployOnBuild=true /p:PublishProfile=Build /p:RunOctoPack=$(RunOctoPack) /p:OctoPackPublishPackageToHttp=$(OctoPackPublishUrl) /p:OctoPackPublishApiKey=$(OctopusApiKey) /p:OctoPackAppendToVersion=$(SemVerPrereleaseTag) /p:OctoPackNuGetProperties=BuildConfiguration=$(BuildConfiguration)

I can’t see anything that looks wrong there. The use of /p:DeployOnBuild=true /p:PublishProfile=Build probably isn’t necessary, OctoPack hooks in to the Build target rather than the Publish target, so there probably isn’t real value there.

I don’t think those arguments would cause what you’re seeing, but it could be worth trying without them. The 409 (Conflict) is what will get returned if that package Id and Version combination already exists in the feed. Is it possible that this same build ran twice or something like that?

I check the logs to see if that build statement ran twice, it did not and we make sure for every build version is different.

This is very intriguing. You said you’d checked it wasn’t a duplicate, was that by checking that version didn’t exist in the Octopus feed for the Merchants.SecurityWS package? Would you be able to attach a screenshot of the versions list from the feed for that package, just in case anything there stands out?

Could I get you to open the nupkg file and check what it shows as it’s version in the nupsec metadata? (either with Nuget Package Explorer or rename it to Zip and extract it to see the nuspec file content). If the version inside the package doesn’t match the filename, for example it’s missing the -main somehow, then this could explain the error you’re seeing.

Shannon

This one failed today
from the psmdcp file we have
dc:identifierMerchants.SecurityWS</dc:identifier>
2018.3.21.1-main

packages.json (30.0 KB)

From that json file, that version is definitely in the feed already.

      "Version": "2018.3.21.1-main",
      "Description": "Windows service that hosts a WCF service that provides security-related functions",
      "Published": "Wednesday, March 21, 2018 2:15 AM",

The Published date shown there is 2:15AM, does that correlate with the time of the failed build?

Shannon

No my build kicks off at 5:30 am every morning and I see that every 2:15 am that it is in the feed how is that possible when I am not building and pushing that out.

I found the guys that copied my build tasks and doing a build on another build server outside of my control.

So how can I over write or determine if the package already exists in my build tasks?

Ah, mystery solved!

I wasn’t sure you could do an overwrite when using OctoPack, but after discussion with the team I learned that you can. You just need an adjustment to the server Url to include ?replace=true.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.