Nuget tempcache files on workers filling up disk

Hi,

When running deployments on octopus workers and using an external package feed, I’m seeing the following path grow and filling up disk space: C:\Windows\Temp\NuGet\TempCache

I related this path to the C:\Octopus\Files\Feeds-NNNN folder, because, when cleaning up both paths and running a deployment the same amount of files are created and if I edit some files on the NuGet\TempCache I see they are xml files refering to the downloaded packages

The issue is also that the TempCache path can’t be set (I’ve tried setting NUGET_PACKAGES environmennt variable but it was not used)

FYI, the system variable Octopus.Action.Package.DownloadOnTentacle is set to true

Can you provide some extra information about how packages are being pulled from the external feed? Do you know if it is possible to override the TempCache path setting?

Hi @pazrodrigo,

Thank you for contacting Octopus Support.

Here is a link to our repo that shows exactly how we handle Nuget feed downloads: Calamari/NuGetV3LibDownloader.cs at master · OctopusDeploy/Calamari · GitHub

Could you tell me what version of Octopus you are currently running? Also, if you haven’t already, can confirm that the Windows Event log isn’t showing an error (access denied or otherwise) for C:\Windows\Temp\NuGet\TempCache?

Let me know at your earliest convenience.

Best Regards,
Donny

Hi Donny, thanks

I guess the targetFilePath parameter would be the Octopus\Tentacle\Files\Feeds-NNNN path? Then there is a tempTargetFilePath on Calamari/InternalNuGetPackageDownloader.cs at master · OctopusDeploy/Calamari · GitHub

Finally a fileSystem.MoveFile(tempTargetFilePath, targetFilePath);

Strangely there is very little info about NuGet\TempCache, but C:\Windows\Temp is the default temp dir for the system user, which runs the octo tentacle. I didn’t see any errors or warnings for that (or any other path) when running a deployment.

I’ll run the tentacle as a user with admin permissions to see if the files are saved on the same path.

We’re running octopus Version 2021.3 (Build 8275)

Hi @pazrodrigo,

Just stepping in for Donny from the Australia based team.

I was able to reproduce the C:\Windows\Temp\NuGet\TempCache folder filling up and found that by changing Tentacle to run as a Service Account instead of using the System account, the folder was no longer being filled.

It seems that this was what you were already going to try so please feel free to let us know if that doesn’t work for you or you have any questions!

Happy Deployments!

Hi Finnian,

I did the same and the nuget temp dir is not being created. I monitored the drive for a bit and found no unusual growth yet. I also did not find any files related to nuget temp or caches, which leads me to believe they are being removed after creation.

This said, we have automated scripts for tentacle installation which use the tentacle.exe CLI with the default settings. Isn’t this issue a bug? Since the standard settings run the service as SYSTEM, it should be able to remove files from the temp dir

EDIT: I’m attaching the process explorer logs for that path. I see that the file is being created but not deleted after acquiring the package via this command:
“C:\Octopus\Tools\Calamari.win-x64\20.4.0\Calamari.exe” download-package

Hi @pazrodrigo,

Thanks for confirming that!

I agree that there does seem to be something strange going on, it looks like there has been some recent changes to Calamari that could be relevant to the PackageCache.

I’ve enquired with the engineer who made the changes and raised some discussions internally. I’ll let you know as soon as I have any updates!

Best Regards,

Hi @pazrodrigo,

Just an update that I’ve created a Github Issue for this if you wanted to track when it’s resolution:

Feel free to reach out again anytime you have any issues or questions, thanks for raising this!

Best Regards,

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.