I seem to be getting the following errors while using the latest beta version of octopus to deploy zip files. Seems to me like there is some problems encountered while extracting zip files.
System.NotSupportedException: Memory stream is not expandable.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte buffer, Int32 offset, Int32 count)
at SharpCompress.IO.RewindableStream.Read(Byte buffer, Int32 offset, Int32 count)
at System.IO.BinaryReader.FillBuffer(Int32 numBytes)
2.NextEntryForCurrentStream() 20:48:40Error at SharpCompress.Reader.AbstractReader2.MoveToNextEntry()
at Calamari.Integration.Packages.ZipPackageExtractor.Extract(String packageFile, String directory, Boolean suppressNestedScriptWarning) in Y:\work\14ffc968155e4956\source\Calamari\Integration\Packages\ZipPackageExtractor.cs:line 19
at Calamari.Deployment.Conventions.ExtractPackageConvention.Install(RunningDeployment deployment) in Y:\work\14ffc968155e4956\source\Calamari\Deployment\Conventions\ExtractPackageConvention.cs:line 33
at Calamari.Deployment.ConventionProcessor.RunInstallConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 60
at Calamari.Deployment.ConventionProcessor.RunConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 28
Running rollback conventions…
System.ArgumentNullException: Value cannot be null.
Parameter name: path
at System.IO.DirectoryInfo…ctor(String path)
at Calamari.Integration.FileSystem.CalamariPhysicalFileSystem.EnumerateFiles(String parentDirectoryPath, String searchPatterns) in Y:\work\14ffc968155e4956\source\Calamari\Integration\FileSystem\CalamariPhysicalFileSystem.cs:line 181
at Calamari.Integration.Scripting.PackagedScriptRunner.FindScripts(RunningDeployment deployment) in Y:\work\14ffc968155e4956\source\Calamari\Integration\Scripting\PackagedScriptRunner.cs:line 55
at Calamari.Integration.Scripting.PackagedScriptRunner.RunScripts(RunningDeployment deployment) in Y:\work\14ffc968155e4956\source\Calamari\Integration\Scripting\PackagedScriptRunner.cs:line 31
at Calamari.Deployment.ConventionProcessor.RunRollbackConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 71
at Calamari.Deployment.ConventionProcessor.RunConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 48
at Calamari.Commands.DeployPackageCommand.Execute(String commandLineArguments) in Y:\work\14ffc968155e4956\source\Calamari\Commands\DeployPackageCommand.cs:line 113
at Calamari.Program.Execute(String args) in Y:\work\14ffc968155e4956\source\Calamari\Program.cs:line 38
The remote script failed with exit code 100
I solved this by rolling out a custom version of calamari. While doing this i also came across another issue with custom calamari. Seems like the tentacle is unable to accept a custom calamari properly anymore. It loads in the custom calamari and deletes it for some reason. I got it to deploy my custom version of calamari by replacing the one in C:\Program Files\Octopus Deploy\Octopus.
Just to add on to this, i replaced the original sharpcompress with ionic.zip. It seems like zip files generated from the teamcity compress plugin doesn’t work too well with sharpcompress.
If anyone is interested in the fixed version it’s here
Thanks for trying out our new 3.3 build.
It indeed seems as though you must be hitting upon a bug within the
sharpcompress library that we use for tar\zip\gzip extraction. We have tested across many zip packages with various compression methods but so far been unable to replicate the failure.
Would it be possible to get you to upload the package that was causing this failure to occur? We have a secure location at https://file.ac/iitU4hihCFg/ that you can upload this package to which no-one else but us can access. Alternatively you can attach and email them through to us at email@example.com. Feel free to create another sample zip package if you wish with some dummy contents, that exhibits the same behaviour.
If we can get to the bottom of this problem hopefully we can easily fix it to prevent it becoming an issue for other users. In the meantime I will further investigate the other issue you encountered regarding using a custom Calamari build.
Thanks again for getting in touch, and bringing these problems to our attention.
I have managed to replicate the failing problem you had with a custom Calamari build refusing to be used by the Tentacle. I have created a GitHub ticket for you to track its progress as it gets resolved.
Thanks again for these issues! We should have most of these kinks ironed out of the beta before full release.
Fanatastic! how about the compressed file issue? I think the compression library bundled with octopus might not be supporting a full range of zip files. Will you guys put in another zip library or improve this somehow?
Ah sorry please disregard my previous post on the compression library. I missed out the first reply. Will try to send you a sample of the affected zip file. Mine is generated using the compress plugin in teamcity (our packages are prepared by teamcity)
Thanks for the extra information, I have just downloaded and tested with the compress plugin that you mentioned. It appears to be able to extract just fine without any problems. Perhaps once I get the offending package I will be able to reproduce the issue and investigate what sort of fix needs to be put in to SharpCompress.
Thanks again for sending this through.
I have uploaded the file to https://file.ac/iitU4hihCFg/.
I actually stripped off all the files within the zip file and it is still crashing. Most likely it has something to do with the way the headers are constructed.
I tested the file with a very small piece of code which runs sharpcompress v0.11.4
static void Main(string args)
var file = @"file.zip";
using (var inStream = new FileStream(file, FileMode.Open, FileAccess.Read))
using (var reader = ZipReader.Open(inStream))
Thanks for providing the zip file. Unfortunately Octopus Deploy is still extracting it correctly during deployments, despite being empty. I have also attempted to run the script you provided with
sharpcompress 0.11.4 and again, with the file you provided it runs just fine. I did have some odd behaviour when trying to open the zip via finder on a Mac as described here. Opening in windows or with 7zip seemed to show the empty folder so the download itself seems ok, but perhaps archived in an unexpected format.
I cant rule out a problem with this external library but would love to reproduce it first before trying to replace or fix.
Perhaps I might be able to discover something If we schedule a support call. Perhaps looking at the packing and deployment directly we might discover the missing piece that allows us to hit the same bug.
Thanks again for reporting this issue, please let us know if you have any questions about setting up this call.
Sorry I didn’t get back to you sooner. It seems like the problem went away after updating to beta 2 so it could be already fixed. Although I found it strange that the library did not unzip even a blank file on my development machine
Great to hear that you no longer have this problem. Please don’t hesitate to contact us again if you see this problem recur. As we are using a 3rd party compression library we also need to make sure we keep up to date with all its updates so that fixes get passed straight on to our users.
We got the same problem here. The package cannot be fully extracted and stops with the expandable memory error message. Our package is quite big as of now (140MB compressed to 85 MB) and our biggest files are around 6 MB.
Update #1: Even with a reduced size package (85MB compressed to 32 MB), it fails.
Update #2: We also use TeamCity to generate the zip file.
We have Octopus 3.3.4 (server and Tentacle).
Thanks a lot
As a workaround, we use tar file from TeamCity instead of zip file and it works fine with it.
We JUST started experiencing this (we also use TeamCity builds to generate our ZIP package) and even after upgrading to the latest OctoDeploy (v3.5.1) and Calamari (v3.5.3) across our agents, we are still experiencing the error: System.NotSupportedException: Memory stream is not expandable
We are getting the same error with our .zip files inside our packages.
If we change the file extention to something else than .zip, it fixes the issue.
So instead of JDK/java8.zip we have JDK/java8.hest this works.
Its possible that this may be a problem with the third party library sharpcompress which we use for package extraction.
So that we can try and debug through this problem, would it be possible to get you to send the package (or any with the same probem) through to us. You can securely upload to via this address: https://file.ac/Lsy2rThn9LI/ which will only be accessable to ourselves.
Once we are able to reliably reproduce the issues, we may be able to patch the problem.
Thanks in advance,
I am currently running 3.4.11 (I’ve been meaning to upgrade for a while) and I’m seeing this issue.
If TeamCity runs a build with changes the package will not deploy–the above error is thrown.
If I run the same build again in TeamCity, the package WILL deploy.
The second build that gets run contains absolutely no commits so should be almost identical to the original package, differing only in build numbers.
My build chain contains 2 levels:
App (2 apps)
Product (1 product)
If I rerun the product build using existing snapshots of the Apps, Octopus will still have the error.
If I rerun one App which then kicks off the Product build, Octopus will still have the error.
If I rerun the entire build chain, Octopus WILL deploy successfully.
This just started happening for me today despite running the same build chain for almost 6 months.
I have attached the deployment log but it’s not of much use–it’s nearly identical to the error messages above.
For confidentiality purposes I cannot upload the package. This inevitably reduces the usefulness of my post–apologies.
ServerTasks-11052.log.txt (11 KB)
The fact that it has been working fine until recently tells me that there must be something either in your build environment or bundled source files that has changed. Without the zip package it is hard for us to diagnose what the problem may be since these sorts of issues are typically problems with the way in which the zip specifications are followed (either by the compressor or extractor).
Based on your error message my guess is that it may have been fixed as part of this previous GitHub issue that was released in
3.5.2. Please upgrade to the latest Octopus version and let us know if you are still experiencing this problem.
I upgraded to 3.10.0 and re-ran two deployments that were repeatedly failing and the issue is currently not reproducible.
Thank you for the prompt response!