I am currently migrating all my projects from using the bundled Azure tools to execution containers. I am using the octopusdeploy/worker-tools v3.3-windows.ltsc2019 image and am having no issues with the deployment of Nuget packages created with the Octo CLI pack command. However, I have other packages that I am creating as Zip files using the zip command in Bash, and these seem to be the ones throwing an exception. Here is an excerpt from the output I’m getting:
Entry is trying to write a file outside of the destination directory.
SharpCompress.Common.ExtractionException
Thanks for reaching out. Sorry to hear you’re running into this.
Would you be able to upload your package for us to take a look at it to see what might be causing it? You can upload via going to www.octopus.com, clicking your profile in the upper right and clicking control center, then clicking Support Files in the left hand side.
Please let me know if that’s something you’re able to do, and if so, when you have finished uploading(we don’t get any notifications of any uploads.) If also possible, can you please upload a package that does work in your new process? Can you also please let me know which version of Octopus you’re running as well as which step type you’re using when it fails, is it a simple Deploy Package step?
I’ve uploaded two packages. PI-SPA-SERVER is deploying correctly. PI-CLIENT-HIRE is not. As previously mentioned, SPA is being packed with the use of the Octo CLI, and CLIENT-HIRE is being zipped up using the zip command in Bash.
I just upgraded my server last night, so when the problem started, I was on 2021.3.8275 but I am currently on 2022.1.2133. The behavior has stayed consistent in both versions.
The step type is Deploy An Azure Web App (Web Deploy).
Both of the packages I’ve uploaded have been working with the same build process for a long time using the bundled Azure tools; it’s just when we tried migrating to the containers that something has gone off track.
You’re very welcome, and thanks for sending those files over! I’ve reproduced the issue and interestingly, it only fails on the Windows worker, windows execution container combo. The Linux worker, linux execution container combo works.
I’m going to pass this on to our engineers and see if we can get to the bottom of this.
Please feel free to reach out in the meantime with any other questions or concerns.
I suspect there’s something in the way the Bash zip compresses the files that the container tools don’t like. I’m going to proceed with my migration, omitting the projects that deploy these types of Zip files for now and look at a possible workaround that leverages the Octo CLI for packing the problematic packages as a replacement for Bash zip, and hopefully we’ll all meet somewhere on the road in the middle to solve this soon. Thanks for verifying that and escalating!
I had some chats with our engineers and we were curious what commands you use to zip the file in bash(the entire process with all parameters would be helpful) were so we can try and reproduce it on our end.
Hey Jeremy, just saw this. Apologies for the delay in replying.
We are building these packages in CircleCI in an Ubuntu-based container. Here’s the snippet of code we use to produce the Zip file. The build outputs to the /project/hire directory and we just use a very basic set of commands to produce the Zip. I hope this helps.
cd ~/project/hire
zip -r ${PACKAGE} client/dist
zip -ur ${PACKAGE} server
zip -ur ${PACKAGE} web.config
zip -ur ${PACKAGE} public
I created some dummy folders+files and used your zip steps and everything went fine on deployment, then I unzipped your zip and rezipped it using your commands and it still had the issue, so it does feel like it’s something within the file itself and not the method.
I’m going to pass my findings along to the engineers and we’ll see if we can pinpoint what’s causing it.
I’ll update you as it progresses but please feel free to reach out in the meantime.
I have absolutely no evidence to back this up, but my hunch is that the Ubuntu zip command creates or updates the zip file in such a way that the Windows deployment container doesn’t like it.
I wonder what would happen if we tried it with just that first Zip command and omitted the subsequent commands to add files to the existing Zip?
We’re actually unable to make this break with anything other than the content of your package. It seems odd that your package specifically could be causing the error. Can you think of anything unique you might have in this package that might not be in a typical package? Or the structure itself? I did create the base folders and threw some blank files in those folders but it didn’t break them.
The only thing that occurs to me - and this seems like maybe a stretch - is the command we are using to add the web.config. Because we are zipping with the -ur flag, I wonder if we are confusing Zip or telling it to do something weird, because that’s a file rather than a directory.
I’ve got another couple of tests I want to try with your specific package to see if I can isolate the cause. In the meantime, would you be able to grab the files before they get zipped in your pipeline on ubuntu, and transfer the raw files out to a windows box, zip them there via other means, and see if they deploy okay? If you’re not in a position to deploy that package, you could upload it and I can try it on my side like I have been. This will at least confirm if it’s the contents of the package or the method of zipping that’s causing it.
I can see about that, but it will probably be the end of the week. If I’m the only person reporting this, I’m starting to think I should pursue changing my builds to use octo pack instead rather than spending time chasing after a weird edge case that only affects me.
Unfortunately, octo pack is still supported but will no longer be maintained, so if you are going to go down that route and change your pipelines, we should look at using something like dotnet pack or another archiving utility.
I did find that unzipping it locally on windows, and zipping only the server folder causes it to break still, but the client and public folders are fine, so I may try a few more divisions and see if I can narrow down what it is.
I narrowed it down to 1 file. \server\node_modules\console-control-strings\README.md~
I believe the issue is the ~ at the end. Would you be able to check some of the other packages having the issue to see if they also have a ~ somewhere in the file/folder names?
Thank you so much for posting those article issues they are extremely helpful (especially the second one) and it does look like they confirm what Jeremy has found with the ~ at the end of the (seems like additional) readme file in this package.
Hopefully @drambo1 can take a look at the links you provided and edit or delete (if its not needed) that readme file and try zipping the folder and migrating it over.
We love it when our customers help us out like this and get involved in discussions so again, thank you for taking the time to post that up!