Does Octopus support hardlinks inside TAR?

Legacy Uncategorised
1

From documentation I figured Octopus understands .tgz (or .tar.gz) package format. We actively use TAR, because it is the only archiver which understands NTFS hardlinks and is capable of storing very short relative links within the package instead of copies of the same file. Our build drops tend to have A LOT of duplicate files in there and using TAR with hardlinks literally reduces the size of the build drop ten-fold, even without GZ compression.

The question is - would Octopus understand and be able to unpack TAR files with relative links encoded accordingly to TAR standard?
FYI we create and unpack such TAR files using 7zip.

Konstantin

2

Hi Konstantin,

Thanks for getting in touch and great question. I had to test this scenario as we weren’t sure of how it would be treated. That said, it doesn’t appear to work. We utilise SharpCompress to help process tar files but it doesn’t fully support tar files. I found the following issue in their repo.

I’m going to follow-up with the rest of the Octopus team to see what are options are for this. We have previously submitted pull-requests with changes to SharpCompress but I’m not sure of the complexity of a change like this. Tar is a fairly complex archive format.

Hope this helps!

Rob

3

Have you looked at https://sevenzipsharp.codeplex.com/ (it is also in NuGet). It is just a wrapper around 7zip library (which is also open source BTW) and 7zip library happily handles hadlinks and symlinks in TAR archives. May be just have a separate code path for TAR which uses different library?

4

Hi Konstantin,

Thanks for the reply. We’ve looked at most of the options and the key thing we found is that there isn’t a perfect solution and every other library has its own flaws and missing functionality. We chose SharpCompress as it has the best support for our needs and it’s open source and improving. The best thing is that the community including Octopus can submit pull requests to improve it. We’ve already done this and it’s a good way to improve it.

In this case, this is the first time we’ve had a need for hard link support in tar and I think there are two options.

  1. Visit our user voice site at http://octopusdeploy.uservoice.com and add a suggestion for this. We review our user voice suggestions regularly and action the the popular ones.
  2. Review SharpCompress and submit a pull request (PR) with added support. We try to stay fairly up to date with its releases so we’d be able to easily pull in an updated release with a fix.

Hope this helps.

Rob

5

Notice:

This issue has been closed due to inactivity. If you encounter the same or a similar issue and require help, please open a new discussion (if we asked for logs or extra details in this thread, consider including them in the new thread). If you are the creator of this thread and believe it should not be closed let us know via our support email.