When will Octopus Cloud workers update to .Net 5.0 and dotnet-script

Are Dynamic Worker Pools for WIndows & Linux with .Net 5/6 and dotnet-script coming?

.Net 5.0 has been out for a year and finally unified Framework/Core. In addition dotnet-script has been 1.x for a year and unified scripting for .Net Windows and Linux. The second unified release .Net 6 is imminent.

In this new world we would like to deprecate PowerShell and use only C# scripting on Windows and Linux. There is an opportunity to update and simplify Octopus Cloud workers to all use .Net 5/6 and dotnet-script. While we can roll our own workers, we can’t get past the limitation that Octopus only supports the now deprecated ScriptCS on Windows, and doesn’t appear to have any C# script support on Linux workers?

As an interim measure, could current Ubuntu workers at least have dotnet-script added (which still works with the only .Net Core 3.1 framework) so that both Windows and Ubuntu workers support C# scripts?

I checked blog articles but couldn’t find anything for Octopus’ plans for moving to .Net 5/6, yet .Net Core 3.1 only has one year left before it is no longer supported, so it would be good to know how/when OD will move to .Net 5/6?

This is the error we get with C# scripts for Octopus Cloud Ubuntu workers right now.

14:43:02   Verbose  |       Considering 'AWSSDK.S3' for extraction
14:43:02   Info     |       Extracting package '/home/Octopus/Files/AWSSDK.S3@S3.7.1.28@FAB3CAAAFD3A004E8194D16697B7F078.nupkg' to '/home/Octopus/Work/20210828144300-1133-30/AWSSDK.S3'
14:43:02   Verbose  |       Considering 'AWSSDK.SimpleSystemsManagement' for extraction
14:43:02   Info     |       Extracting package '/home/Octopus/Files/AWSSDK.SimpleSystemsManagement@S3.7.5.2@724F45949901394FA2A7CA9664C516C7.nupkg' to '/home/Octopus/Work/20210828144300-1133-30/AWSSDK.SimpleSystemsManagement'
14:43:02   Verbose  |       Considering 'Octopus.Client' for extraction
14:43:02   Info     |       Extracting package '/home/Octopus/Files/Octopus.Client@S11.3.3425@FA3C31EB506C4F48AC47A1171F25B78F.nupkg' to '/home/Octopus/Work/20210828144300-1133-30/Octopus.Client'
14:43:03   Verbose  |       Performing variable substitution on '/home/Octopus/Work/20210828144300-1133-30/Script.csx'
14:43:03   Verbose  |       Executing '/home/Octopus/Work/20210828144300-1133-30/Script.csx'
14:43:03   Verbose  |       Setting Proxy Environment Variables
14:43:03   Verbose  |       Unable to get the system proxy settings due to not running under .Net Framework. Calamari will not use any proxy settings.
14:43:03   Error    |       Unable to execute /home/Octopus/Tools/Calamari.linux-x64/19.4.4/ScriptCS/scriptcs.exe, please ensure that /home/Octopus/Tools/Calamari.linux-x64/19.4.4/ScriptCS/scriptcs.exe is installed and is in the PATH.
14:43:03   Error    |       System.Exception: Error when attempting to execute /home/Octopus/Tools/Calamari.linux-x64/19.4.4/ScriptCS/scriptcs.exe: Exec format error
14:43:03   Error    |       ---> System.ComponentModel.Win32Exception (8): Exec format error
....

Hi Aaron,

Thanks for reaching out!

There has been quite a bit of discussion around this topic internally and for the moment it looks like .Net 5.0 or later will not be supported out of the box on any dynamic worker.

Minimising the footprint for each dynamic worker has always been a major factor in deciding the direction of what does and does not get included on the dynamic worker and we are trending towards allowing customer to provide their own tools rather than bloating the software with tools that many customers don’t require.

The current recommendation is to use execution containers - with the expectation that execution containers will be used in an increasing number of solutions when working with dynamic workers as they can be configured with whichever tools are required, such as C# scripting or .Net 6.

Feel free to let me know if you have any questions at all, or if you have any issues configuring workers to use Execution Containers.

Bonus Blog post here with some tips around setting up containers with custom tools :slight_smile:

Best Regards,

1 Like

You’re going to keep using .Net Core 3.1 past EOL? Never update to .Net 5/6? Or you’re thinking to remove .Net from all worker images completely?

Using execution containers is great, and we’re doing that to work around this. It’s similar to the Bitbucket build approach, although they use Kubernetes instead of nested containers. The extra layer of provisioning a container within a container seems slower to provision that than Bitbucket approach. At least on cold runs.

Thanks.

Hi @Aaron_R ,

While we have made the move to .NET 5 with our 2021.2 release we are currently rolling out to Cloud Instances, our Dynamic Workers currently have no timeline for the move away from .NET Core 3.1. I’ve raised some internal discussions around the upcoming EoL for .NET Core 3.1 in Dec 2022 and will keep you posted if I have any more information. It’s likely that we will be upgrading, as the performance improvements in .NET 6 are looking pretty good, but unfortunately can’t confirm anything at this stage.

I’ve also re-ignited the internal discussion around using dotnet-script, which was raised in this feature request, I saw you noted your interest in, and will keep you updated if I hear anything.

That is an interesting approach by Bitbucket, I’ll have to check it out! We do leverage Kubernetes with cached images for our Dynamic Workers but execution containers are nested. Our advice post here goes into detail around how they work, but to avoid cold starts you could keep a worker warm by triggering a deployment to cache the image on a worker, then lease that same worker for the period you need, provided that isn’t hasn’t been idle for 60 mins it can be used for up to 72 hours. I can also slightly modify some of these values, let me know if you’d like any adjustments!

Feel free to let me know if you have any further questions. Sorry I don’t have a better answer for you but I’ll keep you updated if I hear anything from the discussions raised.

Best Regards,

1 Like

Thanks for the references and the info about warm workers. I see you’ve gone ‘old-school’ with VM images/pools more similar to the CodeBuild/CodePipeline model. That explains the lengthy (when cold) provisioning times.

My understanding is that Bitbucket uses multi-tenant Kubernetes nodes with user namespace remapping to keep sufficient isolation while still enabling privileged things like docker builds. Once you get your images pulled on disk at Bitbucket, provisioning times are negligible (~3 seconds), with no need to keep anything ‘warm’ other than the on-disk image cache. The current weakness is that multi-stage Docker builds don’t work fully because each stage get a different user namespace remapping, so the user ids don’t match when copying files between docker stages. But it is much more performant. Provisioning a Bitbucket worker with a custom execution container for a build stage takes ~3s. That’s basically Kubernetes Pod start-up time for a disk-cached image. We can do many-stage Bitbucket builds, with a different worker containers for each stage in less time than warming up a dynamic Octopus worker to deploy the resulting package :grinning:

On the other hand it must be much less stressful to ensure complete customer isolation with VM Octopus/CodeBuild model, plus you can do exotic things like mounting filesystems into your Step workers, like CodeBuild does for EFS. Also, you want Octopus Windows workers too, and doing Windows multi-tenant with Kubernetes is challenging and/or unsolved. So swings and roundabouts to be sure!

Anyway, when warm, Octopus dynamic workers with custom containers are working well for us. And if it is a real problem we can always revert to provisioning our own Octopus worker pools on our own Kubernetes clusters.

Thanks also for looking into dotnet-script again. Microsoft has done a ton of good work to unify and open-source not just the compiler & runtime, but the entire library framework into a compatible, cross-platform nirvana. Between dotnet, dotnet-script, and Blazor WASM, combined with cloud .Net CDKs and serverless, we see a lot of benefits at multiple levels and for varied projects. The .NET is dead, long live the dotnet :wink: