Upgrading to v2020.5.4 from v2019.12.2 can no longer deploy via Calamari

[antmeehan (Anthony Meehan)]

After we upgraded from v2019.12.2 to v2020.5.4, we started getting the following exception from Calamari:

An exception was thrown while executing a resolve operation. See the InnerException for details. —> Deployed package must have a version (See inner exception for details.)
Autofac.Core.DependencyResolutionException
at Autofac.Core.Resolving.ResolveOperation.Execute(IComponentRegistration registration, IEnumerable1 parameters) at Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable1 parameters, Object& instance)
at Autofac.ResolutionExtensions.ResolveService(IComponentContext context, Service service, IEnumerable1 parameters) at Autofac.ResolutionExtensions.Resolve[TService](IComponentContext context, IEnumerable1 parameters)
at Calamari.Common.CalamariFlavourProgram.Run(String args)
–Inner Exception–
Deployed package must have a version
Calamari.Common.Commands.CommandException
at Calamari.Common.Features.Deployment.Journal.DeployedPackage…ctor(String packageId, String packageVersion, String deployedFrom)
at Calamari.Common.Features.Deployment.Journal.DeployedPackage.FromJournalEntryElement(XElement deploymentElement)
at Calamari.Common.Features.Deployment.Journal.JournalEntry…ctor(XElement element)
at Calamari.Common.Features.Deployment.Journal.DeploymentJournal.<>c.b__8_0(XElement element)
at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext() at System.Collections.Generic.List1…ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source)
at Calamari.Common.Features.Deployment.Journal.DeploymentJournal.GetAllJournalEntries()
at Calamari.Common.Features.Deployment.Journal.DeploymentJournal.GetLatestInstallation(String retentionPolicySubset, String packageId, String packageVersion)
at Calamari.Common.Plumbing.Variables.DeploymentJournalVariableContributor.Previous(IVariables variables, IDeploymentJournal journal, String policySet)
at Calamari.Common.Plumbing.Variables.DeploymentJournalVariableContributor.Contribute(ICalamariFileSystem fileSystem, IVariables variables)
at Calamari.Common.Plumbing.Variables.VariablesFactory.Create(CommonOptions options)
at Autofac.Builder.RegistrationBuilder.<>c__DisplayClass11.<ForDelegate>b__0(IComponentContext c, IEnumerable1 p)
at Autofac.Core.Activators.Delegate.DelegateActivator.ActivateInstance(IComponentContext context, IEnumerable1 parameters) at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable1 parameters)
at Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func1 creator) at Autofac.Core.Resolving.InstanceLookup.Execute() at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable1 parameters)
at Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
at Autofac.Core.Activators.Reflection.ReflectionActivator.ActivateInstance(IComponentContext context, IEnumerable1 parameters) at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable1 parameters)
at Autofac.Core.Resolving.InstanceLookup.Execute()
at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable1 parameters) at Autofac.Core.Resolving.ResolveOperation.Execute(IComponentRegistration registration, IEnumerable1 parameters)

I’ve resolved the issue by editing the DeploymentJournal.xml file in C:\Octopus on the deployment target and removing any <Deployment> elements that had empty PackageVersion attributes. The deployments were from many years ago (in fact it seems the XML schema has changed at some point to have child <Package> elements?).

Is this a safe thing to do? I’m going to have to write some PowerShell to perform this clean-up across all our deployment targets.

Many thanks,

Ant.

[antmeehan (Anthony Meehan)]

This is the PowerShell I’ve used (and executed via the Script Console of course )

[xml]$xml = [xml](Get-Content C:\Octopus\DeploymentJournal.xml)
$xml.Save("C:\Octopus\DeploymentJournal.xml.bak");

($xml.Deployments.ChildNodes | Where-Object -Property 'PackageVersion' -EQ '') | ForEach-Object { [void] $_.ParentNode.RemoveChild($_)}
$xml.Save("C:\Octopus\DeploymentJournal.xml");

[Kenneth_Bates]

Hi Anthony,

Thank you for getting in touch and reporting this issue (and for sharing the details of your solution!). My apologies for the headache it caused.

The PowerShell you used is safe, but it certainly shouldn’t error this way so there must be some unknown bug lurking that I’d need to look more into. I imagine you might be right that it’d be due to very old deployments where there would have been some important differences in the data that the upgrade scripts aren’t handling correctly. So that I can attempt a reproduction, do you remember roughly which version you would have been on when these deployments were first created?

I look forward to hearing back!

Best regards,

Kenny

[antmeehan (Anthony Meehan)]

Hi Kenny,

I think latest invalid element was running v2018.4.1. Although note that not all elements from that era are invalid (ie they don’t have a PackageVersion value), but most are invalid.

I’m happy to share an example DeploymentJournal.xml file (securely) if that would help.

Thanks,

Ant.

[Kenneth_Bates]

Hey Ant,

Thanks for following up, and that sounds like a good idea to grab that and have a look. You can email that to us at support@octopus.com, or you can also upload it while logged in on Octopus.com under your Support tab.

I appreciate your report and offering these details. I look forward to hearing back!

Best regards,

Kenny

[antmeehan (Anthony Meehan)]

Thanks. I’ve sent an email to support@octopus.com.

[Kenneth_Bates]

Thank you for sending that through! I can confirm I’ve received it, and will look more into this one. In case of an sensitive information, we can continue the discussion in the newly created email chain.

Thanks again!

Kenny