OctopusDeploy service stops unexpectedly

Lately we are experiencing that our OctopusDeploy server is stopping unexpectedly.

No information in Octopus log directory. The only thing I can find is an entry in the Windows Event Log. OctopusDeploy is running in an virtual Windows Server 2012 enviroment with 4gb ram and 2ghz dual core CPU.

This is the entry for the EventLog:

“The OctopusDeploy service terminated unexpectedly. It has done this 2 time(s).”

Please see attached screenshots.

I’m having the exact same issue. Any solutions?

Hi Thomas, aa,

We think we’ve tracked this down and have posted a new build 2.5.6 to http://octopusdeploy.com/downloads - please let me know if you have any further issues like this after upgrading.

Thanks for the detailed report; all the best!

Nick

I’m having issues even after upgrading to 2.5.6.356. See http://help.octopusdeploy.com/discussions/problems/23424-octopus-server-consuming-100-memory-and-dashboard-never-renders

Today we had another crash: This was the stracktrace from the application event log

Application: Octopus.Server.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.Reflection.TargetInvocationException
Stack:
at System.RuntimeMethodHandle.InvokeMethod(System.Object, System.Object[], System.Signature, Boolean)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(System.Object, System.Object[], System.Object[])
at System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
at Pipefish.Actor+d__0.MoveNext()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Pipefish.Actor+d__0, Pipefish, Version=2.0.2.13, Culture=neutral, PublicKeyToken=null]](d__0 ByRef)
at Pipefish.Actor.OnReceivingAsync(Pipefish.Core.Message)
at Pipefish.PersistentActor1+<OnReceivingAsync>d__0[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].MoveNext() at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Pipefish.PersistentActor1+d__0[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Pipefish, Version=2.0.2.13, Culture=neutral, PublicKeyToken=null]](d__0<System.__Canon> ByRef)
at Pipefish.PersistentActor1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].OnReceivingAsync(Pipefish.Core.Message) at Pipefish.Hosting.ActorControlBlock+<<ScheduleNext>b__6>d__7.MoveNext() at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Pipefish.Hosting.ActorControlBlock+<<ScheduleNext>b__6>d__7, Pipefish, Version=2.0.2.13, Culture=neutral, PublicKeyToken=null]](<<ScheduleNext>b__6>d__7 ByRef) at Pipefish.Hosting.ActorControlBlock.<ScheduleNext>b__6() at System.Threading.Tasks.Task1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].InnerInvoke()
at System.Threading.Tasks.Task.Execute()
at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)
at System.Threading.Tasks.Task.ExecuteEntry(Boolean)
at System.Threading.ThreadPoolWorkQueue.Dispatch()

Hi All,

Just an update that this is a very high priority for us. We have a ticket opened:


And we will update this thread when we have more information.

Thanks!
Vanessa

Just a note, that I’m seeing this on one tentacle as well (this thread appears to be for the server). I don’t know if you consider them separate issues.

“The OctopusDeploy Tentacle service terminated unexpectedly. It has done this 2 time(s).”

If you do, I’ll start another issue.

Hi Mike,

We are aware of a Tentacle crashing issue, and released the fix in 2.5.10 - which version are you currently running?
Please check the following and see if it matches your experience, if not provide as much detail as you can in another thread
GitHub issue: https://github.com/OctopusDeploy/Issues/issues/1232
original post: http://help.octopusdeploy.com/discussions/problems/25347

Thanks for the report!
Vanessa

I am currently seeing the following in 2.5.11:

Application: Tentacle.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.ObjectDisposedException
Stack:
at Pipefish.Hosting.ActivitySpace.Send(Pipefish.Core.Message)
at Octopus.Platform.Deployment.Logging.DetachedActivityLog.SendToLoggerActor(Octopus.Platform.Deployment.Logging.LoggerReference, System.Func`2<System.String,Pipefish.IMessage>)
at Octopus.Platform.Deployment.Logging.DetachedActivityLog.WriteEvent(Octopus.Platform.Diagnostics.TraceCategory, System.Exception, System.String)
at Octopus.Tentacle.Integration.Scripting.OutputBuffer.Flush()
at Octopus.Tentacle.Integration.Scripting.OutputBuffer.TimerTicked(System.Object)
at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.TimerQueueTimer.CallCallback()
at System.Threading.TimerQueueTimer.Fire()
at System.Threading.TimerQueue.FireNextTimers()

The links below link to either a private post (most likely) or one that was deleted. Either that or my browser is buggy, but it shows that I’m logged in. :wink:
Can you provide the details? I will look into upgrading, however need to do so on a schedule.

Hi Mike,

You are correct it is marked private, The error was as follows (and has been fixed in 2.5.10):

Exception Info: System.ObjectDisposedException 
Stack: 
   at System.Threading.TimerQueueTimer.Change(UInt32, UInt32) 
   at System.Threading.Timer.Change(System.TimeSpan, System.TimeSpan) 
   at Octopus.Tentacle.Integration.Scripting.OutputBuffer.TimerTicked(System.Object) 
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) 
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) 
   at System.Threading.TimerQueueTimer.CallCallback() 
   at System.Threading.TimerQueueTimer.Fire() 
   at System.Threading.TimerQueue.FireNextTimers()

What version are you running?

Vanessa

At the time I sent the message I was running 2.5.9. I know it wasn’t 2.5.10. (Looks like you just started adding the Tentacle Version to the logs? Either that or the logs cycled.)

After seeing this, I did upgrade the server instance and then the tentacle instance that was failing and will monitor. Where we had seen it fail was on a build that was scheduled to deploy once a day in the morning during the work week. It had been working for a fair amount of time and then suddenly started failing. (Ended up being 3 mornings in a row.). I did not see the error you mention below in any of the logs I could find. Having the error, I went back and looked in C:\Octopus\Logs and the only place I even see any exception is this

2014-10-21 09:06:57.0299 ERROR ileInfo) [Move-Item], IOException

Which was someone else having a lock on the file, so that error was to be expected.

I do see where I’m getting a connection failure, from a different server (which I think is because we reorganized things)

This morning after installing the update, it did seem the deploy worked, but as mentioned it was only recently the other started failing, so I’ll keep an eye on it.

Thanks!
-Mike

We’re having the same problems on version 2.6.2.845, the deploy service isn’t running on two occasions in one week since we’ve moved to the new version.

Hi rdkleine, could you contact us at support@octopusdeploy.com with your Octopus server logs? There will also usually be something in the Windows Event Viewer about the crash that you can send us too.

Paul