Delete project timeout error under version 3.2.11

Legacy Uncategorised
1

We recently upgraded Octopus Deploy from version 2.6.5.1010 to 3.2.11 now running under SQL Server 2012 Express in the back end. We started experiencing the problem when we try delete the project. I also tried first deleting all the steps within that project and it all works fine. Its still timing out when I request to delete an empty project. Getting following error every time,

Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. —> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TrySetMetaData(_SqlMetaDataSet metaData, Boolean moreInfo) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData() at System.Data.SqlClient.SqlDataReader.get_MetaData() at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader()
at Octopus.Core.RelationalStorage.TransientFaultHandling.IDbCommandExtensions.<>c__DisplayClass5_0.b__0() in Y:\work\refs\tags\3.2.11\source\Octopus.Core\RelationalStorage\TransientFaultHandling\IDbCommandExtensions.cs:line 67
at Octopus.Shared.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func1 func) in Y:\work\refs\tags\3.2.11\source\Octopus.Shared\TransientFaultHandling\RetryPolicy.cs:line 215 at Octopus.Core.RelationalStorage.RelationalTransaction.<Stream>d__241.MoveNext() in Y:\work\refs\tags\3.2.11\source\Octopus.Core\RelationalStorage\RelationalTransaction.cs:line 241
at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext() at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext()
at Octopus.Core.Persistence.Indexes.DocumentUsageFinder.RelationshipBuilder2.FindUsages(IRelationalTransaction transaction, Object from, List1 referencingDocuments) in Y:\work\refs\tags\3.2.11\source\Octopus.Core\Persistence\Indexes\DocumentUsageFinder.cs:line 314
at Octopus.Core.Persistence.Indexes.DocumentUsageFinder.FindReferences[TDocument](IRelationalTransaction transaction, TDocument document) in Y:\work\refs\tags\3.2.11\source\Octopus.Core\Persistence\Indexes\DocumentUsageFinder.cs:line 269
at Octopus.Core.Persistence.Indexes.DeleteCommand.DeleteRecursive(IRelationalTransaction transaction, IDocument document, ReferenceCollection seenBefore) in Y:\work\refs\tags\3.2.11\source\Octopus.Core\Persistence\Indexes\DeleteCommand.cs:line 51
at Octopus.Core.Persistence.Indexes.DeleteCommand.DeleteRecursive(IRelationalTransaction transaction, IDocument document, ReferenceCollection seenBefore) in Y:\work\refs\tags\3.2.11\source\Octopus.Core\Persistence\Indexes\DeleteCommand.cs:line 66
at Octopus.Server.Web.Infrastructure.Api.DeleteOnBackgroundResponseDescriptor2.Responder.Execute() in Y:\work\refs\tags\3.2.11\source\Octopus.Server\Web\Infrastructure\Api\DeleteOnBackgroundResponseDescriptor.cs:line 78 at Octopus.Server.Web.Infrastructure.Api.Responder1.Respond(TDescriptor options, NancyContext context) in Y:\work\refs\tags\3.2.11\source\Octopus.Server\Web\Infrastructure\Api\Responder.cs:line 162
at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2)
at CallSite.Target(Closure , CallSite , Object , Object , NancyContext )
at Octopus.Server.Web.Api.OctopusRestApiModule.<>c__DisplayClass0_0.<.ctor>b__0(Object o) in Y:\work\refs\tags\3.2.11\source\Octopus.Server\Web\Api\OctopusRestApiModule.cs:line 46
at CallSite.Target(Closure , CallSite , Func`2 , Object )
at Nancy.Routing.Route.<>c__DisplayClass4.b__3(Object parameters, CancellationToken context)
ClientConnectionId:d87faa50-1508-4a4f-b81d-b10493e9d279
Error Number:-2,State:0,Class:11

2

Hi,

Sorry to hear you are having problems deleting projects.

We are currently working on a fix for an issue that may be related, you can track it here: https://github.com/OctopusDeploy/Issues/issues/2214

Do you know if you have quite a large Octopus database (over a few hundred meg) or a lot of deployments?

Cheers,
Shane

3

Hello Shane,

Thanks and it’s good to know that the fix is underway. Yes, we do have quite a complex Octopus Deploy automation setup and have a comparatively large SQL DB with size of about 6400 MB. I wonder if Octopus provides an ability to truncate old data based on the specified time. Please let me know when the solution is available and I will work with you to implement it.

Thank You.

Nilesh

4

Hi there,
Just adding my 2 cents:
As I am evaluating the upgrade for our organization – I have recently created a new fresh instance of Octopus-3.2.13 and imported all of the data from our 2.6.4.951 – so they are running side by side and no one is really using the new one yet.

The idea is to compare the 2 and make sure everything looks good and then perform a final export/import to move the delta to the new version, run hydra and leave the old server in ‘read-only” mode to perform the switch.

So while testing I’ve encountered this thread (and others) and I am verifying these issues on our environment.
So I performed a small test – doing the same thing on the 2 servers:

  1.   Created a new empty test project

  2.   Delete it.

On the old version (running against raven) – it works pretty fast (about a second), whereas on the new one (SQL) it takes ~6 seconds.
In general things look much nicer in version 3.x, but I do feel like things are slightly heavier and taking longer to load on the UI.

I haven’t experienced timeouts though…

I hope this helps, and looking forward to hear updates about this one

Thanks
Nir

5

Hi Nilesh and Nir,

We have just released version 3.2.16 which includes some performance enhancements when deleting. There should be a very noticeable decrease in the time taken when deleting and the timeout should not happen.

I hope this helps, please let me know if you encounter and issues.

Cheers,
Shane

6

Thank you Shane. We will need to work on an upgrade plan and will upgrade to the latest release available at that time.

7

Hi Shane, I would like to give you an update that this issue has been resolved once we have upgraded Octopus to version 3.2.19 (latest available as of today). I can proudly say today that we are on the latest version which may not be true tomorrow, all depends on you!.

Anyway, thanks a lot for quickly addressing this issue and really appreciated all your prompt responses. I will now go ahead and close this issue.