Unable to display details of an Azure Subscriptions Account

Legacy Modelling
1

Hi,
We can’t seem to display the details of our first created Azure Subscriptions Account (app#/infrastructure/accounts)
The display of the details ends in timeout.

Doing some SQL monitoring we found out there is a long running query “SELECT * FROM dbo.[VariableSet] WHERE ([JSON] LIKE @accid)” which happens when we try to display this page.

We also found this in the logs:

Octopus v2018.2.4

Exception occured while executing a reader for SELECT * FROM dbo.[VariableSet] WHERE ([JSON] LIKE @accid) ORDER BY [Id]
System.Exception
at Nevermore.Transient.IDbCommandExtensions.ExecuteReaderWithRetry(IDbCommand command, RetryPolicy commandRetryPolicy, RetryPolicy connectionRetryPolicy, String operationName)
at Nevermore.Transient.IDbCommandExtensions.ExecuteReaderWithRetry(IDbCommand command, String operationName)
at Nevermore.RelationalTransaction.Stream
at Octopus.Core.Persistence.Indexes.DeletionDocumentUsageFinder.RelationshipBuilder2.FindUsages(IRelationalTransaction transaction, Object from, List1 referencingDocuments)
at Octopus.Core.Persistence.Indexes.DeletionDocumentUsageFinder.FindReferences[TDocument](IRelationalTransaction transaction, TDocument document)
at Octopus.Server.Web.Api.Actions.AccountUsageListAction.ExecuteRegistered(String id)
at Octopus.Server.Web.Infrastructure.Api.Responder`1.Respond(TDescriptor options, NancyContext context)
at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2)
at Octopus.Server.Web.Infrastructure.OctopusNancyModule.<>c__DisplayClass14_0.<get_Routes>b__1(Object x)
at Nancy.Routing.Route.<>c__DisplayClass4.b__3(Object parameters, CancellationToken context)

–Inner Exception–
SQL Error -2 - Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
System.Data.SqlClient.SqlException
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TrySetMetaData(_SqlMetaDataSet metaData, Boolean moreInfo) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData() at System.Data.SqlClient.SqlDataReader.get_MetaData() at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at Nevermore.Transient.IDbCommandExtensions.<>c__DisplayClass5_0.b__0()
at Nevermore.Transient.RetryPolicy.ExecuteAction[TResult](Func`1 func)
at Nevermore.Transient.IDbCommandExtensions.ExecuteReaderWithRetry(IDbCommand command, RetryPolicy commandRetryPolicy, RetryPolicy connectionRetryPolicy, String operationName)

–Inner Exception–
The wait operation timed out
System.ComponentModel.Win32Exception

2

Doing some more check this happens for any account under the Accounts page

4

Hi Arnaud,

Thanks for getting in touch! I’m sorry to hear you’re hitting this annoying bug. It looks like you’re hitting a known issue which was fixed in version 2018.2.7, and you can reference this here.

Would you be willing to upgrade (preferably to latest) and let us know if that fixes this issue for you?

I look forward to hearing back!

Best regards,

Kenny

5

Thanks, I am now running latest version and IDD the scree is fixed.

I still have an issue with the [Usage] section of the accounts which stays “Loading…” forever.

6

Hi Arnaud,

Thanks for following up! That’s great to hear this issue is resolved for you after the upgrade, but I’m sorry you’re hitting this new issue! I suspect this usage performance behavior that you’re now hitting is the following reported issue that’s currently open.

This implies the usage section eventually loads, though the query is very slow. Does this section eventually populate in your case?

Best regards,

Kenny

7

Hi, the issue may be the same underneath but I confirm that in my case the section never displays no matter how long I wait (I am at 15 minutes now and I am fairly sure plenty of different timeouts have kicked in already)

8

Hi Arnaud,

Thanks for following up, and I’m terribly sorry you’re still hitting this terrible performance issue. Is the account you’re looking at used by a very large number of deployment processes/releases? Do you see the same behavior in any other accounts that may be less used?

I look forward to hearing back!

Kind regards,

Kenny

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.