Issues with re-deploying release

Hello

We seemed to have hit another problem while re-deploying a release to the same environment (and same machines):
We’re running Octopus 3.0.10.2278 version with DB located outside the octopus server.

Error while executing SQL command: Violation of PRIMARY KEY constraint ‘PK_ServerTask_Id’. Cannot insert duplicate key in object ‘dbo.ServerTask’. The duplicate key value is (ServerTasks-914).
The statement has been terminated.
The command being executed was:
INSERT INTO dbo.[ServerTask] (Name, Description, QueueTime, StartTime, CompletedTime, ErrorMessage, ConcurrencyTag, State, HasPendingInterruptions, HasWarningsOrErrors, ServerNode, ProjectId, EnvironmentId, Id, Json) values (@Name, @Description, @QueueTime, @StartTime, @CompletedTime, @ErrorMessage, @ConcurrencyTag, @State, @HasPendingInterruptions, @HasWarningsOrErrors, @ServerNode, @ProjectId, @EnvironmentId, @Id, @Json)

System.Exception: Error while executing SQL command: Violation of PRIMARY KEY constraint ‘PK_ServerTask_Id’. Cannot insert duplicate key in object ‘dbo.ServerTask’. The duplicate key value is (ServerTasks-914).
The statement has been terminated.
The command being executed was:
INSERT INTO dbo.[ServerTask] (Name, Description, QueueTime, StartTime, CompletedTime, ErrorMessage, ConcurrencyTag, State, HasPendingInterruptions, HasWarningsOrErrors, ServerNode, ProjectId, EnvironmentId, Id, Json) values (@Name, @Description, @QueueTime, @StartTime, @CompletedTime, @ErrorMessage, @ConcurrencyTag, @State, @HasPendingInterruptions, @HasWarningsOrErrors, @ServerNode, @ProjectId, @EnvironmentId, @Id, @Json) —> System.Data.SqlClient.SqlException: Violation of PRIMARY KEY constraint ‘PK_ServerTask_Id’. Cannot insert duplicate key in object ‘dbo.ServerTask’. The duplicate key value is (ServerTasks-914).
The statement has been terminated.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite) at System.Data.SqlClient.SqlCommand.ExecuteNonQuery() at Octopus.Core.RelationalStorage.RelationalTransaction.Insert[TDocument](String tableName, TDocument instance, String customAssignedId) in y:\work\refs\heads\master\source\Octopus.Core\RelationalStorage\RelationalTransaction.cs:line 127 --- End of inner exception stack trace --- at Octopus.Core.RelationalStorage.RelationalTransaction.Insert[TDocument](String tableName, TDocument instance, String customAssignedId) in y:\work\refs\heads\master\source\Octopus.Core\RelationalStorage\RelationalTransaction.cs:line 133 at Octopus.Core.RelationalStorage.RelationalTransaction.Insert[TDocument](TDocument instance) in y:\work\refs\heads\master\source\Octopus.Core\RelationalStorage\RelationalTransaction.cs:line 87 at Octopus.Server.Orchestration.Deploy.DeploymentFactory.CreateDeployment(IRelationalTransaction transaction, ICommunicationsConfiguration communications, IWebPortalConfiguration webPortalConfiguration, Deployment deployment, User user, DateTimeOffset queueTime, ILifecycleProgressionEvaluator lifecycleProgressionEvaluator, Func2 promptedVariableProvider) in y:\work\refs\heads\master\source\Octopus.Server\Orchestration\Deploy\DeploymentFactory.cs:line 186
at Octopus.Server.Web.Api.Actions.DeploymentCreateAction.Execute() in y:\work\refs\heads\master\source\Octopus.Server\Web\Api\Actions\DeploymentCreateAction.cs:line 90
at Octopus.Server.Web.Infrastructure.Api.Responder1.Respond(TDescriptor options, NancyContext context) in y:\work\refs\heads\master\source\Octopus.Server\Web\Infrastructure\Api\Responder.cs:line 162 at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2) at CallSite.Target(Closure , CallSite , Object , Object , NancyContext ) at Octopus.Server.Web.Api.OctopusRestApiModule.<>c__DisplayClass5.<.ctor>b__2(Object o) in y:\work\refs\heads\master\source\Octopus.Server\Web\Api\OctopusRestApiModule.cs:line 46 at CallSite.Target(Closure , CallSite , Func2 , Object )
at Nancy.Routing.Route.<>c__DisplayClass4.b__3(Object parameters, CancellationToken context)

Can we get any feedback on this ? Is this a current version issue?

Thanks,
Andrzej

Hi Andrzej,

We are looking into the stack trace you provided. Could you please also run these two SQL commands and send us the results so we can better understand what happened leading up to this problem:

SELECT TOP 4 * FROM [dbo].[ServerTask] ORDER BY Id DESC
SELECT * FROM [dbo].[KeyAllocation]

Thank you,

Ian Mercer

Hello

Thank you for the reply, please find the output in attached file.

Thanks,
Andrzej

octo.xlsx (9 KB)

Is this error something you encountered just the once or are you seeing it regularly? We have investigated but have so far been unable to find any way that an Id value could be reused in this way and we have no other reports of it happening.

Hmm it was happening for couple of deployments and since we started testing it yesterday again it seems to have resolved itself. I’ll keep an eye on it and if i can replicate the issue I’ll update the post.

Thanks for all your help,
Andrzej

OK, thanks. I’ll also keep an eye out for anyone else reporting anything like this.

Ian

Just hit a problem similar to this with Script Console execution …

System.Exception: Error while executing SQL command: Violation of PRIMARY KEY constraint 'PK_Event_Id'. Cannot insert duplicate key in object 'dbo.Event'. The duplicate key value is (Events-944).
The statement has been terminated.
The command being executed was:
INSERT INTO dbo.[Event] (RelatedDocumentIds, ProjectId, EnvironmentId, Category, UserId, Username, Occurred, Message, Id, Json) values (@RelatedDocumentIds, @ProjectId, @EnvironmentId, @Category, @UserId, @Username, @Occurred, @Message, @Id, @Json) ---> System.Data.SqlClient.SqlException: Violation of PRIMARY KEY constraint 'PK_Event_Id'. Cannot insert duplicate key in object 'dbo.Event'. The duplicate key value is (Events-944).
The statement has been terminated.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at Octopus.Core.RelationalStorage.RelationalTransaction.Insert[TDocument](String tableName, TDocument instance, String customAssignedId) in Y:\work\refs\tags\3.2.2\source\Octopus.Core\RelationalStorage\RelationalTransaction.cs:line 127
   --- End of inner exception stack trace ---
   at Octopus.Core.RelationalStorage.RelationalTransaction.Insert[TDocument](String tableName, TDocument instance, String customAssignedId) in Y:\work\refs\tags\3.2.2\source\Octopus.Core\RelationalStorage\RelationalTransaction.cs:line 135
   at Octopus.Core.RelationalStorage.RelationalTransaction.Insert[TDocument](TDocument instance) in Y:\work\refs\tags\3.2.2\source\Octopus.Core\RelationalStorage\RelationalTransaction.cs:line 87
   at Octopus.Server.EventStore.Store[TDocument](IRelationalTransaction session, Event event, TDocument model, IOctopusPrincipal currentPrincipal) in Y:\work\refs\tags\3.2.2\source\Octopus.Server\EventStore.cs:line 69
   at Octopus.Server.Web.Infrastructure.Api.CreateResponseDescriptor`2.Responder.Execute() in Y:\work\refs\tags\3.2.2\source\Octopus.Server\Web\Infrastructure\Api\CreateResponseDescriptor.cs:line 84
   at Octopus.Server.Web.Infrastructure.Api.Responder`1.Respond(TDescriptor options, NancyContext context) in Y:\work\refs\tags\3.2.2\source\Octopus.Server\Web\Infrastructure\Api\Responder.cs:line 162
   at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2)
   at CallSite.Target(Closure , CallSite , Object , Object , NancyContext )
   at Octopus.Server.Web.Api.OctopusRestApiModule.<>c__DisplayClass0_0.<.ctor>b__0(Object o) in Y:\work\refs\tags\3.2.2\source\Octopus.Server\Web\Api\OctopusRestApiModule.cs:line 46
   at CallSite.Target(Closure , CallSite , Func`2 , Object )
   at Nancy.Routing.Route.<>c__DisplayClass4.<Wrap>b__3(Object parameters, CancellationToken context)

Key Allocation Table As Follows:

Name Value
ActionTemplate 60
Channel 43
Deployment 280
DeploymentEnvironment 40
Event 960
LibraryVariableSet 40
Lifecycle 40
Machine 40
Project 100
ProjectGroup 80
Release 260
ServerTask 5160
User 120

Events ordered by occurrence DESC:

EventID Occurred
Events-941 2016-02-18 02:30:24.5379868 +00:00
Events-960 2016-02-18 02:29:25.8401176 +00:00
Events-959 2016-02-18 02:28:13.2178561 +00:00
Events-958 2016-02-18 02:24:51.1336497 +00:00
Events-957 2016-02-18 02:08:46.1151575 +00:00
Events-956 2016-02-18 00:51:54.1878652 +00:00
Events-955 2016-02-18 00:50:53.6398110 +00:00
Events-954 2016-02-18 00:47:47.5632052 +00:00
Events-953 2016-02-18 00:42:18.1302652 +00:00
Events-952 2016-02-17 23:25:02.3547340 +00:00
Events-951 2016-02-17 23:19:49.5424559 +00:00
Events-950 2016-02-15 03:23:14.8174530 +00:00
Events-949 2016-02-08 20:21:49.0715329 +00:00
Events-948 2016-02-02 01:34:41.2789816 +00:00
Events-947 2016-01-28 06:44:18.9729962 +00:00
Events-946 2016-01-28 06:42:34.1525152 +00:00
Events-945 2016-01-28 06:41:57.2688272 +00:00
Events-944 2016-01-28 05:42:15.1866548 +00:00
Events-943 2016-01-28 05:35:49.4470847 +00:00
Events-942 2016-01-28 05:31:08.6840112 +00:00
Events-930 2016-01-28 04:18:09.4259882 +00:00
Events-929 2016-01-28 04:17:29.2639724 +00:00
Events-928 2016-01-28 04:10:42.1832684 +00:00