System Integrity Check failed

gavin.burke · 28 June 2017 12:14

Hi

I have recently upgraded from 3.13.9 to 3.14.15 and since the upgrade the System Integrity Check fails, failing on the step “Unexpected related document id format”, then lists the following:

The RelatedDocumentId ‘Forecasting as a Service’ for Event Id ‘Events-20739’ is not in the expected format.
The RelatedDocumentId ‘Forecasting as a Service’ for Event Id ‘Events-20740’ is not in the expected format.
The RelatedDocumentId ‘Forecasting as a Service’ for Event Id ‘Events-20741’ is not in the expected format.
The RelatedDocumentId ‘Forecasting as a Service’ for Event Id ‘Events-20742’ is not in the expected format.
The RelatedDocumentId ‘Forecasting as a Service’ for Event Id ‘Events-22102’ is not in the expected format.
The RelatedDocumentId ‘Forecasting as a Service’ for Event Id ‘Events-22103’ is not in the expected format.
The RelatedDocumentId ‘Forecasting as a Service’ for Event Id ‘Events-22104’ is not in the expected format.
The RelatedDocumentId ‘NEILWIND’ for Event Id ‘Events-15251’ is not in the expected format.

Is there anything I can do to fix the integrity so it stops failing (as if something else breaks in the future and it’s only flagged up on this check, I’ll most likely by ignoring it by that point)

Thanks
Gavin

Guillaume · 28 June 2017 16:42

Hi,

I’m experiencing a similar issue, here’s my full log:

-Guillaume

Task ID:        ServerTasks-13651
Task status:    Failed
Task queued:    Wednesday, June 28, 2017 12:37 PM
Task started:   Wednesday, June 28, 2017 12:37 PM
Task duration:  less than a second
Server version: 3.14.15+Branch.master.Sha.f0365a084dff72242c275f7a42df01c2349be765
Server node:    IIT-VS-020

                    | Failed: Check System Integrity
12:37:14   Error    |   Unexpected related document id format....................... [ Failed ]
12:37:14   Info     |   Schema...................................................... [ Passed ]
12:37:14   Info     |   Release channel belongs to the appropriate project.......... [ Passed ]
12:37:14   Info     |   Orphaned tenanted deployments............................... [ Passed ]
12:37:14   Info     |   Orphaned channels........................................... [ Passed ]
12:37:14   Info     |   Only one unfrozen deployment process per project............ [ Passed ]
12:37:14   Fatal    |   One or more data errors were found.
                    | 
                    |   Failed: Unexpected related document id format
12:37:14   Error    |     Showing top 20 of 37 errors:
                    |     The RelatedDocumentId 'Reacts' for Event Id 'Events-17691' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts Windows' for Event Id 'Events-19570' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts Windows' for Event Id 'Events-19571' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts UI Testing' for Event Id 'Events-20039' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts UI Testing' for Event Id 'Events-20040' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts UI Testing' for Event Id 'Events-20041' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts UI Testing' for Event Id 'Events-20042' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts UI Testing' for Event Id 'Events-20044' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts UI Testing' for Event Id 'Events-20045' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts UI Testing' for Event Id 'Events-20046' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts UI Testing' for Event Id 'Events-20047' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts Services & IIS' for Event Id 'Events-21695' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts Services & IIS' for Event Id 'Events-21696' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts Services & IIS' for Event Id 'Events-21697' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts Services & IIS' for Event Id 'Events-21698' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts for Windows' for Event Id 'Events-21711' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts for Windows' for Event Id 'Events-21712' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts for Windows' for Event Id 'Events-21713' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts for Windows' for Event Id 'Events-21714' is not in the expected format.
                    |     The RelatedDocumentId 'Reacts for Windows' for Event Id 'Events-21725' is not in the expected format.

Michael_Noonan · 29 June 2017 05:49

Hi Gavin and Guillaume,

Thanks for getting in touch! It looks like you’ve run into this bug here: https://github.com/OctopusDeploy/Issues/issues/3566

You can try using the workaround described in that issue, or you can wait until we ship a fix as part of an upcoming release.

Hope that helps!
Mike

gavin.burke · 29 June 2017 08:28

Thanks Mike

I took your SQL script, turned it into a select statement first to ensure only the rows were returned per the system integrity check and I could see the RelatedID was indeed the project name and not ID

Ran your update script and now the integrity check doesn’t complain about the “Forecasting as a Service” project but still red because the NEILWIND project has been deleted - is there a safe way for me to delete the Events-15251 as it’ll be a very old task for some testing purpose?

Thanks for the swift reply!
Gavin

Michael_Noonan · 29 June 2017 08:34

Hi Gavin,

Glad it worked out for you. That Event isn’t terribly important in my opinion - it was added as a hint to describe a particular data fix. You can safely delete those events.

If you want to keep those events, you’d need to find a reference to that Project in the Events table, grab its ID and pop that in where the Name is now.

Hope that helps!
Mike

gavin.burke · 29 June 2017 08:36

Hmm actually Mike, the last task about NEILWIND isn’t to do with channels or projects but about amending the server node (NEILWIND is the name of our server)… so not sure what the RelatedDocumentId should be other than the name of the node?

SELECT * FROM EventRelatedDocument WHERE RelatedDocumentId = 'NEILWIND' returns this one row about NEILWIND, EventId as Events-15251, so then SELECT * FROM Event WHERE id like '%15251' returns RelatedDocumentIds as |NEILWIND|, NULL Project and EnvironmentId and the Message is OctopusServerNode NEILWIND was modified

Michael_Noonan · 29 June 2017 09:21

Hi Gavin,

Thanks for keeping in touch! Yes, that’s another case we found, where the ServerNode has a non-standard ID. We’ve fixed that for Octopus 3.15 which will ship early next week. In the meantime you can safely ignore that integrity check, or delete the Events and EventRelatedDocuments with that ID, provided you aren’t concerned about them.

Otherwise I’d suggest upgrading to Octopus 3.15 when it ships.

I’m sorry you’ve run into this problem, but I hope this helps!
Mike

gavin.burke · 29 June 2017 09:24

Thanks Mike

It’s now worries, I’ve removed the 1 entry in EventRelatedDocuments and Events and integrity check passes so it most definitely has helped.

Gavin