Every Backup is Failing saying There is not enough space on the disk.(But we have 200GB of space Available)

manoj_singi · 20 May 2016 17:18

Hi Team,

We are facing issues with the backup’s failing, Our Last Successful back up was from 14th May 2016.
From then we dont even have a good Successful backup.
We have 200 GB of Free Space on the Drive and Backup runs for 4 hours and throws an error.

There is not enough space on the disk.

System.IO.IOException: There is not enough space on the disk.

Server stack trace:
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.WriteCore(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.IsolatedStorage.IsolatedStorageFileStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at MS.Internal.IO.Packaging.SparseMemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at MS.Internal.IO.Zip.ZipIOFileItemStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at MS.Internal.IO.Zip.ProgressiveCrcCalculatingStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at MS.Internal.IO.Zip.ZipIOModeEnforcingStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at Octopus.Server.Storage.ImportExport.Exporter.PleaseDoNotDeadlockDueToBugInFrameworkStreamWrapper.Write(Byte[] buffer, Int32 offset, Int32 count) in y:\work\refs\heads\master\source\Octopus.Server\Storage\ImportExport\Exporter.cs:line 179
at System.Security.Cryptography.CryptoStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Compression.DeflateStream.PurgeBuffers(Boolean disposing)
at System.IO.Compression.DeflateStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.Compression.GZipStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.StreamWriter.Dispose(Boolean disposing)
at System.IO.TextWriter.Dispose()
at Raven.Abstractions.Smuggler.SmugglerApiBase.d__1.MoveNext() in c:\Builds\RavenDB-Stable\Raven.Abstractions\Smuggler\SmugglerApiBase.cs:line 169

Exception rethrown at [0]:
at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at Octopus.Server.Orchestration.Backup.BackupOrchestrator.d__0.MoveNext() in y:\work\refs\heads\master\source\Octopus.Server\Orchestration\Backup\BackupOrchestrator.cs:line 51
Octopus.Server version 2.6.5.1010

Thanks,
Manojreddy.

Dalmiro · 20 May 2016 18:40

Hi Mannojreddy,

Thanks for reaching out. The likely cause for this is the task failed but it is still in the RavenDB index.

You can reset the RavenDB indexes by following the instructions on the doc below.

Let me know if that helps,
Dalmiro

manoj_singi · 20 May 2016 18:43

Hi Dalmiro,

Last time we did do this repair and it messed up everything.

our database size was 43Gb when it was successful.

If you have time now, can we have a quick call as it is very urgent for us and we have to migrate to 3.x with Active Active.

Thanks,
Manojreddy.

Vanessa_Love · 23 May 2016 04:30

Hi Manoj,

What document size did you get your Raven DB down to? You are not going to be able to migrate a 43GB database.
We have a way to build the backup but it depends on the following. Both the UI backup and the other second method all rely on the Raven Export feature.

If you browse the Raven DB via the Octopus Manager then go to Tasks -> Export and Export Database. Does this run to completion or are there any errors?

Vanessa

manoj_singi · 23 May 2016 12:52

Hi Vanessa,

we have Deleted all the Events and Documents cam down from 134,000 to 65,000.
But Unfortunately when we restarted the service after deletion, there was No Re-index and the size was same. We restarted the service several times but no use.

we tried taking the backup and restored in another environment and then it cam down to 3gb, but our backup’s are failing since 1 week and we did not take any risk to repair the Production Database.

We Tried exporting Database from Raven Db browser but no use and throws an error after 33,000 documents, It does not complete the backup.

Thanks,
Manojreddy.

Vanessa_Love · 23 May 2016 22:03

Hi Manoj,

What is the error?

Vanessa

manoj_singi · 23 May 2016 23:28

Hi Venessa,

I have Attached the Screen Shots of the Errors we get when we do Export Database and Backup Database from RavenDb Browser.

Thanks,
Manojreddy.

manoj_singi · 25 May 2016 13:01

Hi Venessa,

Issue has been solved without Repairing the RavenDB.
The issue was when it is taking backup to D:Drive initially the temp files are saving on C:Drive and we dont have enough space on C:Drive, But we do have lot of space on D:Drive.
so, we have increased the space on C:Drive and the backups are successful.

Thanks,
Manojreddy.

Vanessa_Love · 27 May 2016 01:01

Hi Manojreddy,

Great! I am interested to hear how your upgrade goes.

Vanessa