Getting system.io exception when trying to upload the packages

[dushan_perera]

Hi

We are getting the below error when we try to upload the package and after clicking upload button for few attempts, it will upload.

Found this error from the octopus logs

2021-06-24 15:50:35.0630 9736 43 ERROR Read
System.IO.IOException
—> Microsoft.AspNetCore.Server.HttpSys.HttpSysException (995): The I/O operation has been aborted because of either a thread exit or an application request.
— End of inner exception stack trace —
2021-06-24 15:50:35.0630 9736 43 ERROR Unhandled error on request: POST https://octopusx.xxx.com.au/api/Spaces-1/packages/raw?replace=true 8000016f-0002-f400-b63f-84710c7967bb by XXXXXX :
System.IO.IOException
—> Microsoft.AspNetCore.Server.HttpSys.HttpSysException (995): The I/O operation has been aborted because of either a thread exit or an application request.
— End of inner exception stack trace —
at Microsoft.AspNetCore.Server.HttpSys.RequestStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.Stream.CopyTo(Stream destination, Int32 bufferSize)
at Nancy.IO.RequestStream.MoveStreamContentsToFileStream()
at Nancy.IO.RequestStream.MoveStreamOutOfMemoryIfExpectedLengthExceedSwitchLength(Int64 expectedLength)
at Nancy.IO.RequestStream…ctor(Stream stream, Int64 expectedLength, Int64 thresholdLength, Boolean disableStreamSwitching)
at Octopus.NancyOwin.NancyMiddleware.<>c__DisplayClass3_0.<b__1>d.MoveNext() in C:\BuildAgent\work\c42ca9d2b8233f1c\source\Octopus.Server\Owin\NancyMiddleware.cs:line 108
— End of stack trace from previous location where exception was thrown —
at Microsoft.AspNetCore.Owin.WebSocketAcceptAdapter.<>c__DisplayClass6_0.<b__0>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.HttpOverrides.HttpMethodOverrideMiddleware.Invoke(HttpContext context)
at Octopus.Server.Web.UnitOfWorkMiddleware.InvokeAsync(HttpContext httpContext, Lazy1 unitOfWork, ILog log, IActionContextAccessor actionContextAccessor) in C:\BuildAgent\work\c42ca9d2b8233f1c\source\Octopus.Server\Web\UnitOfWorkMiddleware.cs:line 42 at Octopus.Server.Web.UnitOfWorkMiddleware.InvokeAsync(HttpContext httpContext, Lazy1 unitOfWork, ILog log, IActionContextAccessor actionContextAccessor) in C:\BuildAgent\work\c42ca9d2b8233f1c\source\Octopus.Server\Web\UnitOfWorkMiddleware.cs:line 42
at Octopus.Server.Web.Middleware.OctopusClientOldVersionWarningMiddleware.InvokeAsync(HttpContext context, IAutomationContext automationContext) in C:\BuildAgent\work\c42ca9d2b8233f1c\source\Octopus.Server\Web\Middleware\OctopusClientOldVersionWarningMiddleware.cs:line 38
at Octopus.Server.Web.Middleware.RequestLoggerMiddleware.InvokeAsync(HttpContext context) in C:\BuildAgent\work\c42ca9d2b8233f1c\source\Octopus.Server\Web\Middleware\RequestLoggerMiddleware.cs:line 42
at Octopus.Server.Web.Middleware.ErrorHandlingMiddleware.InvokeAsync(HttpContext context, CorrelationId correlationId) in C:\BuildAgent\work\c42ca9d2b8233f1c\source\Octopus.Server\Web\Middleware\ErrorHandlingMiddleware.cs:line 82
2021-06-24 15:50:35.0858 9736 43 ERROR Could not send the exception response since the http headers have already been sent to the client.
2021-06-24 15:50:35.0973 9736 43 ERROR ProcessRequestAsync
System.InvalidOperationException: Headers already sent.
at Microsoft.AspNetCore.Server.HttpSys.Response.CheckResponseStarted()
at Microsoft.AspNetCore.Server.HttpSys.FeatureContext.ConsiderEnablingResponseCache()
at Microsoft.AspNetCore.Server.HttpSys.FeatureContext.OnResponseStart()
at Microsoft.AspNetCore.Server.HttpSys.FeatureContext.CompleteAsync()
at Microsoft.AspNetCore.Server.HttpSys.MessagePump.ProcessRequestAsync(Object requestContextObj)
at Microsoft.AspNetCore.Server.HttpSys.MessagePump.ProcessRequestAsync(Object requestContextObj)
2021-06-24 15:50:42.1836 9736 42 INFO Request took 2606ms: POST https://octopusx.xxx.com.au/api/Spaces-1/packages/raw?replace=true 80000101-0006-f700-b63f-84710c7967bb by XXXX

Any idea why this is happening?

[dane.falvo]

Hi Dushan,

I’m sorry that you are running into this issue. Luckily the error has pointed us to the line of code that we need to start looking at.

Can I get a little more information about when you see the issue? As it’s a connection issue, I want to focus on timings.

When you get this error, is it immediately after clicking on upload, or is it after a couple of seconds/minutes?
Did you have the “Upload a Package” window open for a few minutes before attempting to upload the package?
Is the package your trying to upload, a larger package than you usually try to upload?

How often does this happen? Every time you upload a package? Is this the only package you’ve seen the issue happen with?

Have you checked for network connection issues? i.e. a simple continuous ping test to see if you are dropping packets etc?

I hope to hear from you soon, so we can get to the bottom of this.

Regards,

Dane