I have an ASP.NET Web API application running on an Azure App Service which is, in turn, deployed via Octopus after a TFS build definition is completed.
After certain successful deployments (no errors happen nor in TFS nor in Octopus), the whole API stops working and all endpoints respond with the following error:
System.BadImageFormatException Index not found. (Exception from HRESULT: 0x80131124)
and the stack-trace looks like this:
System.BadImageFormatException: Index not found. (Exception from HRESULT: 0x80131124) at System.Web.Mvc.Controller.BeginExecute(RequestContext requestContext, AsyncCallback callback, Object state) at System.Web.Mvc.Controller.System.Web.Mvc.Async.IAsyncController.BeginExecute(RequestContext requestContext, AsyncCallback callback, Object state) at System.Web.Mvc.MvcHandler.<BeginProcessRequest>b__4(AsyncCallback asyncCallback, Object asyncState, ProcessRequestState innerState) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallBeginDelegate(AsyncCallback callback, Object callbackState) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.Begin(AsyncCallback callback, Object state, Int32 timeout) at System.Web.Mvc.Async.AsyncResultWrapper.Begin[TState](AsyncCallback callback, Object callbackState, BeginInvokeDelegate`1 beginDelegate, EndInvokeVoidDelegate`1 endDelegate, TState invokeState, Object tag, Int32 timeout, SynchronizationContext callbackSyncContext) at System.Web.Mvc.MvcHandler.BeginProcessRequest(HttpContextBase httpContext, AsyncCallback callback, Object state) at System.Web.Mvc.MvcHandler.BeginProcessRequest(HttpContext httpContext, AsyncCallback callback, Object state) at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.BeginProcessRequest(HttpContext context, AsyncCallback cb, Object extraData) at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step) at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
The error is a bit mysterious because it doesn’t seem to related to any particular set of changes in the source code or to any particular action. That’s why it looks random. At the moment, the only way I found to temporarily solve this is re-deploying the same Octopus’ package. That restores the API.
My suspicion is that something happens between Octopus and the Azure Web App.
What I also found is that after the first deployment in Octopus (the one that produces the failure), I saw the following line in the logs:
17:16:00 Info | Successfully deployed to Azure. 0 objects added. 48 objects updated. 0 objects deleted.
Now, after the second deploy in Octopus (the re-deploy of the release that failed), one would expect to have “0 objects updated”, but instead I got the following line:
18:48:57 Info | Successfully deployed to Azure. 0 objects added. 3 objects updated. 0 objects deleted.
Some additional information:
Octopus version is v2018.6.2
TFS version is 15.117.27414.0
ASP.NET app targets .NET 4.5.2
What could be possibly going wrong?