Enable AppOffline is True, Azure Web App still fails with ERROR_FILE_IN_USE

Legacy Uncategorised
1

Hi,

First, thanks for making Octopus Deploy such a great product.

I’d like to report an issue and ask for guidance:

Using Octopus 3.3.3, I have a deployment step of type “Deploy an Azure Web App”.
In the step I have the option “Enable AppOffline” set to “True”.
Too often when I deploy, the step fails with the “ERROR_FILE_IN_USE” error from WebDeploy - Stack trace below.

Might there be a bug in the implementation of the “Enable AppOffline” feature? How can I investigate this further towards a resolution?

Thanks,
@urig

Microsoft.Web.Deployment.DeploymentAgentUnavailableException: Could not complete the request to remote agent URL 'https://$[..].scm.azurewebsites.net/msdeploy.axd?site=[..]'. ---> System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.Web.Deployment.AgentClientProvider.GetHttpResponse(HttpWebRequest request)
--- End of inner exception stack trace ---
at Microsoft.Web.Deployment.AgentClientProvider.GetHttpResponse(HttpWebRequest request)
at Microsoft.Web.Deployment.AgentClientProvider.PerformHeadRequestHelper(Boolean getVersionInfo, Version& maximumSupportedVersion, Version& minimumSupportedVersion)
at Microsoft.Web.Deployment.AgentClientProvider..ctor(DeploymentProviderContext providerContext, DeploymentBaseContext baseContext, String serverVersion)
at Microsoft.Web.Deployment.DeploymentManager.CreateObjectPrivate(DeploymentProviderContext providerContext, DeploymentBaseOptions baseOptions, DeploymentObject sourceObject, String serverVersion)
at Microsoft.Web.Deployment.DeploymentManager.CreateDestinationObject(DeploymentProviderOptions providerOptions, DeploymentBaseOptions baseOptions, DeploymentObject sourceObject, String serverVersion)
at Microsoft.Web.Deployment.DeploymentObject.SyncTo(DeploymentProviderOptions providerOptions, DeploymentBaseOptions baseOptions, DeploymentSyncOptions syncOptions)
at Calamari.Azure.Deployment.Conventions.AzureWebAppConvention.Install(RunningDeployment deployment) in Y:\work\14ffc968155e4956\source\Calamari.Azure\Deployment\Conventions\AzureWebAppConvention.cs:line 22
at Calamari.Deployment.ConventionProcessor.RunInstallConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 60
at Calamari.Deployment.ConventionProcessor.RunConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 28
Running rollback conventions...
Microsoft.Web.Deployment.DeploymentAgentUnavailableException: Could not complete the request to remote agent URL 'https://$[..].scm.azurewebsites.net/msdeploy.axd?site=[..]'. ---> System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.Web.Deployment.AgentClientProvider.GetHttpResponse(HttpWebRequest request)
--- End of inner exception stack trace ---
at Microsoft.Web.Deployment.AgentClientProvider.GetHttpResponse(HttpWebRequest request)
at Microsoft.Web.Deployment.AgentClientProvider.PerformHeadRequestHelper(Boolean getVersionInfo, Version& maximumSupportedVersion, Version& minimumSupportedVersion)
at Microsoft.Web.Deployment.AgentClientProvider..ctor(DeploymentProviderContext providerContext, DeploymentBaseContext baseContext, String serverVersion)
at Microsoft.Web.Deployment.DeploymentManager.CreateObjectPrivate(DeploymentProviderContext providerContext, DeploymentBaseOptions baseOptions, DeploymentObject sourceObject, String serverVersion)
at Microsoft.Web.Deployment.DeploymentManager.CreateDestinationObject(DeploymentProviderOptions providerOptions, DeploymentBaseOptions baseOptions, DeploymentObject sourceObject, String serverVersion)
at Microsoft.Web.Deployment.DeploymentObject.SyncTo(DeploymentProviderOptions providerOptions, DeploymentBaseOptions baseOptions, DeploymentSyncOptions syncOptions)
at Calamari.Azure.Deployment.Conventions.AzureWebAppConvention.Install(RunningDeployment deployment) in Y:\work\14ffc968155e4956\source\Calamari.Azure\Deployment\Conventions\AzureWebAppConvention.cs:line 22
at Calamari.Deployment.ConventionProcessor.RunInstallConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 60
at Calamari.Deployment.ConventionProcessor.RunConventions() in Y:\work\14ffc968155e4956\source\Calamari\Deployment\ConventionProcessor.cs:line 50
at Calamari.Azure.Commands.DeployAzureWebCommand.Execute(String[] commandLineArguments) in Y:\work\14ffc968155e4956\source\Calamari.Azure\Commands\DeployAzureWebCommand.cs:line 86
at Calamari.Program.Execute(String[] args) in Y:\work\14ffc968155e4956\source\Calamari\Program.cs:line 38
The step failed: The remote script failed with exit code 100
2

Hi Uri,

Thanks for getting in touch and sharing your experience. After some digging with the team here, it looks like you’re hitting this reported bug. The short of it is that Octopus uses Microsoft web deploy (i.e. msdeploy.exe) under the hood to deploy to Azure and it appears that when the ‘Enable AppOffline’ is set that it’s automatically writing a file named App_Offline.htm whereas Azure expects a file named app_offline.htm. This mismatch (i.e. case sensitivity) appears to be causing the issue your describing. I would suggest adding an app_offline.htm file to the root of your web project to see if it resolves the issue. This file should be removed after the deployment succeeds. The following page has more information on this feature. https://www.iis.net/learn/publish/deploying-application-packages/taking-an-application-offline-before-publishing

On a related note, in Octopus 3.3.21 we improved the retry logic for Azure deployments which may help as well. Though it’s not a solution for your current issue, upgrading Octopus could potentially help if you run in to any future issues with these Azure deployments. You can download this version of a newer one from https://octopus.com/downloads.

Let me know how you go!

Rob

3

Hi Rob.

Just wanted to say thanks for the quick reply. I’ll be following your guidance in the next couple of days and will be sure to let you know how it goes.

Thanks!
urig

4

Hi guys

Is there any advice on how to do this?

I’m hitting the same problem.

Currently I’m resorting to an azure powershell script which runs Restart-AzureWebSite. While this works, using app_offline.htm should be cleaner.

I assume I’ll need to programatically add the app_offline.htm. Adding to the package I assume would be useless as we need the file there BEFORE we deploy (please correct if I’m wrong). And I can’t see an obvious method to add that file to the running website.

5

Hi everyone,

Just a quick update on this issue. David Ebbo from Microsoft has confirmed this is an issue and it was deployed to the Azure App Service infrastructure in December. He also said that it should be fixed in January 2017 (no specific date) and so the issue should go away once this happens.

That said, there are a few workarounds in the short term.

  • Add an app_offline.htm to the root of your web app as I mentioned above.
  • This GitHub issue as it describes a few additional workarounds from the Microsoft community.

Hope this helps!

Rob