IIS AppPool - Create template fails

When running the IIS AppPool - Create template (coming from the step template libarary) is used, we get the following error. This used to work until recently, we aren’t sure what changed exactly:
Exception calling “LoadFrom” with “1” argument(s): "Could not load file or assembly
15:00:10Error
’file:///C:\windows\system32\inetsrv\Microsoft.Web.Administration.dll’ or one of its dependencies. The system cannot
15:00:10Error
find the file specified."
15:00:10Error
At C:\Octopus\Work\20160412130008-30\Script.ps1:95 char:1
15:00:10Error

  • [System.Reflection.Assembly]::LoadFrom( "C:\windows\system32\inetsrv\Microsoft.W …
    15:00:10Error

15:00:10Error
+ CategoryInfo : NotSpecified: (:slight_smile: [], ParentContainsErrorRecordException
15:00:10Error
+ FullyQualifiedErrorId : FileNotFoundException
15:00:10Error
15:00:11Fatal
The remote script failed with exit code 1

Hi Ryan,

Thanks for getting in touch!

I’ve tested this step template on one of my instances and it is working as expected. Looking at the error message it seems that the Microsoft.Web.Administration.dll file no longer exist in C:\Windows\system32\inetsrv…not sure why that would have been removed…

Could you logon to the Tentacle and run the below in a PowerShell window:

[System.Reflection.Assembly]::LoadFrom( "C:\windows\system32\inetsrv\Microsoft.Web.Administration.dll" )

That should at least tell us if the dll is still there.

Thank you and best regards,
Henrik

Running it from Powershell works fine, though it redirects to the GAC version:

GAC    Version        Location
---    -------        --------
True   v2.0.50727     C:\Windows\assembly\GAC_MSIL\Microsoft.Web.Administration\7.0.0.0__31bf3856ad364e35\Microsoft....

Hi Ryan,

Sorry to be a pain, but could I get you to try and run that line of PowerShell in the script console as well…I’m really not sure why you would be getting that error as it should work as the dll is there…

Thank you and warm regards,
Henrik

No problem!

Note the different PowerShell versions below.

Here is the script console output (which works)

Task ID:        ServerTasks-14473
Task status:    Success
Task queued:    Thursday, April 14, 2016 12:15 PM
Task started:   Thursday, April 14, 2016 12:15 PM
Task duration:  3 seconds
Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
                    | == Success: Script run from management console ==
                    | 
                    |   == Success: Run script on: <servername>==
12:15:17   Verbose  |     Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
12:15:18   Verbose  |     Name                           Value
12:15:18   Verbose  |     ----                           -----
12:15:18   Verbose  |     PSVersion                      4.0
12:15:18   Verbose  |     WSManStackVersion              3.0
12:15:18   Verbose  |     SerializationVersion           1.1.0.1
12:15:18   Verbose  |     CLRVersion                     4.0.30319.42000
12:15:18   Verbose  |     BuildVersion                   6.3.9600.17400
12:15:18   Verbose  |     PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
12:15:18   Verbose  |     PSRemotingProtocolVersion      2.2
12:15:18   Info     |     GAC    Version        Location
12:15:18   Info     |     ---    -------        --------
12:15:18   Info     |     True   v2.0.50727     C:\Windows\assembly\GAC_MSIL\Microsoft.Web.Administrat...
12:15:19   Info     |     Exit code: 0
                    |   

And here is the normal (failing) task output:

Task ID:        ServerTasks-14423
Task status:    Failed
Task queued:    Wednesday, April 13, 2016 9:42 AM
Task started:   Wednesday, April 13, 2016 9:42 AM
Task duration:  7 seconds
Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
                    | == Failed: Deploy <packageName> release 224.2016.412.27818 to QAT ==
09:42:00   Verbose  |   Step 1: eMail Users does not apply to the current environment, and will not be executed
09:42:00   Verbose  |   Step 2: Wait for scheduled deployment date does not apply to the current environment, and will not be executed
09:42:00   Verbose  |   Guided failure is not enabled for this task
09:42:08   Fatal    |   The deployment failed because one or more steps failed. Please see the deployment log for details.
                    | 
                    |   == Success: Acquire packages ==
09:42:00   Info     |     Acquiring packages
09:42:00   Info     |     Making a list of packages to download
09:42:00   Verbose  |     Package DatabaseDeploy version 224.2016.412.27817 does not need to be downloaded on any machines
09:42:00   Info     |     Looking up the package location from the built-in package repository...
09:42:00   Verbose  |     SHA1 hash of package <packageName> is: 3fb6c58e7e5759aaffd308a5ea1417b8b8e6540b
09:42:04   Info     |     All packages have been acquired
09:42:04   Verbose  |     Acquire Packages completed
                    |   
                    |     Success: <servername>
                    |     
                    |       Success: Upload package <packageName> version 224.2016.412.27817
09:42:02   Verbose  |         Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
09:42:02   Verbose  |         Name                           Value
09:42:02   Verbose  |         ----                           -----
09:42:02   Verbose  |         PSVersion                      4.0
09:42:02   Verbose  |         WSManStackVersion              3.0
09:42:02   Verbose  |         SerializationVersion           1.1.0.1
09:42:02   Verbose  |         CLRVersion                     4.0.30319.42000
09:42:02   Verbose  |         BuildVersion                   6.3.9600.17400
09:42:02   Verbose  |         PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
09:42:02   Verbose  |         PSRemotingProtocolVersion      2.2
09:42:03   Verbose  |         Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
09:42:03   Verbose  |         Package <packageName> 224.2016.412.27817 hash 3fb6c58e7e5759aaffd308a5ea1417b8b8e6540b has already been uploaded
09:42:03   Info     |         Package <packageName>.224.2016.412.27817 (18.20 MB) found in cache. No need to upload. Using E:\Octopus\Files\<packageName>.224.2016.412.27817.nupkg-9a675e33-2c23-4f4e-8e74-f3cba5a0c7c5
                    |       
                    |   == Success: Step 3: Deploy Database Changes ==
09:42:04   Verbose  |     Deploy Database Changes completed
                    |   
                    |   == Failed: Step 4: IIS AppPool - Create ==
09:42:07   Fatal    |     The step failed: One or more child activities failed.
09:42:07   Verbose  |     IIS AppPool - Create completed
                    |   
                    |     Failed: <servername>
09:42:04   Verbose  |       Octopus Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
09:42:05   Verbose  |       Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
09:42:06   Verbose  |       Name                           Value
09:42:06   Verbose  |       ----                           -----
09:42:06   Verbose  |       PSVersion                      3.0
09:42:06   Verbose  |       WSManStackVersion              3.0
09:42:06   Verbose  |       SerializationVersion           1.1.0.1
09:42:06   Verbose  |       CLRVersion                     4.0.30319.42000
09:42:06   Verbose  |       BuildVersion                   6.2.9200.17065
09:42:06   Verbose  |       PSCompatibleVersions           {1.0, 2.0, 3.0}
09:42:06   Verbose  |       PSRemotingProtocolVersion      2.2
09:42:07   Error    |       Exception calling "LoadFrom" with "1" argument(s): "Could not load file or assembly
09:42:07   Error    |       'file:///C:\windows\system32\inetsrv\Microsoft.Web.Administration.dll' or one of its dependencies. The system cannot
09:42:07   Error    |       find the file specified."
09:42:07   Error    |       At C:\Octopus\Work\20160413074204-38\Script.ps1:95 char:1
09:42:07   Error    |       + [System.Reflection.Assembly]::LoadFrom( "C:\windows\system32\inetsrv\Microsoft.W ...
09:42:07   Error    |       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
09:42:07   Error    |       + CategoryInfo          : NotSpecified: (:) [], ParentContainsErrorRecordException
09:42:07   Error    |       + FullyQualifiedErrorId : FileNotFoundException
09:42:07   Fatal    |       The remote script failed with exit code 1
                    |     
                    |   Canceled: Step 5: IIS Application - Create
09:42:07   Verbose  |     Step "IIS Application - Create" runs only when all previous steps succeeded; skipping
                    |   
                    |   Canceled: Step 6: Deploy Website
09:42:07   Verbose  |     Step "Deploy Website" runs only when all previous steps succeeded; skipping
                    |   

Hi Ryan,

That is very strange indeed as I’m pretty sure we invoke the scripts all the same in Calamari. I’ve put it to the team to see if we can figure out why the step template is invoked using PS3.0 instead of PS4.0.

In the meantime, could I get you to test one more thing, if you comment out the [System.Reflection.Assembly]::LoadFrom statement in the step template (it shouldn’t actually be needed, we load the Web Administration module in our IIS feature script and we don’t load the specific dll) and see if that works.

Sorry for the inconvenience caused by this, but I hope we can get it all sorted soon.

Thank you and best regards,
Henrik

This also fails:

 Failed: <servername>
08:34:17   Verbose  |       Octopus Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
08:34:24   Verbose  |       Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
08:34:27   Verbose  |       Name                           Value
08:34:27   Verbose  |       ----                           -----
08:34:27   Verbose  |       PSVersion                      3.0
08:34:27   Verbose  |       WSManStackVersion              3.0
08:34:27   Verbose  |       SerializationVersion           1.1.0.1
08:34:27   Verbose  |       CLRVersion                     4.0.30319.42000
08:34:27   Verbose  |       BuildVersion                   6.2.9200.17065
08:34:27   Verbose  |       PSCompatibleVersions           {1.0, 2.0, 3.0}
08:34:27   Verbose  |       PSRemotingProtocolVersion      2.2
08:34:28   Error    |       New-Object : Cannot find type [Microsoft.Web.Administration.ServerManager]: make sure the assembly containing this
08:34:28   Error    |       type is loaded.
08:34:28   Error    |       At C:\Octopus\Work\20160418063417-40\Script.ps1:105 char:9
08:34:28   Error    |       + $iis = (New-Object Microsoft.Web.Administration.ServerManager)
08:34:28   Error    |       +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
08:34:28   Error    |       + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
08:34:28   Error    |       + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand
08:34:28   Fatal    |       The remote script failed with exit code 1

Hi Ryan,

We’re scratching our heads here as to why this would be happening, so I think it’d be best if we scheduled a support call to see if we can figure out what is going on here, you can schedule a time that suits you here.

If you could also try another thing for me, create a new step template (could just be Write-Host "Hello World") and run that through that same Tentacle and see if that also runs under PowerShell 3.0 instead of 4.0.

Thank you and warm regards,
Henrik

I have scheduled the call for tomorrow.

Here is the output of a Hello World step:

Task ID:        ServerTasks-14710
Task status:    Success
Task queued:    Thursday, April 21, 2016 8:44 AM
Task started:   Thursday, April 21, 2016 8:44 AM
Task duration:  33 seconds
Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
                    | == Success: Deploy <package> release 225.2016.420.28013 to QAT ==
08:44:58   Info     |   Step 3: Deploy <package> Database Changes was manually skipped, and will not be executed
08:44:58   Info     |   Step 4: IIS AppPool - Create was manually skipped, and will not be executed
08:44:58   Info     |   Step 5: IIS Application - Create was manually skipped, and will not be executed
08:44:58   Info     |   Step 6: Deploy Website was manually skipped, and will not be executed
08:44:58   Verbose  |   Step 1: eMail Users does not apply to the current environment, and will not be executed
08:44:58   Verbose  |   Step 2: Wait for scheduled deployment date does not apply to the current environment, and will not be executed
08:44:58   Verbose  |   Guided failure is not enabled for this task
08:45:31   Info     |   The deployment completed successfully.
                    | 
                    |   == Success: Step 7: Debug Step ==
08:45:31   Verbose  |     Debug Step completed
                    |   
                    |     Success: <servername>
08:44:58   Verbose  |       Octopus Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
08:44:58   Info     |       Executing script on '<servername>'
08:45:29   Verbose  |       Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
08:45:30   Verbose  |       Name                           Value
08:45:30   Verbose  |       ----                           -----
08:45:30   Verbose  |       PSVersion                      4.0
08:45:30   Verbose  |       WSManStackVersion              3.0
08:45:30   Verbose  |       SerializationVersion           1.1.0.1
08:45:30   Verbose  |       CLRVersion                     4.0.30319.42000
08:45:30   Verbose  |       BuildVersion                   6.3.9600.17400
08:45:30   Verbose  |       PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
08:45:30   Verbose  |       PSRemotingProtocolVersion      2.2
08:45:30   Info     |       Hello World
                    |     
                    |     Success: <servername2>
08:44:58   Verbose  |       Octopus Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
08:44:58   Info     |       Executing script on '<servername2>'
08:45:29   Verbose  |       Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
08:45:30   Verbose  |       Name                           Value
08:45:30   Verbose  |       ----                           -----
08:45:30   Verbose  |       PSVersion                      4.0
08:45:30   Verbose  |       WSManStackVersion              3.0
08:45:30   Verbose  |       SerializationVersion           1.1.0.1
08:45:30   Verbose  |       CLRVersion                     4.0.30319.42000
08:45:30   Verbose  |       BuildVersion                   6.3.9600.17400
08:45:30   Verbose  |       PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
08:45:30   Verbose  |       PSRemotingProtocolVersion      2.2
08:45:31   Info     |       Hello World

Hi Ryan,

Thanks for testing that for me, at least it means that we don’t have a bug in how we run step templates, but it doesn’t explain why the other step template isn’t working.

To make sure there isn’t anything wrong with the step template itself, could you setup a new project with just that step template in it, and run that on the Tentacle where it fails, and configure a temporary (dummy) IIS app pool? I’m wondering if something with the step might have become corrupted and is causing it to run under PS3.0 instead of 4.0 as all the other steps do.

I’ll send through the meeting details closer to the scheduled time.

I’ll speak to you then, thank you and best regards,
Henrik

I created a new project, with only the Create IIS Pool step. Here is the result:

Task ID:        ServerTasks-14749
Task status:    Failed
Task queued:    Friday, April 22, 2016 8:23 AM
Task started:   Friday, April 22, 2016 8:23 AM
Task duration:  3 seconds
Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
                    | == Failed: Deploy Dummy release 0.0.3 to QAT ==
08:23:17   Verbose  |   Guided failure is not enabled for this task
08:23:20   Fatal    |   The deployment failed because one or more steps failed. Please see the deployment log for details.
                    | 
                    |   == Failed: Step 1: IIS AppPool - Create ==
08:23:19   Fatal    |     The step failed: One or more child activities failed.
08:23:19   Verbose  |     IIS AppPool - Create completed
                    |   
                    |     Failed: servername
08:23:17   Verbose  |       Octopus Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
08:23:17   Info     |       Executing script on 'servername'
08:23:18   Verbose  |       Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
08:23:19   Verbose  |       Name                           Value
08:23:19   Verbose  |       ----                           -----
08:23:19   Verbose  |       PSVersion                      4.0
08:23:19   Verbose  |       WSManStackVersion              3.0
08:23:19   Verbose  |       SerializationVersion           1.1.0.1
08:23:19   Verbose  |       CLRVersion                     4.0.30319.42000
08:23:19   Verbose  |       BuildVersion                   6.3.9600.17400
08:23:19   Verbose  |       PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
08:23:19   Verbose  |       PSRemotingProtocolVersion      2.2
08:23:19   Error    |       New-Object : Cannot find type [Microsoft.Web.Administration.ServerManager]:
08:23:19   Error    |       verify that the assembly containing this type is loaded.
08:23:19   Error    |       At E:\Octopus\Work\20160422062316-1166\Script.ps1:105 char:9
08:23:19   Error    |       + $iis = (New-Object Microsoft.Web.Administration.ServerManager)
08:23:19   Error    |       +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
08:23:19   Error    |       + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentExcepti
08:23:19   Error    |       on
08:23:19   Error    |       + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewOb
08:23:19   Error    |       jectCommand
08:23:19   Fatal    |       The remote script failed with exit code 1
                    |     
                    |     Failed: servername2
08:23:17   Verbose  |       Octopus Server version: 3.3.6+Branch.master.Sha.ef711d651c56c0d0097bfecf797bb02a0abb1d00
08:23:17   Info     |       Executing script on 'servername2'
08:23:17   Verbose  |       Octopus Deploy: Calamari version 3.3.5+Branch.master.Sha.755501eda21d63861022f1a2d87c98d5e7f9f23b
08:23:18   Verbose  |       Name                           Value
08:23:18   Verbose  |       ----                           -----
08:23:18   Verbose  |       PSVersion                      4.0
08:23:18   Verbose  |       WSManStackVersion              3.0
08:23:18   Verbose  |       SerializationVersion           1.1.0.1
08:23:18   Verbose  |       CLRVersion                     4.0.30319.42000
08:23:18   Verbose  |       BuildVersion                   6.3.9600.17400
08:23:18   Verbose  |       PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
08:23:18   Verbose  |       PSRemotingProtocolVersion      2.2
08:23:19   Error    |       New-Object : Cannot find type [Microsoft.Web.Administration.ServerManager]:
08:23:19   Error    |       verify that the assembly containing this type is loaded.
08:23:19   Error    |       At E:\Octopus\Work\20160422062317-1749\Script.ps1:105 char:9
08:23:19   Error    |       + $iis = (New-Object Microsoft.Web.Administration.ServerManager)
08:23:19   Error    |       +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
08:23:19   Error    |       + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentExcepti
08:23:19   Error    |       on
08:23:19   Error    |       + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewOb
08:23:19   Error    |       jectCommand
08:23:19   Fatal    |       The remote script failed with exit code 1
                    |     


Hi Ryan,

Thanks, did you uncomment the line beginning with [System.Reflection.Assembly]::LoadFrom that previously asked you to comment out.

It looks like the script now at least runs under PS4.0, so we might be on to something…

Thank you,
Henrik

Yes, that line is still commented out.

OK, can you uncomment that line and try again please.

Thank you,
Henrik

That succeeded with deploying! Now we have to wonder why it doesn’t work in an existing project…

Yes, that is a mystery. We have had other scenarios where a step have become corrupted (somehow) and removing it/adding it back makes it work again. If you that in your existing project, does it work?

Thank you and best regards,
Henrik

Argh - found the issue. I went to go copy the task into a new one as you suggest, and immediately noticed that it was set to run on Octopus Server instead of on the Tentacle! Not sure how that got changed, but setting it back fixed it. Maybe it would be a good idea to add that radio button selection into the log output? Sorry for wasting your time with such a silly thing, but thank you for the help!

Ah, yes, that would do it.

Great that you found it!
Did you guys recently upgrade Octopus by any chance? I wonder if you could have been bitten by this bug

Thank you and best regards,
Henrik

That is very likely, especially since we are the ones who originally reported it. :slight_smile:

Thank you once again!

Ah sorry, I totally missed that! (I only checked our release notes and saw it there :)).

You are most welcome, and have a great weekend.

Thank you and warm regards,
Henrik