HTTP 500.19 on Exchange virtual directories: broken IIS configuration files

Fix it now

HTTP 500.19 is IIS reporting that the configuration data for that path is invalid. Microsoft’s published description points at the applicationHost.config file or the web.config for the application, so IIS never reached Exchange code at all. Browse the failing URL from the server console, where IIS returns the detailed page naming the file and the entry at fault.

appcmd list config "Default Web Site/owa"
Get-WebAppPoolState MSExchangeOWAAppPool
iisreset /noforce

1. Browse to the failing URL from the Exchange server’s own console. Locally IIS returns the detailed 500.19 page, which names the configuration file and often the offending line.
2. Note that file path. It is normally a web.config under the virtual directory, or a file shared by the Exchange Back End site.
3. Compare it with the same file on a healthy Exchange server running the same build. Do not start editing from memory.
4. If it was hand-edited, restore the original. If you cannot, recreate the virtual directory with the matching Remove and New cmdlets, which lays down the file Exchange ships.
5. Run iisreset /noforce and request the path again from the console.

When you recreate an Exchange virtual directory, use the -Role parameter to say whether you mean the Client Access copy on the Default Web Site or the Mailbox copy on Exchange Back End. Recreating the wrong one changes nothing.

If the path answers normally, you are done. If it now fails differently, the next section covers the neighbouring codes and what each one actually means.

Why it happens

Every IIS application reads a merged configuration: the machine-level applicationHost.config first, then each web.config down the path to the application. If any of those cannot be parsed, contains a section the application is not permitted to override, or references a module that is not installed, IIS refuses the request rather than guessing. Microsoft’s published description for 500.19 says exactly that – the problem is in the associated applicationhost.config file or in the associated web.config file.

On an Exchange server those files are written by setup and rewritten during every cumulative update. Anything added by hand – a security header, a rewrite rule, an authentication tweak, the residue of a mitigation script – either survives in a half-merged state or is left pointing at a module that is no longer registered. That is why 500.19 so often appears the morning after an update rather than the morning after the edit.

Two neighbours are worth separating. HTTP 500.24 is published as “An ASP.NET impersonation configuration does not apply in Managed Pipeline mode” – note the wording: the setting is not forbidden, it does not apply, because the application is running in the integrated pipeline. Event ID 5011 is different again: it is a Warning from the Windows Process Activation Service reading “A process serving application pool <name> suffered a fatal communication error with the Windows Process Activation Service”, and Microsoft names it as the indicator of a worker process crash. A crashing pool is not a configuration parse failure and should be chased separately.

The useful property of 500.19 is that IIS is unusually forthcoming about it, but only locally. From a remote browser you get a bare status; from the server console you get the file, the line and the entry. That is one reason to work on the console for this particular error even if you normally do not.

Get-OwaVirtualDirectory -Server EX01 | Format-List Identity,InternalUrl,ExternalUrl
Remove-OwaVirtualDirectory -Identity "EX01\owa (Default Web Site)"
New-OwaVirtualDirectory -Server EX01 -Role ClientAccess
Set-OwaVirtualDirectory -Identity "EX01\owa (Default Web Site)" -InternalUrl https://mail.contoso.com/owa
iisreset /noforce

Full reference

Reading the detailed page before changing anything

Recreating an Exchange virtual directory properly

The cmdlets follow one pattern across the virtual directories: a Get to record what is there, a Remove, a New to lay down what Exchange ships, and a Set to put your URLs back. The parameter that people miss is -Role, which takes ClientAccess or Mailbox and decides whether you are creating the front end copy on the Default Web Site or the back end copy on Exchange Back End. Both exist for OWA, ECP, ActiveSync, EWS, MAPI, OAB, PowerShell and Autodiscover, and they are configured differently on purpose.

Hand-editing IIS configuration on an Exchange server is how most of these faults are created in the first place. Prefer restoring or recreating over editing, and take a copy of any file before you change it.

When the pool is crashing rather than the configuration failing

Event ID 5011 changes the investigation completely. Microsoft’s IIS troubleshooting guidance identifies it as the System log record of a worker process crash, carrying the application pool name and the process id, with the error number in the data field. The configuration parsed; the process died afterwards. Chasing web.config at that point wastes the window.

• Check whether the pool is in rapid-fail protection, which stops it after repeated crashes and produces 503 rather than 500.19 for subsequent requests.
• Look for an Application Error entry naming the faulting module at the same timestamp.
• Check whether a third-party filter, agent or inspection product is loaded into the worker process.
• Fix the crash rather than raising the rapid-fail limits, which only hides how often it happens.

Keeping this from recurring at the next update

• Keep a written record of every deliberate change to an Exchange IIS configuration file, with the reason and the date. A cumulative update overwrites them, and the record is what tells you which ones to put back.
• Prefer settings applied through Exchange cmdlets over settings applied in IIS Manager, because the cmdlets survive a directory being recreated in a way that hand edits do not.
• Take a copy of the web.config tree before a cumulative update, so a comparison afterwards takes minutes.
• Check the prerequisites list against the installed Windows features after any server rebuild, before the first CU rather than during it.

Every code this article covers

Confirm the fix worked

1. The previously failing URL returns a normal response when browsed from the server console.
2. appcmd list config for that path returns without a configuration error.
3. The relevant application pool is started and is still started an hour later.
4. No new Event ID 5011 entries appear in the System log, which would mean the pool is now dying for a different reason.
5. The URLs and authentication settings you recorded before recreating the directory are back in place.

Questions people ask about this