FLTMGR_FILE_SYSTEM 0x000000F5 – antivirus minifilter crashes the kernel

Fix it now

0x000000F5 is an unrecoverable failure in the Filter Manager, raised because a minifilter broke one of the rules the manager enforces. Parameter 1 says which rule. The driver at fault is whichever minifilter it caught, not fltmgr.sys, so the job is to identify the filter and replace or update it.

fltmc filters

1. Read parameter 1 of the stop from the System log event. 0x66 is a missing post-operation callback, 0x67 is an internal object out of space, 0x68, 0x6D and 0x6E are reference counting errors on names and contexts, and 0x6A is a create that could not be cancelled because handles already exist.
2. Match every third-party name in the fltmc filters output to an installed product. Anything you cannot match is a leftover and should come off.
3. Uninstall the suspect security or backup agent with the vendor’s own removal utility rather than Programs and Features, then reboot.
4. Run fltmc filters again and confirm the filter has gone rather than merely gone quiet.
5. Install a current supported release of the one endpoint product you are keeping. Do not put back the build that crashed.

Data loss prevention agents, encryption products and some backup clients install scanning minifilters too. If you are only counting antivirus, you are undercounting the stack.

If the filter list is now what you intended and the host is stable, stop here. If it is not, the next section covers what the Filter Manager checks and how the three companion codes fit around it.

Why it happens

A minifilter does not talk to the file system directly. It registers with the Filter Manager, declares an altitude that fixes its position in the stack, and supplies pre-operation and post-operation callbacks for the I/O it cares about. The Filter Manager calls those callbacks in altitude order going down and in reverse coming back, and it owns the bookkeeping: which callback data belongs to which request, which file objects are still valid and how many references each context holds.

0x000000F5 is that bookkeeping being violated, and unusually for a bugcheck it tells you exactly which rule was broken. A minifilter that returns FLT_PREOP_SUCCESS_WITH_CALLBACK or FLT_PREOP_SYNCHRONIZE from a pre-operation callback without registering the matching post-operation callback gives you subtype 0x66. A context dereferenced once too often gives 0x6D; a context referenced after it was freed gives 0x6E. A create that could not be cancelled because a handle had already been created for the file gives 0x6A.

That last one is worth holding on to, because it is the same story as 0x000000E8 told from the other side. 0x000000E8 says an invalid file object was passed to IoCancelFileOpen and that the calling driver is at fault; the object should have had a reference count of one. Both are a filter trying to make a file open go away after it has already handed out a handle.

The other two companions describe neighbouring failures in the same layer. 0x00000018 is an object whose reference count is illegal for its current state, which Microsoft attributes to a driver dereferencing an object too many times. 0x00000076 is a driver that failed to release pages it locked for an I/O operation, or tried to unlock pages that were not locked. All of these are the sort of mistake an anti-virus minifilter makes when it is scanning a file at the moment something else deletes, renames or reparses it.

Full reference

Filter Manager subtypes and what each one tells the vendor

Subtypes 0x6B and 0x6C are internal state errors that the documentation describes as non-recoverable, with no reader-side action. If you have one of those, the dump goes to the vendor and there is nothing to configure in the meantime.

Narrowing it down before you remove anything

Finding the driver that leaks locked pages

0x00000076 is the one case in this family where Windows will name the driver for you rather than leaving you to read a stack. Microsoft documents a tracking mode for exactly this.

1. Under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management, create or edit the TrackLockedPages value and set it to DWORD 1.
2. Restart the machine. The system then saves stack traces for locked page operations.
3. Wait for the fault to recur. It is then reissued as bug check 0xCB, DRIVER_LEFT_LOCKED_PAGES_IN_PROCESS, with the name of the offending driver displayed on the stop screen.
4. Remove the tracking value once you have the name, because it costs memory and performance to keep it on.

Commands for this layer

When the filter list is clean and it still stops

• Check the altitude of the surviving filter against the vendor’s documentation. A filter loading at an altitude it was not designed for interacts with the layers around it differently.
• Look at what else touches file objects: file screening, quota, deduplication and search indexing all sit in this path on a file server.
• Read subtype 0x67 literally if you have it. It is a pool exhaustion symptom, so find what is consuming non-paged pool rather than blaming the filter that noticed.
• Collect three dumps before you open a vendor case. One dump gives you a module name; three that agree give you a case.

Every code this article covers

Confirm the fix worked

1. Run fltmc filters and confirm only the filters you expect are present.
2. Confirm no new bugcheck entries appear in the System log over a full working day.
3. Reproduce the workload that previously stopped the machine and confirm it completes.
4. Confirm the surviving agent reports itself healthy in its own console, so you have not traded a stop for an unprotected machine.
5. If you set TrackLockedPages, confirm it has been removed once the driver was identified.

Questions people ask about this