Reproducing the TrustedInstaller.exe leak

Earlier today I blogged about the extraordinary amount of time it was taking to install updates on a Windows 7 virtual machine.  I believed that the issue came down to TrustedInstaller.exe taking what seemed to be an ever increasing amount of RAM and opening an inordinate number of handles.

After writing this blog post, I wondered who at Microsoft might be interested in the issue.  I ended up tweeting a link to Mark Russinovich in the hope that it would whet his interest (the blog did have pictures of Process Explorer, one of Mark’s great Sysinternals utilities ☺):

And verily, Mark did reply:

As Mark was kind enough to respond I knew I had to go and reproduce the issue!  So I built a new VM of Windows 7 Ultimate x64 with stock standard settings, but only 1GB of RAM (oops):

Stock Windows 7 Ultimate x64 install on a VM

I configured Performance Monitor to trace Working Set and Handle counters on the TrustedInstaller.exe process, and saved the settings into a Data Collector Set so that I could capture the whole update process.
 
Then I started the Windows Update:

Windows Update: I selected the additional important update, but none of the optional updates

Three hours later (I did say whoops about the RAM, didn’t I?) Windows finished installing updates, and I was able to generate a report from the Data Collector Set that showed the following graph:

Graph of Working set and Handle Count + summary of Working Set detail

and:

Handle Count detail

At the end of the process we see 34,500 open handles, and a peak of 600MB memory usage.  As one would expect, the RAM usage went up and down quite a bit as updates were installed.  More disturbingly, the Handle Count really did not decrease significantly through the process.  So the real message of the graph is the trend for both the green Handle Count, and the red Working Set, which are both clearly heading the wrong way: up.

There were fewer updates to install than with the other VM, which was also installing Microsoft Office Updates; I’m guessing this had an impact on the overall numbers.

Minor note: I didn’t fix the time zone for the VM.  I wasn’t really doing this at 3AM!

15 thoughts on “Reproducing the TrustedInstaller.exe leak

  1. Any updates on this one? Today my domain controller crashed due to trustedinstaller.exe hogging 3.5GB RAM. FYI this was in my event log:

    Log Name: System
    Source: Microsoft-Windows-Resource-Exhaustion-Detector
    Date: 13/12/2011 1:09:36
    Event ID: 2004
    Task Category: Resource Exhaustion Diagnosis Events
    Level: Warning
    Keywords: Events related to exhaustion of system commit limit (virtual memory).
    User: SYSTEM
    Computer:
    Description:
    Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: TrustedInstaller.exe (908) consumed 3599282176 bytes, dllhost.exe (1332) consumed 62541824 bytes, and lsass.exe (500) consumed 54759424 bytes.

  2. I just googled this problem and found this thread after my pc stuttered significantly, and my processmanager showing trustedinstaller.exe using 1.6GB ram, using up all my ram.

  3. Hi Marc,

    Good find, experiencing the same here. Although my machine is neither a server nor a VM, but a gaming machine.

    I’m running Windows Home Premium, a new Ivy Bridge CPU and 12GB of memory – of which 11GB is being utilized by TrustedInstaller.exe.

    After waiting an hour for one update last night, I just assumed there was an issue with the server and left it over night. When I woke this morning I discovered my machine was on it’s knees and TrustedInstaller.exe was utilizing 99% of my system memory.

    Hopefully they find a solution for this.

  4. This post has an entry that seems to sort it out on my server 2008 R2 server. (still uses high cpu and ram but drops right back when idle)

    I just removed a folder called 6.1.7601.17592 from the c:\windows\servicing\version folder

    I went take ownership to be able to remove it.

    SNIP
    http://www.petri.co.il/forums/showthread.php?t=38462

    KB955430 / SP2 hang indefinitely at about 90%

    The solution: A bit of reverse logic.
    I found Process Monitor shows KB955430 apparently looking for more files in
    C:\Windows\Servicing\Version\6.0.6001.18000.

    How to fix this.
    Delete folder 6.0.6001.18000.
    (It has to be done during boot, so you need a utility to do that).
    Then, restart, launch KB955430 or SP2 which includes that at the start- hey presto – success.

    So: the back story:
    When you buy a PC with Vista pre-installed, (some) manufacturers include this folder.

    KB955430 has a bug. If it sees that folder it goes into an infinite loop.
    I assume if SP1 were installed ‘normally’ the folder would not exist.

    This has appears to have potential significance for many.

    I have advised MS update escalation support. Hope this helps!

  5. So Marc,

    Has there been any success in reducing the resources that TrustedIntaller collects. I have a system the its taking close to a 1GB of RAM.

Leave a Reply

Your email address will not be published. Required fields are marked *