Earlier today I blogged about the extraordinary amount of time it was taking to install updates on a Windows 7 virtual machine. I believed that the issue came down to TrustedInstaller.exe taking what seemed to be an ever increasing amount of RAM and opening an inordinate number of handles.
After writing this blog post, I wondered who at Microsoft might be interested in the issue. I ended up tweeting a link to Mark Russinovich in the hope that it would whet his interest (the blog did have pictures of Process Explorer, one of Mark’s great Sysinternals utilities ☺):
And verily, Mark did reply:
As Mark was kind enough to respond I knew I had to go and reproduce the issue! So I built a new VM of Windows 7 Ultimate x64 with stock standard settings, but only 1GB of RAM (oops):
 |
| Stock Windows 7 Ultimate x64 install on a VM |
I configured Performance Monitor to trace Working Set and Handle counters on the TrustedInstaller.exe process, and saved the settings into a Data Collector Set so that I could capture the whole update process.
Then I started the Windows Update:
 |
| Windows Update: I selected the additional important update, but none of the optional updates |
Three hours later (I did say whoops about the RAM, didn’t I?) Windows finished installing updates, and I was able to generate a report from the Data Collector Set that showed the following graph:
 |
| Graph of Working set and Handle Count + summary of Working Set detail |
and:
 |
| Handle Count detail |
At the end of the process we see 34,500 open handles, and a peak of 600MB memory usage. As one would expect, the RAM usage went up and down quite a bit as updates were installed. More disturbingly, the Handle Count really did not decrease significantly through the process. So the real message of the graph is the trend for both the green Handle Count, and the red Working Set, which are both clearly heading the wrong way: up.
There were fewer updates to install than with the other VM, which was also installing Microsoft Office Updates; I’m guessing this had an impact on the overall numbers.
Minor note: I didn’t fix the time zone for the VM. I wasn’t really doing this at 3AM!
Did you get a response from Mark after showing him this post with the added details?
Got a tweet yesterday; he hasn’t heard from the team that is responsible.
Any updates on this one? Today my domain controller crashed due to trustedinstaller.exe hogging 3.5GB RAM. FYI this was in my event log:
Log Name: System
Source: Microsoft-Windows-Resource-Exhaustion-Detector
Date: 13/12/2011 1:09:36
Event ID: 2004
Task Category: Resource Exhaustion Diagnosis Events
Level: Warning
Keywords: Events related to exhaustion of system commit limit (virtual memory).
User: SYSTEM
Computer:
Description:
Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: TrustedInstaller.exe (908) consumed 3599282176 bytes, dllhost.exe (1332) consumed 62541824 bytes, and lsass.exe (500) consumed 54759424 bytes.
I just googled this problem and found this thread after my pc stuttered significantly, and my processmanager showing trustedinstaller.exe using 1.6GB ram, using up all my ram.
The story is not over yet:
Further analysis on the TrustedInstaller.exe memory leaks
Hi Marc,
Very helpful findings, any update on this issue?
你好 力馬, Sorry, no more from Microsoft. Did you read my more recent blog which details some more technical aspects of the issue at http://marc.durdin.net/2012/02/further-analysis-on-trustedinstallerexe.html ?
Hi Marc,
Yes, I did. It seems no solution yet. I think MS will help it if someone open a support ticket. Definitely, need to pay for it, hehe.
Same issue on server 2008 trustedinstaller using 800MB after service restart 7MB of RAM
cheers,
ISP
Hi Marc,
Good find, experiencing the same here. Although my machine is neither a server nor a VM, but a gaming machine.
I’m running Windows Home Premium, a new Ivy Bridge CPU and 12GB of memory – of which 11GB is being utilized by TrustedInstaller.exe.
After waiting an hour for one update last night, I just assumed there was an issue with the server and left it over night. When I woke this morning I discovered my machine was on it’s knees and TrustedInstaller.exe was utilizing 99% of my system memory.
Hopefully they find a solution for this.
This post has an entry that seems to sort it out on my server 2008 R2 server. (still uses high cpu and ram but drops right back when idle)
I just removed a folder called 6.1.7601.17592 from the c:\windows\servicing\version folder
I went take ownership to be able to remove it.
SNIP
http://www.petri.co.il/forums/showthread.php?t=38462
KB955430 / SP2 hang indefinitely at about 90%
The solution: A bit of reverse logic.
I found Process Monitor shows KB955430 apparently looking for more files in
C:\Windows\Servicing\Version\6.0.6001.18000.
How to fix this.
Delete folder 6.0.6001.18000.
(It has to be done during boot, so you need a utility to do that).
Then, restart, launch KB955430 or SP2 which includes that at the start- hey presto – success.
So: the back story:
When you buy a PC with Vista pre-installed, (some) manufacturers include this folder.
KB955430 has a bug. If it sees that folder it goes into an infinite loop.
I assume if SP1 were installed ‘normally’ the folder would not exist.
This has appears to have potential significance for many.
I have advised MS update escalation support. Hope this helps!
Anonymous, thank you for the post but I would be hesitant to start deleting folders without understanding the root cause of the issue!
So Marc,
Has there been any success in reducing the resources that TrustedIntaller collects. I have a system the its taking close to a 1GB of RAM.
No news to report here, sorry.
This issue has at least half year old. I am seeing this all the time during patching w7 pro workstations. The issue was introduced after they patch svchost.exe memory leak.
Example screenshot: https://postimg.org/image/8ul2vnbkp/
Here’s a status over 6 years(!) later:
It’s still infuriatingly broken, and there’s absolutely no sign og an official fix or workaround from Microsoft.
Lose all faith, ye who enter here.