Category Archives: Windows

Spy++ in the case of the missing message

Computing, Development, Suggestions, Tools, Windows

Yesterday, I wrote about WM_QUERYENDSESSION and the app that would not shut down. In that blog, I found a thread that never checked its message queue, and this meant that Windows thought the app was not responding.

So why didn’t Spy++ show the sent message in its log when it was sent? Spy++ uses a SetWindowsHookEx(WH_CALLWNDPROC) hook to log messages sent to windows. It turns out that in Windows NT 4.0 and later versions, the WH_CALLWNDPROC hook is called just before the message is sent to the window procedure. So our message never made it to the hook.

However, in Win9x and NT 3.51, the hook was called when the SendMessage function was called. Spy++ has its heritage in that era. Perhaps now the message delivery title in Spy++ should be changed from “sent” to “received” in order to clarify this difference.

A debug tool wish list item: MsgMon

Computing, Development, Tools, Windows

After running through a painful message debugging session today, I realised again that at the top of my debug tool wishlist tool is MsgMon, a “Procmon for messages”. Spy++ and Winspector Spy are just not up to the task. Once you start using Procmon, with its caching of call stacks, filtering, phenomenal performance and more, life seems very hard each time you start Spy++.

I would love to write this tool, if only I had the time… It would be nice to be able to use the existing Procmon framework as it is so solid…

Debugging an application that would not behave with WM_QUERYENDSESSION

Bugs, Computing, Development, Windows

I was recently tasked with addressing a problem with an application that would not shut down when given a WM_QUERYENDSESSION message. On the surface, this seemed easy enough to fix — just make sure that the WM_QUERYENDSESSION message was appropriately handled by all the top level windows in the application.

Step 1 was to add a handler into the main top level window that avoided the normal shutdown dialog boxes that warned of backup and similar tasks. Where possible, Microsoft advises, you should not block the shutdown of Windows with dialog boxes, etc.

That done, I tested and found that the application was still not shutting down when it was supposed to. So I fired up Spy++ and got started on watching the WM_QUERYENDSESSION chatter in the application. Here I ran into my first little catch-22. I decided that I wanted to do a real shutdown, not a simulated one, to make sure that I was testing the correct scenario.

Problem was, Spy++ was always being shut down before I could capture the messages needed in my target application. Eventually I realised that I needed to start Spy++ after my application, because Windows would ask each application in turn to WM_QUERYENDSESSION, and as we know, my application would cancel the process because it wasn’t accepting a shutdown! Then I realised I could also make sure the shutdown was cancelled even when my application behaved properly, simply by loading Notepad after my application and typing a character or two into it. Then during the shutdown sequence, Iwould be asked if I want to save changes to the text document, and I could just click Cancel. Crude but effective.

With that out of the way, I ran through a Spy++ session and was able to spot an offending window that returned 0 to WM_QUERYENDSESSION. A little tracing and I found a utility window procedure that failed to call DefWindowProc. Naughty. Easily solved.

Well so I thought. Started testing again. The application was still failing to shutdown. I ran through Spy++, traced all the WM_QUERYENDSESSION messages for the process, and every single window procedure responded to the message, virtually instantly, with a nice clean 1, indicating that the window was happy to end the session. After 5 seconds, Windows would show its “Application is not responding” message, indicating that it had not received a response from one of the top level windows.

This was rather puzzling. I threw in a bit of logging into a dll using SetWindowsHookEx for WH_CALLWNDPROC and WH_CALLWNDPROCRET, just in case I was missing something with Spy++. No difference, except that it was somewhat easier to read.

So then I got thinking. This application runs quite a few background threads for loading data from a database. What if, I wondered, one of those threads had a window created but was not running a message loop? It sounded logical but I wasn’t very confident that that was the cause. Problem was, how to tell? I looked for ways to check the queue status for a thread in Windbg but could not find an easy solution. Eventually I decided to add some debugging into the common ThreadProc wrapper for the application, called GetQueueStatus(QS_SENDMESSAGE), and logged the response. Re-ran the application, and bingo, there was a thread closing down with a message in its queue. No prizes if you guess what that message was!

After that, it was simple. Reviewing the thread’s main function I quickly discovered a call to WaitForSingleObject. The thread did not appear to create any windows in any of our code, but it did use COM in its database connectivity, which meant that a helper window was created behind the scenes. I changed the WaitForSingleObject to a MsgWaitForMultipleObjects, and processed any outstanding messages when signalled with a PeekMessage loop.

And then it finally worked (and I forgot to start Notepad, so Windows restarted…) But it really stumped me for a bit today.