Every now and then I receive a payment via PayPal. That’s not unusual, right? PayPal would send me an email notifying me of the payment, and I’d open up Outlook to take a look at it. All well and good. In the last week, however, something changed. When I clicked on any PayPal email, Outlook would take the best part of a minute to open the email, and what’s more, would freeze its user interface entirely while doing this. But this was only happening with emails from PayPal — everything else was fine.
Not good. At first I suspected an addin was mucking things up, so I disabled all the Outlook addins and restarted Outlook for good measure. No difference. Now I was getting worried — what if this was some unintended side-effect of some malware that had somehow got onto my machine, and it was targeting PayPal emails?
So I decided to do some research. I fired up SysInternals’ Process Monitor, set it up to show only Outlook in its filtering, and turned on the Process Monitor trace.
Process Monitor filter window – filtering for OUTLOOK.EXE |
Then I went and clicked on a PayPal email. Waited the requisite time for the email to display, then went back to Process Monitor and turned off the trace. I added the Duration column to make it easier to spot an anomalous entry. This doesn’t always help but given the long delay, I was expecting some file or network activity to be taking a long time to run.
Adding the Duration column |
Then scrolling up the log I quickly spotted the following entry. It had a duration of nearly 3 seconds which stood out like a sore thumb.
The first offending entry |
This entry was a Windows Networking connection to connect to a share on the remote host \\102.112.2o7.net. This came back, nearly 3 seconds later, with ACCESS DENIED. Then there were a bunch of follow up entries that related to this, all in all taking over 30 seconds to complete. A quick web search revealed that this domain with its dubious looking name 102.112.2o7.net belongs to a well known web statistics company called Omniture. That took some of the load off my mind, but now I was wondering how on earth opening a PayPal email could result in Internet access when I didn’t have automatic downloads of pictures switched on.
One of the emails that caused the problem, redacted of course 🙂 |
I opened the source of the PayPal HTML email and searched for “102.112“. And there it was.
The HTML email in notepad |
That’s a classic web bug. Retrieving that image of a 1×1 pixel size, no doubt transparent, with some details encoded in the URL to record my visit to the web page (or in this case, opening of the email):
What was curious about this web bug was the use of the “//” shorthand to imply the same protocol (e.g. HTTP or HTTPS) as the base page. That’s all well and good in a web browser, but in Outlook, the email is not being retrieved over HTTP. So Outlook interprets this as a Windows Networking address, and attempts a Windows Networking connection to the host instead, \\102.112.2o7.net\b….
At this point I realised this could be viewed as a security flaw in Outlook. So I wrote to Microsoft instead of publishing this post immediately. Microsoft responded that they did not view this as a vulnerability (as it does not result in system compromise), but that they’d pass it on to the Outlook team as a bug.
Nevertheless, this definitely has the potential of being exploited for tracking purposes. One important reason that external images are not loaded by default is to prevent third parties from being notified that you are reading a particular email. While this little issue does not actually cause the image to load (it is still classified as an external image), it does cause a network connection to the third party server which could easily be logged for tracking purposes. This network connection should not be happening.
So what was my fix? Well, I don’t really care about Omniture or whether PayPal get their statistics, so I added an entry in my hosts file to block this domain. Using an invalid IP address made it fail faster than the traditional use of 127.0.0.1:
0.0.0.0 102.112.2o7.net
And now my PayPal emails open quickly again.
Terrific workaround, but what about your customers? PayPal needs to fix this imediately.
Thanks Axel for the comment. PayPal’s email is not doing anything wrong – the // scheme shorthand is entirely appropriate as it means no JavaScript is needed to match the pages’s scheme as other HTML boilerplate often uses, e.g. Google Analytics. PayPal or Omniture may be well advised to change this in order to avoid the issue, but the more serious issue is with Outlook, which is leaking information; what if this was spam or phishing? This technique could be use to verify email addresses (as web bugs were used for a few years ago until image downloading was disabled by default in email client updates)
The problem actually is not within Outlook, but in Word which for some retarded reason is the engine Outlook uses for displaying html mails.
The // shortcut is entirely inappropriate in an email context. The interpretation has been that the user agent should load this resource with the same protocol that the HTML was loaded, which is neither HTTP nor HTTPS in this instance. Paypal/Omniture should hardcode the https: in the URLs, the // shortcut only makes sense for HTML content which could be served either by HTTP or HTTPS off a web server.
I always find it a bit funny when someone uses Outlook on Windows and then complains about stuff like this. Really, what did you *think* was going to happen? And how’s that version of IE treating you?
epc, yes, you are right when I think about it a little more. Sorry Axel 🙂 So both Microsoft and PayPal should be fixing their respective issues.
For those who were asking, when using 0.0.0.0 for the hosts entry, the SMB (Windows Networking) lookup fails with BAD NETWORK PATH in roughly 1ms on my netbook; when using 127.0.0.1, the lookup failed with LOGIN FAILURE and took the best part of a second (a bit faster on subsequent attempts). So that’s why I used 0.0.0.0. As monitored with Procmon 🙂
Thanks for pointing it out – I’ve notified our development team to get it fixed asap. Cheers!
Praveen, that’s awesome! A fantastic feeling when I can reach the right people to address a problem that I encounter 🙂 Everyone wins.
just wanted to confirm – it’s fixed last night. Thanks again for pointing the issue to us – really appreciate it.
Praveen, thanks heaps for proactive fix. PayPal emails this morning looking good, workaround dismantled 🙂
Interesting fix however it won’t stop the scammers now doing the same that the secret is out.
This is why secure network’s only permit traffic they actually know about and block everything else!
This is the work around I am looking for long time. It is really frustrating.