Category Archives: Networking

Risks with third party scripts on Internet Banking sites

This morning, Firefox stalled while loading the login page for the ANZ Internet Banking website. Looking at the status bar, I could see that Firefox was attempting to connect to a website, australianewzealandb.tt.omtrdc.net. This raised immediate alarm bells, because I didn’t recognise the website address, and it certainly wasn’t an official anz.com sub-domain.

ANZ login delay - note the status message at the bottom of the window
ANZ login delay – note the status message at the bottom of the window

The connection eventually started, and the page finished loading — just one of those little glitches loading web pages that everyone encounters all the time, right? But before I entered my ID and password, I decided I wasn’t comfortable to continue without knowing what that website was, and what resources it was providing to the ANZ site.

And here’s where things got a little scary.

It turns out that australianewzealandb.tt.omtrdc.net is a user tracking web site run by marketing firm Omniture, now a part of Adobe.  The ANZ Internet Banking login page is requesting a Javascript script from the server, which currently returns the following tiny piece of code:

if (typeof(mboxFactories) !== 'undefined') {mboxFactories.get('default').get('SiteCatalyst: event', 0).setOffer(new mboxOfferDefault()).loaded();}

The Scare Factor

This script is run within the context of the Internet Banking login page. What can scripts that run within that context do? At worst, a script can be used to watch your login and password and send them (pretty silently) to a malicious host. This interaction may even be undetectable by the bank, and it would be up to you and your computer to be aware of this and block it — a big ask!

At worst, a script can be used to watch your login and password and send them to a malicious host.

The Relief

Now, this particular script is fortunately not malicious! In fact, as the mboxFactories variable actually is undefined on this page, this script does nothing at all. In other words, it’s useless and doesn’t even need to be there!  (It’s defintely possible that the request for the script is being used on the server side to log client statistics, given the comprehensive parameters that are passed in the HTTPS request for the script.)

What are the risks?

So what’s the big deal with running third party script on a website?

The core issue is that scripts from third party sites can be changed at any time, without the knowledge of the ANZ Internet Banking team. In fact, different scripts can be served for different clients — a smart hacker would serve the original script for IP addresses owned by ANZ Bank, and serve a malicious script only to specific targeted clients. There would be no reliable way for the ANZ Internet Banking security team to detect this.

Scripts from third party sites can be changed at any time, without the knowledge of the Internet Banking team.

Another way of looking at this: it’s standard practice in software development to include code developed by other organisations in applications or websites. This is normal, sensible, and in fact unavoidable. The key here is that any code must be vetted and validated by a security team before deployment. If the bank hosts this code on their own servers, this is a straightforward part of the deployment process. When the bank instead references a third party site, this crucial security step is impossible.

Banking websites are among the most targeted sites online, for obvious reasons. I would expect their security policies to be conservative and robust. My research today surprised me.

Trust

How could third party scripts go wrong?

First, australianewzealandb.tt.omtrdc.net is not controlled by ANZ Bank.  It’s controlled by a marketing organisation in another country.  We don’t know how much emphasis they place on security. We are required to trust a third party from another country in order to login to our Internet Banking.

This means we need to trust that none of their employees are malicious, that they have strong procedures in place for managing updates to the site, the servers and infastructure, and that their organisation’s aims are coincident with the tight security requirements of Internet Banking. They need to have the same commitment to security that you would expect your bank to have. That’s a big ask for a marketing firm.

Security

The ANZ Internet Banking website is of course encrypted, served via HTTPS, the industry standard method of serving encrypted web pages.

We can tell, just by looking at the address bar, that anz.com uses an Extended Validation certificate.

With a little simple detective work, we can also see that anz.com serves those pages using the TLS_RSA_WITH_AES_256_CBC_SHA encryption suite, using 256-bit keys.  This is a good strong level of encryption, today.

However, australianewzealandb.tt.omtrdc.net does not measure up. In fact, this site uses 128-bit RC4+SHA encryption and integrity and does not have an Extended Validation certificate. RC4 is not a good choice today, and neither is SHA. This suggests immediately that security is not their top concern, which should then be an immediate concern to us!

ANZ vs Omtrdc Security
ANZ vs Omtrdc Security

I should qualify this a little: Extended Validation certificates are not available for wildcard domains, which is the type of certificate that tt.omtrdc.net is using. This is for a good reason: “in order to ensure that EV SSL Certificates are not issued fraudulently or misused after issuance.” It’s worth thinking through that reason and seeing how it applies to this context.

Malicious Actors

So how could a nasty person steal your money?

In theory, if a nasty person managed to hack into the Adobe server, they could simply supply a script to the Internet Banking login page that captures your login details and sends them to a server, somewhere, anywhere, on the Internet. This means that we have to trust (there’s that word again) that the marketing firm will be proactive in updating and patching not only their Internet-facing servers, but their infrastructure behind those servers as well.

If a bad actor has compromised a certificate authority, as has happened several times recently, they can target these third party servers . Together with a DNS cache poisoning or Man-In-The-Middle (MITM) attack, even security-savvy users will be unlikely to notice fraudulent certificates on the script servers.

heartbleedSecurity flaws like Heartbleed are exacerbated by this setup. Not only do the bank security team have to patch their own servers, they also have to push the third party vendors to patch theirs as a priority.

Protect Yourself

As a user, run security software. That’s an important first step. Security software is regularly refreshed with blacklists of known malicious sites, and this will hopefully minimize any window of opportunity that an untargeted attack may have. I’m not going to recommend any particular brand, because I pretty much hate them all.

If you want to unleash your inner geek and be aware of how sites are using third party script servers, you can use Developer Tools included in your browser — press F12 in Internet Explorer, Chrome or Firefox, and look for the Network tab to see a list of all resources referenced by the site. You may need to press Ctrl+F5 to trigger a ‘hard’ refresh before the list is fully populated.

I’ve shown below the list of resources, filtered for Javascript, for the National Australia Bank Internet Banking site.  You can see two scripts are loaded from one site — again, a market research firm.

nab-resources

Simplistic Advice for Banks

Specifically to mitigate this risk, banks should consider the following:

  • Serve all scripts from your own domain and vet any third party scripts that you serve before deployment.
  • In particular, check third party scripts for back end communication, via AJAX or other channels.
  • Minimize the number of third party scripts anywhere that secure content must be presented.
  • Use the Content-Security-Policy HTTP header to prevent third party scripts on supported browsers (most browsers today support this).

There are of course other mitigations, such as Two Factor Authentication (2FA), which do reduce risk. However, even 2FA should not be considered a silver bullet: it is certainly possible to modify the login page to take over your current login in real time — all the user would see is a message that they’d mistyped their password, and as they login again, the malicious hacker is actively draining money from their account.

A final thought on 2FA: do you really want a hacker to have your banking password, even if they don’t have access to your phone? Why do we have these passwords in the first place?

Browser Developers

I believe that browser vendors could mitigate the situation somewhat by warning users if secure sites reference third party sites for resources, in particular where these secure sites have lower quality protection than the first party site. This protection is already in place where content is requested over HTTP from a HTTPS site, known as mixed content warnings.

There is no value in an Extended Validation certificate if any of the resources requested by the site are served from a site with lower quality encryption! Similarly, if a bank believes that 256-bit AES encryption is needed for their banking website, a browser could easily warn the user that resources are being served with lower quality 128-bit RC4 encryption.

Australian Banks

After this little investigation, I took a quick look at the big four Australian banking sites — ANZ, Commonwealth Bank, National Australia Bank, and Westpac.  Here’s what I found; this is a very high-level overview and contains only information provided by any web browser!

Bank Bank site security # 3rd party scripts Third party sites Third party security
ANZ Bank 256-bit AES (EV certificate) 1 australianewzealandb.tt.omtrdc.net 128-bit RC4
NAB 256-bit AES (EV certificate) 2 survey.112.2o7.net 256-bit AES
Westpac 128-bit AES (EV certificate) None!
Commonwealth Bank 128-bit AES (EV certificate) 9! ssl.google-analytics.com 128-bit AES
commonwealthbankofau.tt.omtrdc.net 128-bit RC4
google-analytics.com 128-bit AES
d1wscoizcbxzhp.cloudfront.net 128-bit AES
cba.demdex.net 128-bit AES

Do you see how the *.tt.omtrdc.net subdomains are used by two different banking sites? In fact, this domain is used by a large number of banking websites. That would make it an attractive target, wouldn’t you think?

I reached out to all 4 banks via Twitter (yeah, I know, I know, “reached out”, “Twitter”, I apologise already), and NAB was the first, and so far only, bank to respond:

Kudos are due to NAB and Westpac — NAB for responding so promptly, and Westpac, for not having the issue in the first place!

Updates (6:10am, 9 Sep 2014), with thanks, in no particular order:

Many thanks to Troy Hunt for suggesting I write this, then tweeting it — and for his continual and tireless work in websec!

Stefano Di Paola mentioned a previous Omniture vulnerability and referenced 3rd party script risks in his blog:

hillbrad⚡ mentioned a W3C project to make validation of sub-resource integrity possible:

Erlend Oftedal reminded me that this is not a new issue and mentioned his blog post from 2009:

A story, or will pay anybody five pounds to remove database from one computer to another

Today I needed to download a recent backup of a database from a server in the data centre for testing major changes to the database locally…  Things don’t really go as well as I expect:

  1. Backup is 12GB.  Yike, that’s going to take ages to download!
  2. OK, so let’s zip it.  20 minutes later… we have a 2GB zip file.  Well, that was worth doing.
  3. Now copy (encrypted) zip to a server with HTTP access (much faster than downloading over SSH).  About 1 minute (yep, nice fast network in the data centre)
  4. Download zip to database server on local network.  Takes about 45 minutes.
  5. So try to unzip on server with Win Server 2003 Zip tool.  Huh.  It doesn’t support files > 4GB (plus it’s running low on disk space, so a bit of a juggling act to have enough space to unzip anyway).
  6. OK, move the zip file to a client machine.  5 minutes on LAN
  7. Unzip locally then move 12GB backup back to server.  Best part of an hour.  Yes, it’s a slow client machine and the LAN is only 100mbit.
  8. Try to restore to SQL Server.  Oops, old version of SQL Server on that server.
  9. So install SQL Server 2008 on another server.  That takes 45 minutes (including adding .NET framework).
  10. Can’t do anything with it until we have patched SQL Server.  So download and run SQL Server 2008 SP3.  20 minutes to download.
  11. Installer starts while I’m not at computer, automatically cancels when screensaver starts (consent.exe)
  12. Drat.  Download SQL Server 2008 SP3 again as IE has deleted installer from cache.  This time I save the service pack installer.  20 minutes.
  13. Finally! Install SQL Server 2008 SP3.  10 minutes.
  14. Restore database to new server.  5 minutes.

And finally it works.

Why do the little things take so much energy?

The case of the terribly slow PayPal emails

Every now and then I receive a payment via PayPal.  That’s not unusual, right?  PayPal would send me an email notifying me of the payment, and I’d open up Outlook to take a look at it.  All well and good.  In the last week, however, something changed.  When I clicked on any PayPal email, Outlook would take the best part of a minute to open the email, and what’s more, would freeze its user interface entirely while doing this.  But this was only happening with emails from PayPal — everything else was fine.

Not good.  At first I suspected an addin was mucking things up, so I disabled all the Outlook addins and restarted Outlook for good measure.  No difference.  Now I was getting worried — what if this was some unintended side-effect of some malware that had somehow got onto my machine, and it was targeting PayPal emails?

So I decided to do some research.  I fired up SysInternals’ Process Monitor, set it up to show only Outlook in its filtering, and turned on the Process Monitor trace.

Process Monitor filter window – filtering for OUTLOOK.EXE

Then I went and clicked on a PayPal email.  Waited the requisite time for the email to display, then went back to Process Monitor and turned off the trace.  I added the Duration column to make it easier to spot an anomalous entry.  This doesn’t always help but given the long delay, I was expecting some file or network activity to be taking a long time to run.

Adding the Duration column

Then scrolling up the log I quickly spotted the following entry.  It had a duration of nearly 3 seconds which stood out like a sore thumb.

The first offending entry

This entry was a Windows Networking connection to connect to a share on the remote host \\102.112.2o7.net.  This came back, nearly 3 seconds later, with ACCESS DENIED.  Then there were a bunch of follow up entries that related to this, all in all taking over 30 seconds to complete.  A quick web search revealed that this domain with its dubious looking name 102.112.2o7.net belongs to a well known web statistics company called Omniture.  That took some of the load off my mind, but now I was wondering how on earth opening a PayPal email could result in Internet access when I didn’t have automatic downloads of pictures switched on.

One of the emails that caused the problem, redacted of course 🙂

I opened the source of the PayPal HTML email and searched for “102.112“.  And there it was.

The HTML email in notepad

That’s a classic web bug.  Retrieving that image of a 1×1 pixel size, no doubt transparent, with some details encoded in the URL to record my visit to the web page (or in this case, opening of the email):

What was curious about this web bug was the use of the “//” shorthand to imply the same protocol (e.g. HTTP or HTTPS) as the base page.  That’s all well and good in a web browser, but in Outlook, the email is not being retrieved over HTTP.  So Outlook interprets this as a Windows Networking address, and attempts a Windows Networking connection to the host instead, \\102.112.2o7.net\b….

At this point I realised this could be viewed as a security flaw in Outlook.  So I wrote to Microsoft instead of publishing this post immediately.  Microsoft responded that they did not view this as a vulnerability (as it does not result in system compromise), but that they’d pass it on to the Outlook team as a bug.

Nevertheless, this definitely has the potential of being exploited for tracking purposes.  One important reason that external images are not loaded by default is to prevent third parties from being notified that you are reading a particular email.  While this little issue does not actually cause the image to load (it is still classified as an external image), it does cause a network connection to the third party server which could easily be logged for tracking purposes.  This network connection should not be happening.

So what was my fix?  Well, I don’t really care about Omniture or whether PayPal get their statistics, so I added an entry in my hosts file to block this domain.  Using an invalid IP address made it fail faster than the traditional use of 127.0.0.1:

0.0.0.0    102.112.2o7.net

And now my PayPal emails open quickly again.

How to connect to a Netgear DG834G router in Windows 7

Trying to connect my Windows 7 beta AspireOne to our wireless network (with a Netgear DG834G router), was receiving an unspecified error – “Windows failed to connect”. After playing with a multitude of settings, including security, access control, manual configuration and more, I discovered that changing wireless mode on the router from “g only” to “g & b” solved all my problems…

After that I went back and turned on all the security settings on the router again!