Category Archives: JavaScript

dynamic import on node.js with circular dependencies leads to interesting failure modes

So, given these three ES modules below, what output do you expect to get when you run node one.js?

one.js:

import { two } from './two.js';

await two();

export function one() {
  console.log('one');
}

two.js:


export async function two() {
  console.log('two');
  const { three } = await import('./three.js');
  three();
}

three.js:

import { one } from './one.js';
export function three() {
  console.log('three');
  one();
}

Following through the calls, you’d expect to get:

two
three
one

But instead you get just the following output:

two

And node exits with exit code 13, which means “Unfinished Top-Level Awaitawait was used outside of a function in the top-level code, but the passed Promise never resolved.” There are no messages reported to the console, which is very disconcerting!

It turns out that the await import() causes this, because of the circular dependency introduced in module three.js, back to one.js. Now, if there is no async involved, then this circular dependency just works; that is, instead of using import('./three.js') you use import { three } from './three.js':

two
three
one

Not sure yet how to move forward with this. Tested in Node 20.9.0 and 18.14.1. Should I be raising it as a bug? It’s certainly not a documented behaviour that I’ve been able to find (node documentationmdn documentation).

location.hash can be re-entrant on Safari Mobile

So Josh and I were observing some really strange behaviour in the Keyman for iOS beta. When typing rapidly on the touch keyboard, we would sometimes get the wrong character emitted. We could not see anything immediately wrong in the code. So Josh added some logging. Then things got really weird: we would get the start of a touch event, then before the touch event handler finished, we’d get logging that indicated another touch event was received.

Whoa. In JavaScript, that shouldn’t be possible, right?

Each message is processed completely before any other message is processed. This offers some nice properties when reasoning about your program, including the fact that whenever a function runs, it cannot be pre-empted and will run entirely before any other code runs (and can modify data the function manipulates). This differs from C, for instance, where if a function runs in a thread, it can be stopped at any point to run some other code in another thread.

— https://developer.mozilla.org/en-US/docs/Web/JavaScript/EventLoop

After we dug in further, we found that it was indeed possible for a new touch event (and perhaps others) to be received when location.hash was set from the JavaScript code, in Safari for iOS.

Here’s a fairly minimal repro. Load this up on your iPhone, and start rapidly touching the Whack div. You’ll probably need to use two fingers repeatedly to trigger the event (I can usually get it to happen with about 50-100 rapid touches). When it happens, you’ll get a log message with a call stack showing how the whackIt() function has apparently called itself!

<!doctype html>
<html>
  <head>
    <meta charset='utf8'>
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no" /> 
    <title>location.hash re-entrancy</title>
    <style>
      #whack { font-size: 64pt }
      #log, #stack { font-family: courier }
    </style>
  </head>
  <body>
    <div id=whack>Whack</div>
    
    <div id=log></div>
    
    <div id=stack></div>
    
    <script>
    
      var tick = 0, inWhack = false;
      
      function whackIt(e) {
        e.preventDefault();

        tick++;
        
        if(inWhack) {
          stack.innerHTML = 'Re-entrant event: '+(new Error()).stack;
          return false;
        }
        
        var localTick = tick;
        log.innerHTML = tick;

        inWhack = true;

        // Setting location.hash seemingly can cause the 
        // event queue to be polled, resulting in a 
        // re-entrant touch event
        location.hash = '#'+localTick;

        inWhack = false;
        
        if(localTick != tick) {
          // alert('We had a re-entrant event');
        }
        
        return false;
      }
      
      whack.addEventListener('touchstart', whackIt, false);
      whack.addEventListener('touchend', whackIt, false);
      
    </script>
  </body>
</html>

We’ll be reporting this to Apple… Just a heads up.

The KeymanWeb Hello World app

A couple of days ago I helped some developers put together a very basic web page that uses KeymanWeb to showcase their keyboard layout. The idea is that a visitor to the page will immediately be able to try out their keyboard layout without any fiddling around.

Another requirement was that the page work well on desktop and mobile devices. This requires a few little tricks, mostly because KeymanWeb has some special requirements to work smoothly on mobiles and tablets.

I’ve presented snippets of the code, in the order they appear in the document.

And here are screenshots just for posterity.

The usual HTML 5 boilerplate is thrown in at the top of the page. Our first trick is to put the page into an appropriate zoom and scroll mode for mobile and tablet devices with a viewport meta tag. This is done with the following code:

<!-- at present, KMW requires this for mobile -->
<meta name="viewport" content="width=device-width, user-scalable=no">

At the time of writing, KeymanWeb (build 408) requires the viewport meta tag in order to render correctly on a variety of devices. Both clauses in the content attribute are required, and the keyboard will not scale correctly without them.

Next, I add a link to the stylesheet for the on screen keyboard. This is optional, because KeymanWeb will inject the link itself anyway, but it does prevent a Flash Of Unstyled Content.

<!-- this is optional but eliminates a flash of unstyled content -->
<link href='https://s.keyman.com/kmw/engine/408/src/osk/kmwosk.css' rel='stylesheet' type='text/css'>

I throw in some basic styling for the desktop and touch devices. You’ll see a selector of body.is-desktop on a couple of the style selectors. I use this instead of a @media query because this is differentiating between viewport-controlled devices and plain old boring desktop browsers. There may be a better way of doing this now, but I haven’t found it yet. The code for setting the is-desktop class is found further on in this post.

/* Styles for the page */

* {
  font-family: Tahoma, sans-serif;
}

body.is-desktop {
  padding: 22px 2px 0 2px;
  width: 900px;
  margin: auto;
}

h1 {
  font-size: 24pt;
  font-weight: bold;
  margin: 0 0 24px 0;
  padding: 0;
}

body.is-desktop h1 {  
  font-size: 48pt;
}

/* Style the text area */

textarea {
  width: 100%;
  height: 250px;
  font-family: "Khmer OS";
  font-size: 24px;
}

I need to talk a bit more about the On Screen Keyboard (OSK) styling. While I could use the default styling (as shown below), this looks a little dated and I wanted to show how the keyboard could be styled as you like. There is some complexity to the OSK styling as it has many moving parts, around sizing, scaling and positioning of each individual key. I don’t want to throw away that baby, so instead I keep all the bathwater and just add some bubbles to jazz up the display the way I want it. That’s a mixed metaphor, but it was late when I wrote this.

The default KeymanWeb keyboard style

First, this CSS rule-set removes the thick red border and dark background, and adjusts the padding around the sides to compensate.

/* Style the on screen keyboard */

body .desktop .kmw-osk-inner-frame {
  background: white;  
  border: solid 1px #404040;
  padding-bottom: 2px;
  padding-top: 8px;
}

Next, I hide the header and footer for the OSK. I don’t need them for this demo.


body .desktop .kmw-title-bar,
body .desktop .kmw-footer {
  display: none;
}

I tweak the spacing between keys and rows and make the keys slightly less rounded.

body .desktop .kmw-key-square {
  border-radius: 2px;
  padding-bottom: 8px;
}
body .desktop .kmw-key {
  border-radius: 2px;
  border: solid 1px #404040;
  background: #f2f4f6;
}
body .desktop .kmw-key:hover {
  background: #c0c8cf;
}

And that’s it for the CSS changes. It’s really pretty easy to restyle the keyboard without losing the benefits of a scalable, cross-platform keyboard. You’ll note that sneaky .desktop selector creeping in there. That’s because I’ve opted to style only the desktop keyboard; the touch layouts are already pretty nice and I’ll keep them as is.

I load the KeymanWeb code from the KeymanWeb CDN. You can load from the KeymanWeb CDN (running on Azure) or keep a locally hosted copy, of course. I’ve commented out the source version and because we have only a single keyboard, I have elected not to include a menu for switching languages.

<!-- Uncomment these lines if you want the source version of KeymanWeb (and remove the keymanweb.js reference)
<script src="https://s.keyman.com/kmw/engine/408/src/kmwstring.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwbase.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/keymanweb.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwosk.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwnative.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwcallback.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwkeymaps.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwlayout.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwinit.js"></script>
-->

<script src="https://s.keyman.com/kmw/engine/408/keymanweb.js"></script>

<!-- Uncomment if you want the user to be able to switch keyboards with a toggle popup ...
or you can create a custom switch if you prefer
<script src='https://s.keyman.com/kmw/engine/408/kmwuitoggle.js'></script>
-->

Now, no self-respecting web developer is going to include inline script, except in a Hello World demo. So let’s call this the KeymanWeb Hello World demo, so I can keep my self respect. Or something.

I use a simple page load listener. You could use an earlier listener, such as DOMContentLoaded, but window.onload has a rich and nearly honourable history.

// This should really be in a separate file. 
// But for now in one file is easier to understand
       
// After everything has loaded
window.addEventListener('load', function() {

Next some voodoo. Allows me to style the touch and desktop versions differently, as I touched on earlier. Why? Because KeymanWeb works very differently on touch devices. If you think about this, it must be so. A touch device has its own soft keyboard, which KeymanWeb must, by slightly convoluted means, hide, and replace with its own soft keyboard. Whereas, on a desktop device, KeymanWeb can show a utility soft keyboard, but does most of its interaction by getting in there between the hardware keyboard and the input fields.

Don’t ask me about Windows touch devices. Does anyone actually use those? (Okay, I’m sure they do, but it’s a rocky path for us poor keyboard devs to tread!)

if(!tavultesoft.keymanweb.util.isTouchDevice()) {
  document.body.className += 'is-desktop';
}

Next, make sure KeymanWeb is initialised. If I don’t do this myself, KeymanWeb will do so when things are ready, but I need KeymanWeb to be ready in order to load keyboards and attach events, and so on.

tavultesoft.keymanweb.init();

I could add another keyboard, or a stock keyboard from the repository. I haven’t, but this shows you how.

//tavultesoft.keymanweb.addKeyboards('@eng'); // Loads default English keyboard from Keyman Cloud (CDN) if you want it

In this case, I have a custom keyboard, developed by Lyheng Phuoy. This is an early beta version of the keyboard but already it’s very impressive.

// Add our custom keyboard        
tavultesoft.keymanweb.addKeyboards({
  name: 'Khmerism',       // Display name of the keyboard
  id: 'lyhengkeyboard',   // ID of the keyboard for reference in code
  filename: './lyhengkeyboard-1.0.js',  // source of the keyboard for dynamic load
  version: '1.0',         // version of the keyboard, optional
  language: [{
    name: 'Khmer',        // language name for UI elements
    id: 'khm',            // language ID for tagging text
    region: 'as'          // region of the language, for UI elements, optional
  }]
});

In this section, I want to control the size, position and flexibility of the keyboard. I don’t want it to be resizable or movable. So I set nomove and nosize accordingly.

var ta = document.getElementsByTagName('textarea')[0];

// Watch for the keyboard being shown and set its position
        
tavultesoft.keymanweb.osk.addEventListener('show', function() {
  var rectParams = {
    left: ta.offsetLeft, 
    top: ta.offsetTop + ta.offsetHeight + 8, // a small gap below the text area
    width: ta.offsetWidth, 
    height: ta.offsetWidth * 0.4, // a pleasing aspect ratio
    nomove: true,
    nosize: true
  };
  tavultesoft.keymanweb.osk.setRect(rectParams);
});
      
// Focus text area after everything loads
        
ta.focus(); 

Finally, in the body element, I set a special magic class, osk-always-visible, so that KeymanWeb doesn’t hide its on screen keyboard after displaying it the first time. And then we have basically the world’s simplest page with a textarea.

<!-- The osk-always-visible class tells KeymanWeb not to hide the osk on blur, after it is
     shown the first time -->
<body class="osk-always-visible">
  <h1>Typing Khmer</h1>
  <textarea rows="6" cols="60"></textarea>	
</body>

And that’s it! I’d love to see what you do with KeymanWeb on your own sites.

Here’s the full keyboard source, and that link to the demo page again, including Lyheng’s keyboard (with his permission).

My Disordered Rebuttal to “AngularJs Performance in Large Applications”

After reading Abraham Polishchuk’s article AngularJS Performance in Large Applications, I wanted to dig a bit deeper on his suggestions before I took any of them on board. My basic concern was many of his suggestions seemed focused around technology-specific performance micro-optimisations. In today’s browser world, optimising around specific technologies (i.e. browsers and browser versions) is an expensive task with a limited life and usefulness: both improvements in the Javascript engines and differences in browser implementations mean that these fine-grained optimisations should really be limited to very specific performance critical situations.

Conversely, I don’t think we can expect Angular’s performance to improve in the same way as browser engines have. While Angular has undergone significant iterative performance improvements in the 1.x series, it is clear from the massive redesign of version 2.0 that most future application performance gains will only be won by rewriting significant chunks of your own code. Be aware!

Understanding computational complexity is crucial for web developers, even though with Angular it is all Javascript and browser and cloud and magic. You still need that software engineering education.

Onto the article.

1-2 Introduction, Tools of the Trade

No arguments here. Helpful.

3 Software Performance

Ok.

4.1 Loops

This advice, to move function calls out of loops, really depends on what the function you are calling is. Now, clearly, it’s sensible to move calls out of a loop if you can – that’s a normal O(n) to O(1) optimisation. And Objects.keys, as shown in the original article, is very heavy function, so that’d be a good one to avoid.

But don’t take this advice too far. The actual cost of the function call itself is minimal (http://jsperf.com/for-loop-perf-demo-basic/2). In general, inlining code to avoid a function call is not advisable: it results in less maintainable code for a minimal performance gain.

4.2 DOM access

While modifications to the DOM should be approached carefully, the reality is a web application is all about modifying the DOM, whether directly, or indirectly. It cannot be avoided! Applying inline styles (as opposed to using CSS selectors predefined in stylesheets?) vary significantly depending on both the styles being set and the browser. It’s certainly not a hard-and-fast rule, and not really a useful one. In the example, I compare background color (which should not force a reflow), and font-size (which would): (http://jsperf.com/inline-style-vs-class). The results vary dramatically across browsers.

4.3 Variable Scope and Garbage Collection

No arguments here. In fact I would go further and say that this is the single biggest performance concern, both in terms of CPU and resource usage, in Angular. In my limited experience, anyway. It’s very easy to leak closures without realising.

4.4 Arrays and Objects

Yes. Well, somewhat. As Gregory Jacobs commented, there is no significant difference between single-type and multi-type arrays. The differences come in what you are doing with the members of that array, and again browsers vary. http://jsperf.com/object-tostring-vs-number-tostring

I would again urge you to avoid implementation-specific optimisations. Designing an application to minimize the number of properties in an object just because today’s implementation of V8 has a special internal representation of small objects is not good advice. Design your objects according to your application’s requirements! This is a good example of a premature optimization antipattern.

Finally, as Wills Bithrey commented, Array.delete isn’t really slow.

5.1 Scopes and the Digest Cycle

This is a clear description of Angular scopes and the digest cycle. Good stuff. Abraham does a good job of unpacking the ‘magic’ of Angular.

6.1. Large Objects and Server Calls

Ok, so in my app I return full rows of data, and there are a few columns I don’t *currently* use in that data. I measured the performance impact on my application, and the cost of keeping those columns was insignificant: I was actually unable measure any difference. Again, this is a premature optimization antipattern. Your wins here will ultimately not be from excluding columns from your data but from making sure your data is normalized appropriately. This is about good design, rather than performance-constrained design.

Of course, there are plenty of other reasons to minimize your data across the wire. But remember that a custom serializer for each database object is another function to test, and another failure point. Measure the cost and the benefit.

6.2 Watching Functions

Yes. Never is a little too strong for me, but yeah, in general, avoid binding to functions.

6.3 Watching Objects

Yes, in general, deep watching of objects does introduce performance concerns. But I do not agree that this is a terrible idea. The alternative he gives (“relying on services and object references to propagate object changes between scopes”) gets pretty complicated and introduces maintenance and reliability concerns. I would approach this point as potential low hanging fruit for optimisation.

7.1 Long Lists

Yes.

7.2 Filters

As DJ commented, filters don’t work by hiding elements by CSS. Filters, as used in ng-repeat, return a transform of the array, which may be what Abraham meant?

7.3 Updating an ng-repeat

Pretty much. I would add that for custom syncing to work, you still need a unique key, just as you do for track by. If you are using Angular 1.3, there’s no benefit to the custom sync approach. Abraham has detailed the maintenance cost of custom syncing.

8. Rendering Problems

ng-if and ng-show are both useful tools. Choose your weapons wisely.

9.1 Bindings

Yes.

9.2 $digest() and $apply()

Yes, if you must. This is a great way to introduce inconsistencies in your application which can be very hard to debug!

9.3 $watch()

I really disagree with this advice. scope.$watch is a very practical way of separating concerns. I don’t think it’s indicative of bad architecture, quite the opposite in fact! While trying to find the discussions that Abraham refers to (but sadly doesn’t link to), it seemed that most of the alternatives given were in fact in themselves bad architecture. For example, Ben Lesh suggested (although he has since relaxed his opinion) using an ng-change on an input field instead of a watcher on the target variable. Why is that bad? From an architectural point of view, that means each place in your code that you change that variable, you have to remember to make that function call there as well. This is antithetical to the MVC pattern and seriously WET.

9.4 $on, $broadcast, and $emit

Apart from the advice to unbind $on(‘$destroy’) which I don’t think is correct, yes, being aware of the performance impact of events is important in an event-driven system. But that doesn’t mean don’t use them: events are a weapon to use wisely. Seeing a pattern here?

9.5 $destroy

Yes. Apart from the advice to “explicitly call your $on(‘$destroy’)” — I’m not sure what this means.

10.1 Isolate Scope and Transclusion

Only one quibble here: isolate scopes or transclusions can be faster than occupying the parent scope, because a tidy directive can use $apply to just update its own isolated scope, and avoid the digest of the entire scope tree. Choose your tools wisely.

10.2 The compile cycle

Basically, yes.

11 DOM Event Problems

Yes, addEventListener is more efficient, but so is not using Angular in the first place! This results in a significant code increase and again you the benefit of Angular in doing so. Going back to raw DOM is always a trade-off. In the majority of cases, the raw DOM event model is a whole lot of additional work for little gain.

12 Summary

In summary, I really think this post is about micro-optimisations that in many cases will not bring you the performance benefits you are looking for. Abraham’s summary suggests avoiding many of the most useful tools in Angular. If you have to roll your own framework code to avoid ng-repeat, or ng-click, you have already lost.

Have I got the wrong end of the stick?

IE11, Windbg, JavaScript = joy

I was trying to debug a web application today, which is running in an Internet Explorer MSHTML window embedded in a thick client application. Weirdly, the app would fail, silently, without any script errors, on the sixth refresh — pressing F5 six times in a row would crash it each and every time.  Ctrl+F5 or F5, either would do it.

I was going slightly mad trying to trace this, with no obvious solutions. Finally I loaded the process up in Windbg, not expecting much, and found that three (handled) C++ exceptions were thrown for each of the first 6 loads (initial load, + 5 refreshes):

(958.17ac): C++ EH exception - code e06d7363 (first chance)
(958.17ac): C++ EH exception - code e06d7363 (first chance)
(958.17ac): C++ EH exception - code e06d7363 (first chance)

However, on the sixth refresh, this exception was happening four times:

(958.17ac): C++ EH exception - code e06d7363 (first chance)
(958.17ac): C++ EH exception - code e06d7363 (first chance)
(958.17ac): C++ EH exception - code e06d7363 (first chance)
(958.17ac): C++ EH exception - code e06d7363 (first chance)

This gave me something to look at! At the very least there was an observable difference between the refreshes within the debugger. A little more spelunking revealed that the very last exception thrown was the relevant one, and it returned the following trace (snipped):

(958.17ac): C++ EH exception - code e06d7363 (first chance)
ChildEBP RetAddr  
001875e8 75d3359c KERNELBASE!RaiseException+0x58
00187620 72ba1df2 msvcrt!_CxxThrowException+0x48
00187664 72d659cb jscript9!Js::JavascriptExceptionOperators::ThrowExceptionObjectInternal+0xb4
0018767c 72ba1cda jscript9!Js::JavascriptExceptionOperators::ThrowExceptionObject+0x15
001876a8 72bb5175 jscript9!Js::JavascriptExceptionOperators::Throw+0x6e
001876f4 6e9acd03 jscript9!CJavascriptOperations::ThrowException+0x9c
0018771c 6f3029f5 MSHTML!CFastDOM::ThrowDOMError+0x70
00187750 72b68e9d MSHTML!CFastDOM::CWebSocket::DefaultEntryPoint+0xe2
001877b8 72cff6a2 jscript9!Js::JavascriptExternalFunction::ExternalFunctionThunk+0x165
00187810 72bb849c jscript9!Js::JavascriptFunction::CallAsConstructor+0xc7
00187830 72bb8537 jscript9!Js::InterpreterStackFrame::NewScObject_Helper+0x39
0018784c 72bb858c jscript9!Js::InterpreterStackFrame::ProfiledNewScObject_Helper+0x53
0018786c 72bb855e jscript9!Js::InterpreterStackFrame::OP_NewScObject_Impl<Js::OpLayoutCallI_OneByte,1>+0x24
00187ba8 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x3c44
00187dbc 132f1ae1 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
WARNING: Frame IP not in any known module. Following frames may be wrong.
00187dc8 72c372dd 0x132f1ae1
00187ed0 138be456 jscript9!Js::JavascriptFunction::EntryApply+0x265
00187f38 72b6669e 0x138be456
00188288 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
001883bc 132f0681 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
001883c8 72b6669e 0x132f0681
00188718 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018887c 132f1b41 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
00188888 72b6669e 0x132f1b41
00188bd8 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
00188d1c 132f1c21 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
00188d28 72bb7642 0x132f1c21
00189080 72bd501f jscript9!Js::InterpreterStackFrame::Process+0x2175
001890b8 72bd507e jscript9!Js::InterpreterStackFrame::OP_TryCatch+0x49
001893f8 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x4e84
00189594 132f0081 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
00189624 72d0134c 0x132f0081
00189650 72bf3de5 jscript9!Js::JavascriptOperators::GetProperty_Internal<0>+0x48
00189678 72b677a7 jscript9!Js::InterpreterStackFrame::OP_ProfiledLoopBodyStart<0,1>+0xa2
001899b8 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x24f7
00189b34 132f0089 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
00189b40 72b6669e 0x132f0089
00189e98 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018a004 132f0099 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018a010 72b6669e 0x132f0099
0018a368 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018a4bc 132f1c69 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018a4c8 72b6669e 0x132f1c69
0018a818 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018a95c 132f1c71 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018a968 72b6669e 0x132f1c71
0018acb8 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018ade4 132f1fb1 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018adf0 72c372dd 0x132f1fb1
0018ae90 72b60685 jscript9!Js::JavascriptFunction::EntryApply+0x265
0018aee0 72bd4fcc jscript9!Js::JavascriptFunction::CallFunction<1>+0x88
0018b220 72bd501f jscript9!Js::InterpreterStackFrame::Process+0x442e
0018b258 72bd507e jscript9!Js::InterpreterStackFrame::OP_TryCatch+0x49
0018b598 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x4e84
0018b71c 132f1ed9 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018b728 72b6669e 0x132f1ed9
0018ba78 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018bbac 132f1ca1 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018bbb8 72bb7642 0x132f1ca1
0018bf00 72bd501f jscript9!Js::InterpreterStackFrame::Process+0x2175
0018bf38 72bd507e jscript9!Js::InterpreterStackFrame::OP_TryCatch+0x49
0018c278 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x4e84
0018c3ac 132f1e21 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018c3b8 72bb7642 0x132f1e21
0018c700 72bd501f jscript9!Js::InterpreterStackFrame::Process+0x2175
0018c738 72bd507e jscript9!Js::InterpreterStackFrame::OP_TryCatch+0x49
0018ca78 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x4e84
0018cbb4 132f1e21 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018cbc0 72b6669e 0x132f1e21
0018cf08 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018d03c 132f1e51 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018d048 72b6669e 0x132f1e51
0018d398 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018d4bc 132f0341 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018d4c8 72bb7642 0x132f0341
0018d810 72bd501f jscript9!Js::InterpreterStackFrame::Process+0x2175
0018d848 72bd507e jscript9!Js::InterpreterStackFrame::OP_TryCatch+0x49
0018db88 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x4e84
0018dd44 132f1f99 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018dd50 72b60685 0x132f1f99
0018dd98 72bd4fcc jscript9!Js::JavascriptFunction::CallFunction<1>+0x88
0018e0d8 72bd501f jscript9!Js::InterpreterStackFrame::Process+0x442e
0018e110 72bd507e jscript9!Js::InterpreterStackFrame::OP_TryCatch+0x49
0018e450 72c5bfce jscript9!Js::InterpreterStackFrame::Process+0x4e84
0018e488 72c5bf0f jscript9!Js::InterpreterStackFrame::OP_TryFinally+0xb1
0018e7c8 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x68eb
0018e8f4 132f0351 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018e900 72b6669e 0x132f0351
0018ec48 72b66548 jscript9!Js::InterpreterStackFrame::Process+0xbd7
0018ed94 132f1cf9 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018eda0 72b66ce9 0x132f1cf9
0018f0f8 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x1cd7
0018f264 132f1d01 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018f270 72b66ce9 0x132f1d01
0018f5c8 72b66548 jscript9!Js::InterpreterStackFrame::Process+0x1cd7
0018f70c 132f1d49 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
0018f718 72b60685 0x132f1d49
0018f760 72b6100e jscript9!Js::JavascriptFunction::CallFunction<1>+0x88
0018f7cc 72b60f60 jscript9!Js::JavascriptFunction::CallRootFunction+0x93
0018f814 72b60ee7 jscript9!ScriptSite::CallRootFunction+0x42
0018f83c 72b6993c jscript9!ScriptSite::Execute+0x6c
0018f898 72b69878 jscript9!ScriptEngineBase::ExecuteInternal<0>+0xbb
0018f8b0 6eaab78f jscript9!ScriptEngineBase::Execute+0x1c
0018f964 6eaab67c MSHTML!CListenerDispatch::InvokeVar+0x102
0018f990 6eaab21e MSHTML!CListenerDispatch::Invoke+0x61
0018fa28 6eaab385 MSHTML!CEventMgr::_InvokeListeners+0x1a2
0018fb98 6e8a98bb MSHTML!CEventMgr::Dispatch+0x35a
0018fbc0 6e92d0c0 MSHTML!CEventMgr::DispatchEvent+0x8c
0018fd18 6e92da30 MSHTML!CXMLHttpRequest::Fire_onreadystatechange+0x8b
0018fd2c 6e9343ea MSHTML!CXMLHttpRequest::DeferredFire_onreadystatechange+0x52
0018fd5c 6e8a9472 MSHTML!CXMLHttpRequest::DeferredFire_progressEvent+0x10
0018fdac 773f62fa MSHTML!GlobalWndProc+0x1cd
0018fdd8 773f6d3a USER32!InternalCallWinProc+0x23
0018fe50 773f77c4 USER32!UserCallWinProcCheckWow+0x109
0018feb0 773f788a USER32!DispatchMessageWorker+0x3bc

So now I knew that somewhere deep in my Javascript code there was something happening that was bad. Okay… I guess that helps, maybe?

But then, after some more googling, I discovered the following blog post, published a mere 9 days ago: Finally… JavaScript source line info in a dump. Magic! (In fact, I’m just writing this blog post for my future self, so I remember where to find this info in the future.)

After following the specific incantations outlined within that post, I was able to get the exact line of Javascript that was causing my weird error. All of a sudden, things made sense.

0:000> k
ChildEBP RetAddr  
00186520 76bd22b4 KERNELBASE!RaiseException+0x48
00186558 5900775d msvcrt!_CxxThrowException+0x59
00186588 590076a3 jscript9!Js::JavascriptExceptionOperators::ThrowExceptionObjectInternal+0xb7
001865b8 5901503d jscript9!Js::JavascriptExceptionOperators::Throw+0x77
00186604 6365e79b jscript9!CJavascriptOperations::ThrowException+0x9c
0018662c 63e609aa mshtml!CFastDOM::ThrowDOMError+0x70
00186660 58f1f9a3 mshtml!CFastDOM::CWebSocket::DefaultEntryPoint+0xe2
001866c8 58f29994 jscript9!Js::JavascriptExternalFunction::ExternalFunctionThunk+0x165
00186724 58f2988c jscript9!Js::JavascriptFunction::CallAsConstructor+0xc7
00186744 58f29a4a jscript9!Js::InterpreterStackFrame::NewScObject_Helper+0x39
00186760 58f29a7a jscript9!Js::InterpreterStackFrame::ProfiledNewScObject_Helper+0x53
00186780 58f29aa9 jscript9!Js::InterpreterStackFrame::OP_NewScObject_Impl<Js::OpLayoutCallI_OneByte,1>+0x24
00186b58 58f26510 jscript9!Js::InterpreterStackFrame::Process+0x402b
00186d6c 14ea1ae1 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
WARNING: Frame IP not in any known module. Following frames may be wrong.
00186d78 58faa407 js!Anonymous function [http://marcdev2010:4131/app/main/main-controller.js @ 251,7]
00186e70 15511455 jscript9!Js::JavascriptFunction::EntryApply+0x267
00186ed8 58f268e6 js!d [http://marcdev2010:4131/lib/angular/angular.min.js @ 34,479]
001872c8 58f26510 jscript9!Js::InterpreterStackFrame::Process+0x899
001873f4 14ea0681 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
00187400 58f268e6 js!instantiate [http://marcdev2010:4131/lib/angular/angular.min.js @ 35,101]
001877e8 58f26510 jscript9!Js::InterpreterStackFrame::Process+0x899
00187944 14ea1b41 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
00187950 58f268e6 js!Anonymous function [http://marcdev2010:4131/lib/angular/angular.min.js @ 67,280]
00187d38 58f26510 jscript9!Js::InterpreterStackFrame::Process+0x899
00187e74 14ea1c21 jscript9!Js::InterpreterStackFrame::InterpreterThunk<1>+0x1e8
...

When I looked at the line in question, line 251 of main-controller.js, I found:

// TODO: We need to handle the web socket connection going down -- it really should be wrappered
//  try {
      $scope.watcher = new WebSocket("ws://"+location.host+"/");
//  } catch(e) {
//    $scope.watcher = null;
//  }
    
    $scope.$on('$destroy', function() {
      if($scope.watcher) {
        $scope.watcher.close();
        $scope.watcher = null;
      }
    });

It turns out that if I press F5, or call browser.navigate from the container app, the $destroy event was never called, and what’s worse, MSHTML wasn’t shutting down the web socket, either.  And by default, Internet Explorer has a maximum of six concurrent WebSocket connections. So six page loads and we’re done. The connection error is not triggering a script error in the MSHTML host, sadly, which is why it was so hard to track down.

Yeah, if I had already written the error handling for the WebSocket connection in the JavaScript code, this issue wouldn’t have been so hard to trace. But at least I’ve learned a useful new debugging technique!

Speculation and further investigation:

  • I’m still working on why the web socket is staying open between page loads when embedded.  It’s not straightforward, and while onbeforeunload is a workaround that, well, works, I hate workarounds when I don’t have a clear answer as to why they are needed.
  • It is possible that there is a circular reference leak or similar, but I did kinda think MS had sorted most of those with recent releases of IE.
  • This problem is only reproducible within the MSHTML embedded browser, and not within Internet Explorer itself. Changing $scope.watcher to window.watcher made no difference.
  • TWebBrowser (Delphi) and TEmbeddedWB (Delphi) both exhibited the same symptoms.
  • A far simpler document with a web socket call did not cause the problem.

Updated a few minutes later:

Yes, I have found the cause. It’s a classic Internet Explorer leak: a function within a closure that references the DOM. Simplified, it looks something like:

  window.addEventListener( "load", function(event) {
    var aa = document.getElementById('aa');
    var watcher = new WebSocket("ws://"+location.host+"/");
    watcher.onmessage = function() {
      aa.innerHTML = 'message received';
    }
    aa.innerHTML = 'WebSocket started';
  }, false);

This will persist the connection between  page loads, and quickly eat up your available WebSocket connections.  But it still only happens in the embedded web browser. Why that is, I am still exploring.

Updated 30 Nov 2015:

The reason this happens only in the embedded web browser is that by default, the embedded web browser runs in an emulation mode for IE7. Even with the X-UA-Compatible meta tag, you still need to add your executable to the FEATURE_BROWSER_EMULATION registry key.

Extending $resource in AngularJS

I’ve recently dived into the brave new world (for me) of AngularJS, for a development project for a client. I always enjoy learning new tools and frameworks, especially when there are good design principles and practices that I can apply to both the new project and filter back into existing code.

In this project, we have an existing backend that is delivering data through a RESTful JSON interface. And this is what $resource was designed for. The front end is a HTML document embedded in an existing thick-client application window. Yes, this is the real world.

The data returned by $resource can be either a single item, or an array of items — a collection. $resource automatically wraps each item in the array with the “class” of the single item, which is nice. This makes it trivial to extend items with helper functions, such as, in my case, a time conversion function for a specific field in the JSON data (pseudocode):

angular.module('appServices').factory('Widget', ['$resource',
  function($resource) {
    var Widget= $resource('/data/widgets/:widgetId.json', {}, {
      query: {method:'GET', params:{widgetId:''}, isArray:true}
    });

    Widget.prototype.createTimeInMinutes = function() {
      var m = moment(this.createDateTime);
      return m.hours()*60 + m.minutes();
    };
    
...

However, finding a way to extend the collection was also of interest to me. For example, to add an itemById function which would return a single item from the array identified by a unique identifier field. This is of course me applying my existing object-oriented brain to a Angular (FWIW, this post was the best intro to Angular that I have found, even though it’s about coming from jQuery and not from an OO world).

It seemed nice to me to be able to write something like collection.itemById(), or item.createTimeInMinutes(), associating these functions with the data that they manipulate. Object orientation doing what it does best.  While I was aware of advice around the dangers of extending built-in object prototypes — monkey-patching, I really wasn’t sure that the same concerns applied to extending an ‘instance’ of Array.

There were several answers on Stack Overflow that related to this, in some way, and helped me think through the problem. I (and others) came up with several possible solutions, none of which were completely beautiful to me:

  1. Extend the array returned from $resource.  This is actually hard to do, but in theory possible with transformResponse. Unfortunately, because AngularJS does not preserve extensions to Array objects, you lose those extensions very easily. I won’t add the code here because it is ultimately unhelpful.
  2. Wrap the array in a helper object, when loading in the controller:
    Resource.query().$promise.then(function(collection) {
      $scope.collection = new CollectionWrapper(collection);
    });

    This worked, again, but added a layer of muck to every collection which was unpalatable to me, and pushed implementation into the controller, which just felt like the wrong plce.

  3. Add a helper object:
    var CollectionHelper = {
      itemById = function(collection, id) {
        ...
      }
    };
    
    ...
    
    var item = CollectionHelper.itemById(collection, id);

    Again, this didn’t feel clean, or quite right, although it worked well enough.

  4. Finally, James suggested using a filter.
    angular.module('myapp').filter('byId', function() {
        return function(collection, id) {
          ...
        }
      });
    
    ...
    
    var item = $filter('byId')(collection, id);
    // or you can go directly if injected:
    var item = byIdFilter(collection,id);
    // and within the template you can use:
    {{collection | byId:id }}
    

This last is certainly the most Angular way of doing it.  I’m still not 100% satisfied, because filters have global scope, which means that we need to give them ugly names like collectionDoWonk, instead of just doWonk.

Is this the best way to skin this cat?

Giro-Your-Strava updated to give Le Tour treatment to the climbs!

Update May 2014: As Strava’s ride pages have changed format significantly, Giro-Your-Strava no longer works. The good news is that Mesmeride does the same and more!

After the Strava API debacle, my little Tour Segment Gradient tool no longer works, which is sad.  I’d put together a number of other Strava API-based widgets, but this was the only one which was really at all popular.  Yesterday, DC Rainmaker himself mentioned (thank you Ray!) the Giro-Your-Strava elevation graph tool (which does still work) on his blog, so what better time to update the Tour Segment Gradient tool?

In short, what I have done is to dump both the Giro and Le Tour gradient mashups into the same bookmarklet.  One click and you get beautiful isometric graphs for your ride (in Giro style) and your efforts (in Le Tour style).  Yes, I get the inconsistency, but what would life be without idiosyncracies?

1. Install the bookmarklet.

Here’s the bookmarklet.  Just drag it to your Bookmarks toolbar to install it:

Giro-Your-Strava

2. Load a favourite Strava ride and click the bookmark.

(It’s best to wait for the page to load completely before clicking the bookmark.)

Presto, you’ll get two spiffy new buttons, one for your ride:

And one for the segment view:

So go ahead and click the Giro button, and you’ll see:

Click the Le Tour button for the new elevation profile for a segment:

Have fun!

One danger with bookmarklets that fiddle with an existing site in this way is that they will tend to break when the site updates.  There are no stability guarantees that APIs (ususally!) provide, so YMMV.  However, if the Strava site layout changes, it’s probably only a simple tweak to the code to get it working again.

There’s nothing beautiful about the code on the backend.  It really needs rewriting and modularisation etc etc etc but hey, it works 😉  Do what you want with the code, just share it with us all if you improve things!

Giro-style elevation graphs for Strava

Update May 2014: As Strava’s ride pages have changed format significantly, Giro-Your-Strava no longer works. The good news is that Mesmeride does the same and more!
Update 13 July 2013: An updated version of Giro-Your-Strava is now available here.

Just in time for the last few days of the Giro, I’ve finished a little after-hours Strava mashup project that builds on the segment graphs that I originally created for the Hobart 10,000.

I’m sure you’ve seen some of the Giro elevation graphs. Here’s one, from Stage 11:

Now, here’s a bookmarklet.  Drag it to your bookmarks toolbar, load up a favourite hilly ride on Strava, and click the bookmarklet.

Giro-Your-Strava

A mysterious new button will appear in the graph menu.  Go, on click it!

And up pops an elevation graph that makes it look like you’ve been riding the Giro!

The algorithm picks the categorised climbs from your ride (and tries to figure out the most appropriate segment where multiple segments finish at the top of a hill). Non-categorised segments are currently ignored.  The whole project is published on GitHub, so you can tweak it and improve it to your heart’s content.  Your first step should be to tidy up the mess I’ve left you 🙂

Plotting Pretty Elevation Profiles with the Strava API

Update 13 July 2013: After the deprecation of the Strava API, this tool no longer works. However, an updated version of this segment gradient tool is now available here.

So this tool plots elevation profiles from Strava data. OK, so maybe the profiles aren’t amazingly pretty.  But I had fun making them look somewhat like the elevation profiles from a certain famous cycling event!

Lipscome + Nicholas climb

This little hacked-together piece of JavaScript will plot the elevation from a Strava segment (in metric units only, of course!) and uses the familiar green-blue-red-black styling to represent the severity of the gradient.

I wrote the code to fulfill a specific purpose: generating graphs for the Hobart 10,000 ride.  But I figured I’d make the code and tool available for anyone to use or fiddle with as they see fit.

This tool requires IE9, Firefox, Safari, Chrome or any other canvas-aware browser.  If you plug in bad data, you’ll get bad results.  So don’t.  All the parameters dynamically refresh the profile, except for the segment ID field, after which you’ll need to click Load Segment.

If you want to play with the source yourself, the only thing you need to do server-side is plug in the data from: http://www.strava.com/api/v1/stream/segments/segmentid

Go knock yourself out here: http://hobart10000.com/segment-gradient.php

Updated 15 Aug 2012: The tool now does isometric projection, which I think looks quite a lot nicer, and I’ve tidied up the user interface and added a few more controls.  As noted by Jonathan in the comments, it doesn’t do too well with downhill segments — the tool assumes it is an uphill segment at present. 

Updates to kmwString functions

A couple of weeks ago I published on this blog a set of functions for handling supplementary characters in JavaScript.  We have since fixed a few bugs in the code and have now made the kmwString script available on github at https://github.com/tavultesoft/keyman-tools

This update:

  • Fixes an error which could occur when only 1 parameter was passed to kmwSubstring and that parameter was negative
  • Fixed an incorrect NaN return value for kmwCodePointToCodeUnit when passed an offset at the very end of a string
  • Adds kmwCodeUnitToCodePoint function
  • Adds kmwCharAt function — which is basically just kmwSubstr(index,1)

Again, suggestions, comments, fixes and flames all gratefully received.