Category Archives: Computing

Giro-Your-Strava now does Strava Routes as well

Update May 2014: As Strava’s ride pages have changed format significantly, Giro-Your-Strava no longer works. The good news is that Mesmeride does rides — but unfortunately not routes as yet.

After a comment from Mike on my previous Giro-Your-Strava post asking if the bookmarklet could support Strava’s new Routes feature, I took a few minutes over breakfast to spelunk and found it wouldn’t be too hard.

Firebug’s DOM explorer actually made the task much, much simpler. My approach was simply to look for Great Big Arrays Of Numbers. I soon found, amongst all the Google, jQuery, Modernizr, Optimizely and other objects, a pageView object, which contained a number of juicy functions, including pageView.chartContext().dataContext().elevationStream() and pageView.pageContext().routeSegments(), which gave me all the information I needed. The data structures for routes are somewhat different to those for rides, so I opted to massage the Great Big Arrays those functions returned into the same basic structure as the ride data already used by the bookmarklet, rather than touch the rendering code at all.

So… after that overly detailed introduction, here ’tis. I’ve updated the bookmarklet to draw Giro-style elevation profiles for Strava Routes as well as for Rides. And of course, Le Tour-style elevation profiles for segments still work, within individual ride pages.

Note: this still won’t work in IE10, as Github returns the wrong Content-Type for the Javascript and IE gets a little panicky about it, and, well, just use a different browser.

1. Install the bookmarklet.

Here’s the bookmarklet. Just drag it to your Bookmarks toolbar to install it:

Giro-Your-Strava

2. Load a favourite Strava Route and click the bookmark.

(It’s best to wait for the page to load completely before clicking the bookmark.)

Here’s one of my favourite Strava Routes — Hobart 10,000 Day 1:

H10K 2013 Day 1 - Strava Route 2013-09-05 08-26-54

After a few seconds the gradient graphic may refresh with the correct font — this takes a second or two to download.

That’s it! This new bookmarklet still works with rides (so delete the old one!)

The source is still all on GitHub.  Again, if you improve the code, or accidentally hurt yourself while reading it, please do share with a comment on this post.

And as DC Rainmaker says, thanks for reading!

I am happy to announce that the missing characters have been found!

Last year I published a blog post about characters missing from print jobs with Internet Explorer 9 and my adventures in tracing and reporting the issue to Microsoft.

We saw situations where a letter would be printed with random letters missing, as per the following example:

Here’s how it should have been printed (oh yes, that’s a completely fictional name we used for testing non-English Latin script letters in printing):

Today, I was notified that Microsoft have finally publicly published a hotfix!  We received the hotfix from them a few weeks ago, before it was publicly available, and it certainly solved the problem on our test machines and end user computers.

Download the hotfix from:

http://support.microsoft.com/kb/2853777 (Windows 7 SP1, Windows Server 2008 SP1) 

http://support.microsoft.com/kb/2855336 (Windows 8, Windows Server 2012)

(The first link notes that the hotfix is included in the rollup in the second link, though the second link doesn’t mention the hotfix!)

Sadly, they’ve only published a solution for Windows 7 Service Pack 1 onwards, and not for Vista, which could be an issue for some users. 

It was interesting to see that the issue was indeed a race condition (two threads ending up with the same random seed, because, and I quote:

This issue occurs because a conflict causes the text that uses the font to become corrupted when the two threads try to install the font at the same time. The name of the font is generated by the RAND function together with a SEEDvalue whose time value is set to srand(time(NULL)). When the two documents are printed at the same time, the SEEDvalue for the font is the same in both documents. Therefore, the conflict occurs.

So not related to the threading model, which was a little bit of misdirection on my part 😉  That’s par for the course though when debugging complex issues without source…

Fantastic to finally have the fix 🙂

Giro-Your-Strava updated to give Le Tour treatment to the climbs!

Update May 2014: As Strava’s ride pages have changed format significantly, Giro-Your-Strava no longer works. The good news is that Mesmeride does the same and more!

After the Strava API debacle, my little Tour Segment Gradient tool no longer works, which is sad.  I’d put together a number of other Strava API-based widgets, but this was the only one which was really at all popular.  Yesterday, DC Rainmaker himself mentioned (thank you Ray!) the Giro-Your-Strava elevation graph tool (which does still work) on his blog, so what better time to update the Tour Segment Gradient tool?

In short, what I have done is to dump both the Giro and Le Tour gradient mashups into the same bookmarklet.  One click and you get beautiful isometric graphs for your ride (in Giro style) and your efforts (in Le Tour style).  Yes, I get the inconsistency, but what would life be without idiosyncracies?

1. Install the bookmarklet.

Here’s the bookmarklet.  Just drag it to your Bookmarks toolbar to install it:

Giro-Your-Strava

2. Load a favourite Strava ride and click the bookmark.

(It’s best to wait for the page to load completely before clicking the bookmark.)

Presto, you’ll get two spiffy new buttons, one for your ride:

And one for the segment view:

So go ahead and click the Giro button, and you’ll see:

Click the Le Tour button for the new elevation profile for a segment:

Have fun!

One danger with bookmarklets that fiddle with an existing site in this way is that they will tend to break when the site updates.  There are no stability guarantees that APIs (ususally!) provide, so YMMV.  However, if the Strava site layout changes, it’s probably only a simple tweak to the code to get it working again.

There’s nothing beautiful about the code on the backend.  It really needs rewriting and modularisation etc etc etc but hey, it works 😉  Do what you want with the code, just share it with us all if you improve things!

Giro-style elevation graphs for Strava

Update May 2014: As Strava’s ride pages have changed format significantly, Giro-Your-Strava no longer works. The good news is that Mesmeride does the same and more!
Update 13 July 2013: An updated version of Giro-Your-Strava is now available here.

Just in time for the last few days of the Giro, I’ve finished a little after-hours Strava mashup project that builds on the segment graphs that I originally created for the Hobart 10,000.

I’m sure you’ve seen some of the Giro elevation graphs. Here’s one, from Stage 11:

Now, here’s a bookmarklet.  Drag it to your bookmarks toolbar, load up a favourite hilly ride on Strava, and click the bookmarklet.

Giro-Your-Strava

A mysterious new button will appear in the graph menu.  Go, on click it!

And up pops an elevation graph that makes it look like you’ve been riding the Giro!

The algorithm picks the categorised climbs from your ride (and tries to figure out the most appropriate segment where multiple segments finish at the top of a hill). Non-categorised segments are currently ignored.  The whole project is published on GitHub, so you can tweak it and improve it to your heart’s content.  Your first step should be to tidy up the mess I’ve left you 🙂

RSA Key Exchange between CryptoAPI and CNG (BCrypt)

Microsoft, a few years ago, wrote a new cryptography API for Windows Vista called Cryptography Next Generation (CNG).  This replaces the existing cryptography API in Windows XP and earlier versions which was simply called CryptoAPI.

Now, compatibility between the two APIs is touted as a feature of the new CNG API, but a somewhat more sombre reality tends to set in pretty quickly once you start working with them.  A motley collection of loosely documented data structures, endianness differences, and “Not Supported” footnotes pile up as quickly as you can open MSDN documentation tabs in your browser.  In my research, I also found a dearth of concrete examples of how to interoperate between the two APIs online.  In fact, there was only one page which really helped, buried in Microsoft’s Output Protection Manager documentation.

There is something magical about all your bits lining up in a row when mixing crypto libraries of any ilk, and I wanted to share!  Hence this story.

My requirement was pretty simple: I wanted to generate an RSA key pair using CryptoAPI in my client application, hand off the public key to another application, which happens to use CNG, wave my hands a little, and magically transact a key exchange.  Here’s how to do it (with some error handling elided for your own reading sanity).  You should be able to load this into any recent version of Visual Studio, add bcrypt.lib to your library includes and build.  The code should be reasonably self-explanatory, so I’ll leave it there.

The usual disclaimers apply. Source on GitHub.

RIGHTeously tripping over T-SQL’s LEN function

We tripped over recently on our understanding of Microsoft SQL Server’s T-SQL LEN function.  The following conversation encapsulates in a nutshell what any sensible developer would assume about the function.

@marcdurdin I guess the answer is, removing the line would give the same result. Right? 🙂

— S Krupa Shankar (@tamil) April 23, 2013

Now, I wouldn’t be writing this blog post if that was the whole story.  Because, like so many of these things, it’s not quite that simple.

Originally, we were using RIGHT(string, LEN(string) – 2) to trim two characters off the front of our string.  Or something like that.  Perfectly legitimate and reasonable, one would think.  But we were getting strange results, trimming more characters than we expected.

It turns out that T-SQL’s LEN function does not return the length of the string.  It returns the length of the string, excluding trailing blanks.  That is, excluding U+0020, 0x20, 32, ‘ ‘, or whatever you want to call it.  But not tabs, new lines, zero width spaces, breaking or otherwise, or any other Unicode space category characters.  Just that one character.  This no doubt comes from the history of the CHAR(n) type, where fields were always padded out to n characters with spaces.  But today, this is not a helpful feature.

Of course, RIGHT(string) does not ignore trailing blanks

But here’s where it gets really fun.  Because a post about strings is never complete until you’ve done it in Unicode.  Some pundits suggest using DATALENGTH to get the actual length of the string.  This returns the length in bytes, not characters (remember that NCHAR is UTF-16, so 2 bytes per character… sorta!).  Starting with SQL Server 2012, UTF-16 supplementary pairs can be treated as a single character, with the _SC collations, so you can’t even use DATALENGTH*2 to get the real length of the string!

OK.  So how do you get the length of a string, blanks included, now that we’ve established that we can’t use DATALENGTH?  Here’s one simple way:

  SET @realLength = LEN(@str + ‘.’) – 1

Just to really do your head in, let’s see what happens when you use LEN with a bunch of garden variety SQL strings.  I should warn you that this is a display artefact involving the decidedly unrecommended use of NUL characters, and no more, but the weird side effects are too fun to ignore.

First, here’s a set of SQL queries for our default collation (Latin1_General_CI_AS):

Note the output.  Not exactly intuitive!  Now we run the Unicode version:

Not quite as many gotchas in that one?  Or are there?  Notice how the first line shows a wide space for the three NUL characters — but not quite as wide as the Ideographic space…

Now here’s where things go really weird.  Running in Microsoft’s SQL Server Management Studio, I switch back to the first window, and, I must emphasise, without running any queries, or making any changes to settings, take a look at how the Messages tab appears now!

That’s right, the first line in the messages has magically changed!  The only thing I can surmise is that switching one window into Unicode output has affected the whole program’s treatment of NUL characters.  Let that be a warning to you (and I haven’t even mentioned consuming said NULs in C programs).

The Fragile Abstract Factory Delphi Pattern

When refactoring a large monolithic executable written in Delphi into several executables, we ran into an unanticipated issue with what I am calling the fragile abstract factory (anti) pattern, although the name is possibly not a perfect fit.  To get started, have a look at the following small program that illustrates the issue.

program FragileFactory;

uses
  MyClass in 'MyClass.pas',
  GreenClass in 'GreenClass.pas',
  //RedClass in 'RedClass.pas',
  SomeProgram in 'SomeProgram.pas';

var
  C: TMyClass;
begin
  C := TMyClass.GetObject('TGreenClass');
  writeln(C.ToString);
  C.Free;

  C := TMyClass.GetObject('TRedClass');  // oh dear… that’s going to fail
  writeln(C.ToString);
  C.Free;
end.
unit MyClass;

interface

type
  TMyClass = class
  protected
    class procedure Register;
  public
    class function GetObject(ClassName: string): TMyClass;
  end;

implementation

uses
  System.Contnrs,
  System.SysUtils;

var
  FMyClasses: TClassList = nil;

{ TMyObjectBase }

class procedure TMyClass.Register;
begin
  if not Assigned(FMyClasses) then
    FMyClasses := TClassList.Create;
  FMyClasses.Add(Self);
end;

class function TMyClass.GetObject(ClassName: string): TMyClass;
var
  i: Integer;
begin
  for i := 0 to FMyClasses.Count – 1 do
    if FMyClasses[i].ClassNameIs(ClassName) then
    begin
      Result := FMyClasses[i].Create;
      Exit;
    end;

  Result := nil;
end;

initialization
finalization
  FreeAndNil(FMyClasses);
end.
unit GreenClass;

interface

uses
  MyClass;

type
  TGreenClass = class(TMyClass)
  public
    function ToString: string; override;
  end;

implementation

{ TGreenClass }

function TGreenClass.ToString: string;
begin
  Result := 'I am Green';
end;

initialization
  TGreenClass.Register;
end.

What happens when we run this?

C:\src\fragilefactory>Win32\Debug\FragileFactory.exe
I am Green
Exception EAccessViolation in module FragileFactory.exe at 000495A4.
Access violation at address 004495A4 in module ‘FragileFactory.exe’. Read of address 00000000.

Note the missing TRedClass in the source.  We don’t discover until runtime that this class is missing.  In a project of this scale, it is pretty obvious that we haven’t linked in the relevant unit, but once you get a large project (think hundreds or even thousands of units), it simply isn’t possible to manually validate that the classes you need are going to be there.

There are two problems with this fragile factory design pattern:

  1. Delphi use clauses are fragile (uh, hence the name of the pattern).  The development environment frequently updates them, typically they are not organised, sorted, or even formatted neatly.  This makes validating changes to them difficult and error-prone.  Merging changes in version control systems is a frequent cause of errors.
  2. When starting a new Delphi project that utilises your class libraries, ensuring you use all the required units is a hard problem to solve.

Typically this pattern will be used with in a couple of ways, somewhat less naively than the example above:

  1. There will be an external source for the identifiers.  In the project in question, these class names were retrieved from a database, or from linked resources.
  2. The registered classes will be iterated over and each called in turn to perform some function.

Of course, this is not a problem restricted to Delphi or even this pattern.  Within Delphi, any unit that does work in its initialization section is prone to this problem.  More broadly, any dynamically linking registry, such as COM, will have similar problems.  The big gotcha with Delphi is that the problem can only be resolved by rebuilding the project, which necessitates rollout of an updated executable — much harder than just re-registering a COM object on a client site for example.

How then do we solve this problem?  Well, I have not identified a simple, good clean fix.  If you have one, please tell me!  But here are a few things that can help.

  1. Where possible, use a reference to the class itself, such as by calling the class’s ClassName function, to enforce linking the identified class in.  For example:
    C := TMyClass.GetObject(TGreenClass.ClassName);
    C := TMyClass.GetObject(TRedClass.ClassName);
  2. When the identifiers are pulled from an external resource, such as a database, you have no static reference to the class.  In this case, consider building a registry of units, automatically generated during your build if possible.  For example, we automatically generate a registry during build that looks like this:
    unit MyClassRegistry;
    
    // Do not modify; automatically generated 
    
    initialization
    procedure AssertMyClassRegistry;
    
    implementation
    
    uses
      GreenClass,
      RedClass;
    
    procedure AssertMyClassRegistry;
    begin
      Assert(TGreenClass.InheritsFrom(TMyClass));
      Assert(TRedClass.InheritsFrom(TMyClass));
    end; 
    
    end.

    These are not assertions that are likely to fail but they do serve to ensure that the classes are linked in.  The AssertMyClassRegistry function is called in the constructor of our main form, which is safer than relying on a use clause to link it in.

  3. Units that can cause this problem can be identified by searching for units with initialization sections in your project (don’t forget units that also use the old style begin instead of initialization — a helpful grep regular expression for finding these units is (?s)begin(?:.(?!end;))+\bend\.).  This at least gives you a starting point for making sure you test every possible case.  Static analysis tools are very helpful here.
  4. Even though it goes against every encapsulation design principle I’ve ever learned, referencing the subclass units in the base class unit is perhaps a sensible solution with a minimal cost.  We’ve used this approach in a number of places.
  5. Format your use clauses, even sorting them if possible, with a single unit reference on each line.  This is a massive boon for source control.  We went as far as building a tool to clean our use clauses, and found that this helped greatly.

In summary, then, the fragile abstract factory (anti) pattern is just something to be aware of when working on large Delphi projects, mainly because it is hard to test for: the absence of a unit only comes to light when you actually call upon that unit, and due to the fragility of Delphi’s use clauses, unrelated changes are likely to trigger the issue.

Cursing over cursor keys

A follow-up post, 29 Sep 2014: Cursor Keys Have Not Improved.

As you may know, I do a fair bit of work with keyboards.  This means I have collected quite a number of hardware keyboards over the last few years.  I do a lot of testing with various keyboards and use a number of computers with different hardware keyboards for both testing and development purposes.

One of the more irritating issues I experience moving between these keyboards is in the way manufacturers keep rearranging the cursor key block.  As a touch typist, I am continually pressing the wrong cursor key, or Delete instead of Home, and so on.  And that’s just the desktop keyboards.  When it comes to notebooks I have lost all hope.

The ISO/IEC 9995-5 standard does require keyboard manufacturers to keep those cursor keys in one of two shapes:

That’s it.  Not very prescriptive.  One almost wonders why they bothered!  So let’s have a look at the problem with the keyboards in my office.  Starting with desktop keyboards, I found 12 keyboards on and around my desk (and one in a second office), with no less than 6 different arrangements of this cursed cursor zone. I have included 8 of the keyboards in the image below:

Eight keyboards.  Can you spot the ones with identical cursor layouts?

I drew up the six different layouts I found, grouping the keys by colour (sorry if you are colour blind).  The top two layouts seem to be where most keyboards are going today.  The middle two are more traditional, and the final two layouts were invented, I am convinced, purely to cause me grief.

I aligned the layout diagrams with the right hand side of the main alphanumeric keyboard block: this reflects where my hands sit normally.  I hardly need to explain the difficulties with the constant rearrangement of the Home, End, PgUp, PgDn, Ins and Del keys, but I will note that the inconsistency of the cursor block position is almost as much of a problem: I get used to the position of the down arrow key on one keyboard, switch to another keyboard and find myself hitting left arrow instead from muscle memory.

Notebook manufacturers have somewhat more of an excuse: they are trying to fit a bunch of keys into a much smaller space.  Even so, the variety is pretty amazing.  I didn’t even include the couple of dead laptops that have not yet made their way into the rubbish.  Apple and Acer take top marks for consistency.  Toshiba, not so much…  I threw in a Microsoft Surface RT tablet keyboard for good measure!  Some of the keyboards use the Fn key to access PgUp, PgDn, Home or End (or even Del).  These are of course very inconsistently mapped, if you can even find them…

Surface RT, MacBook, Acer netbook, Acer ultrabook,
Toshiba notebook, MacBook, Toshiba notebook

Even soft keyboards suffer.  The following 5 images are all from the same company, Microsoft.  Of note is that the position of the Fn key changes between Windows 7 and Windows 8. In 3 of the images, the position of the up arrow key is just slightly misaligned with the down arrow; this may not be a problem in use but it is visually irritating!

Windows 8 Accessibility, Windows 7 Accessibility,
Windows XP, Win 8 Touch Optimised, Win 8 Touch Full.

 

How not to re-raise an exception in Delphi

Just a quick exception management tip for today.

I was debugging a weird cascade of exceptions in an application today — it started with an EOleException from a database connection issue, and rapidly degenerated into a series of EAccessViolation and EInvalidPointer exceptions: often a good sign of Use-After-Free or Double-Free scenarios.  Problem was, I could not see any place where we could be using a object after freeing it, even in the error case.

Here’s a shortened version of the code:

procedure ConnectToDatabase;
begin
  try
    CauseADatabaseException;  // yeah... bear with me here.
  except
    on E: EDatabaseError do
    begin
      E.Message := 'Failed to connect to database; the '+ 
                   'error message was: ' + E.Message;
      raise(E);
    end;
  end;
end;

Can you spot the bug?

I must admit I read through the code quite a few times before I spotted it.  It’s not an in-your-face-look-at-me type of bug!  In fact, the line in question appears to be completely logical and plausible.

Okay, enough waffle.  The bug is in the last line of the exception handler:

      raise(E);

When we call raise(E) we are telling Delphi that here is a shiny brand new exception object that we want to raise.  After Delphi raises the exception object, the original exception object is freed, and … wait a minute, that’s the exception object we were raising!  One delicious serve of Use-After-Free or Double-Free coming right up!

We should be doing this instead:

      raise;  // don't reference E here!

Another thing to take away from this is: remember that you don’t control the lifetime of the exception object.  Don’t store references to it and expect it to survive.  If you want to maintain knowledge of an inner exception object, use Delphi’s own nested exception management, available since Delphi 2009.

The Southern Ocean… According to Apple

Just like everyone else, I found Apple’s new train wreck release of iOS 6 maps bizarre. It’s rather unlike Apple to publish such an unpolished feature. The straight-line vector graphics look amateur when zoomed in and generally a step back from the Google maps. But worse than the lack of polish is of course the outright data errors. I can understand a misplaced town in an insignificant country such as the United Kingdom. But to misplace a whole ocean? I mean, it’s not like there’s that many of them to keep track of.

Here’s the evidence. You’ll also note the comparatively minor issue of a missing river. Minor compared to an ocean, anyway (on Google Maps):




However, before you hanker for the good old Google Maps, let’s take a quick look at the same location:

Looks good, river staying right where it belongs like a good little river, and the Southern Ocean has decided to skip town.  But there’s something different about that Derwent Park Rd: it seems to be a major through-road on Google Maps but is a dead end on Apple’s map:

Google should know it’s not a major through road.  Their Street View car even found the evidence. It’s a private road.  With a gate.

What’s the moral of the story?  Use Bing Maps?  Nope.  They include our little dried-up ocean, fortunately nameless, but also perpetuate that little issue with the closed road (and yes, both Bing and Google will happily direct you to crash through the gate, if you ask them for directions).

Still, I think Apple’s maps take home the prize!