Debugging a stalled Delphi process with Windbg and memory searches

October 30, 2013Bugs, Computing, Delphi, Development, Tools, WindowsMarc Durdin

Today I’ve got a process on my machine that is supposed to be exiting, but it has hung. Let’s load it up in Windbg and find what’s up. The program in question was built in Delphi XE2, and symbols were generated by our internal tds2dbg tool (but there are other tools online which create similar .dbg files). As usual, I am writing this up for my own benefit as much as anyone else’s, but if I put it on my blog, it forces me to put in enough detail that even I can understand it when I come back to it!

Looking at the main thread, we can see unit finalizations are currently being called, but the specific unit finalization section and functions which are being called are not immediately visible in the call stack, between InterlockedCompareExchange and FinalizeUnits:

0:000> kb
ChildEBP RetAddr  Args to Child              
0018ff3c 0040908c 0018ff78 0040909a 0018ff5c audit4_patient!System.Sysutils.InterlockedCompareExchange+0x5 [C:\Program Files (x86)\Embarcadero\RAD Studio\9.0\source\rtl\sys\InterlockedAPIs.inc @ 23]
0018ff5c 004094b2 0018ff88 00000000 00000000 audit4_patient!System.FinalizeUnits+0x40 [C:\Program Files (x86)\Embarcadero\RAD Studio\9.0\source\rtl\sys\System.pas @ 17473]
0018ff88 74b7336a 7efde000 0018ffd4 76f99f72 audit4_patient!System..Halt0+0xa2 [C:\Program Files (x86)\Embarcadero\RAD Studio\9.0\source\rtl\sys\System.pas @ 18599]
0018ff94 76f99f72 7efde000 35648d3c 00000000 kernel32!BaseThreadInitThunk+0xe
0018ffd4 76f99f45 01a5216c 7efde000 00000000 ntdll!__RtlUserThreadStart+0x70
0018ffec 00000000 01a5216c 7efde000 00000000 ntdll!_RtlUserThreadStart+0x1b

So, the simplest way to find out where we were was to step out of the InterlockedCompareExchange call. I found myself in System.SysUtils.DoneMonitorSupport (specifically, the CleanEventList subprocedure):

0:000> p
eax=01a8ee70 ebx=01a8ee70 ecx=01a8ee70 edx=00000001 esi=00000020 edi=01a26e80
eip=0042dcb1 esp=0018ff20 ebp=0018ff3c iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00200202
audit4_patient!CleanEventList+0xd:
0042dcb1 33c9            xor     ecx,ecx

After a little more spelunking, and a review of the Delphi source around this function, I found that this was a part of the System.TMonitor support. Specifically, there was a locked TMonitor somewhere that had not been destroyed. I stepped through a loop that was spinning, waiting for the object to be unlocked so its handle could be destroyed, and found a reference to the data in question here:

0:000> p
eax=00000001 ebx=01a8ee70 ecx=01a8ee70 edx=00000001 esi=00000020 edi=01a26e80
eip=0042dcaf esp=0018ff20 ebp=0018ff3c iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00200202
audit4_patient!CleanEventList+0xb:
0042dcaf 8bc3            mov     eax,ebx

Looking at the record pointed to by ebx, we had a reference to an event handle handy:

0:000> dd ebx L2
01a8ee70  00000001 00000928

  TSyncEventItem = record
    Lock: Integer;
    Event: Pointer;
  end;

Although Event is a Pointer, internally it’s just cast from an event handle. So I guess that we can probably find another reference to that handle somewhere in memory, corresponding to a TMonitor record:

  TMonitor = record
  strict private
    // ... snip ...
    var
      FLockCount: Integer;
      FRecursionCount: Integer;
      FOwningThread: TThreadID;
      FLockEvent: Pointer;
      FSpinCount: Integer;
      FWaitQueue: PWaitingThread;
      FQueueLock: TSpinLock;

And if we search for that event handle:

0:000> s -[w]d 00400000 L?F000000 00000928
01a8ee74  00000928 00000000 0000092c 00000000  (.......,.......
07649334  00000928 002c0127 0012ccb6 0000002e  (...'.,.........
0764aa14  00000928 01200125 004b8472 000037ce  (...%. .r.K..7..
07651e24  00000928 05f60125 01101abc 00000e86  (...%...........
08a47544  00000928 00000000 00000000 00000000  (...............

Now one of these should correspond to a TMonitor record. The first entry (01a8ee74) is just part of our TSyncEventItem record, and the next three don’t make sense given that the FSpinCount (the next value in the memory dump) would be invalid. So let’s look at the last one. Counting quickly on all my fingers and toes, I establish that that makes 08a47538 the start of the TMonitor record. And… so we search for a pointer to that.

0:000> s -[w]d 00400000 L?F5687000 08a47538
076b1d24  08a47538 08aa3e40 076b1db1 0122fe50  8u..@>....k.P.".

Just one! But here it gets a little tricky, because the PMonitor pointer is in a ‘hidden’ field at the end of the object. So we need to locate the start of the object.

0:000> dd 076b1d00
076b1d00  0122fe50 00000000 00000000 076b1df1
076b1d10  0122fe50 00000000 00000000 076b0ce0
076b1d20  004015c8 08a47538 08aa3e40 076b1db1
076b1d30  0122fe50 00000000 00000000 076b1d51
... snip ...

I’m just stabbing in the dark here, but that 004015c8 that’s just four bytes back smells suspiciously like an object class pointer. Let’s see:

0:000> da poi(4015c8-38)+1
004016d7  "TObject&"

Ta da! That all fits. A TObject has no data members, so the next 4 bytes should be the TMonitor (search for hfMonitorOffset in the Delphi source to learn more). So we have a TObject being used as a TMonitor lock reference. (Learn about that poi(address-38)+1 magic). But what other naughty object is hanging about, using this TObject as its lock?

0:000> s -[w]d 00400000 L?F5687000 076b1d20
098194b0  076b1d20 00000000 00000000 09819449   .k.........I...

Just one. Let’s trawl back in memory just a little bit and have a look at this one.

0:000> dd 09819480  
09819480  00f0f0f0 00ffffff 00000000 09819641
09819490  00447b7c 00000000 00000000 00000000
098194a0  00000000 098150c0 00448338 098195c8
098194b0  076b1d20 00000000 00000000 09819449
... snip ...

0:000> da poi(00448338-38)+1
004483c0  "TThreadList&"

And what does a TThreadList look like?

  TThreadList = class
  private
    FList: TList;
    FLock: TObject;
    FDuplicates: TDuplicates;

Yes, that definitely looks hopeful! That FLock is pointing to our lock TObject… I believe that’s called a Quality Match.

This is still a little bit too generic for me, though. TThreadList is a standard Delphi class used by the bucketload. Let’s try and identify who is using this list and leaving it lying about. First, we’ll quickly have a look at that TThreadList.FList to see if it has anything of interest — that’s the first data member in the object == object+4.

0:000> dd poi(098194ac)
098195c8  00447b7c 00000000 00000000 00000000
... snip ...
0:000> da poi(447b7c-38)+1
00447cad  "TList'"

Yep, it’s a TList. Just making sure. It’s empty, what a shame (TList.FCount is the second data member in the object == 00000000, as is the list pointer itself).

So how else can we find the usage of that TThreadList? Is it TThreadList referenced anywhere then? Break out the search tool again!

0:000> s -[w]d 00400000 L?F5687000 098194a8
076d3410  098194a8 00000000 09819580 00000000  ................

Yes. Just once, again. Again, we scroll back in memory to find the base of that object.

0:000> dd 076d33c0  
076d33c0  08abfe60 00000000 00000000 076d4c39
076d33d0  01616964 0000091c 00001850 00000100
076d33e0  00000001 00000000 00000000 00000000
076d33f0  076d33d0 00000000 01616d38 076d33d0
076d3400  00000000 00000000 00000000 00000000
076d3410  098194a8 00000000 09819580 00000000
076d3420  00000000 076d2ea9 0075e518 00000000
076d3430  00401ecc 00000000 00000000 00000000

There was a false positive at 076d33f8, but then magic happened at 076d33d0:

0:000> da poi(01616964-38)+1
016169e8  "TAnatomyDiagramTileLoaderThread&"

Wow! Something real! Let’s dissect this a bit. Looking at the definition of TThread, we have the following data:

076d33d0  01616964  class pointer TAnatomyDiagramTileLoaderThread
076d33d4  0000091c  TThread.FHandle
076d33d8  00001850  TThread.FThreadID
076d33dc  00000100  TThread.FCreateSuspended, .FTerminated, .FSuspended, .FFreeOnTerminate (watch that endianness!)
076d33e0  00000001  TThread.FFinished
076d33e4  00000000  TThread.FReturnValue
076d33e8  00000000  TThread.FOnTerminate.Code
076d33ec  00000000  TThread.FOnTerminate.Data
076d33f0  076d33d0  TThread.FSynchronize.TThread (= Self)
076d33f4  00000000  padding
076d33f8  01616d38  TThread.FSynchronize.FMethod.Code (= last Synchronize target method)
076d33fc  076d33d0  TThread.FSynchronize.FMethod.Data (= Self)
076d3400  00000000  TThread.FSynchronize.FProcedure
076d3404  00000000  TThread.FSynchronize.FProcedure
076d3408  00000000  TThread.FFatalException
076d340c  00000000  TThread.FExternalThread

Then, into TAnatomyDiagramTileLoaderThread:

076d3410  098194a8  TAnatomyDiagramTileLoaderThread.FTiles: TThreadList

So… we can tell that the thread has exited (FFinished == 1), and verify that by looking at running threads, looking for thread id 1850:

0:000> ~
.  0  Id: 1c98.1408 Suspend: 1 Teb: 7efdd000 Unfrozen
   1  Id: 1c98.16c8 Suspend: 1 Teb: 7efda000 Unfrozen
   2  Id: 1c98.11a0 Suspend: 1 Teb: 7ee1c000 Unfrozen
   3  Id: 1c98.1bbc Suspend: 1 Teb: 7ee19000 Unfrozen
   4  Id: 1c98.10dc Suspend: 1 Teb: 7ee04000 Unfrozen
   5  Id: 1c98.12f0 Suspend: 1 Teb: 7edfb000 Unfrozen
   6  Id: 1c98.1d38 Suspend: 1 Teb: 7efaf000 Unfrozen
   7  Id: 1c98.1770 Suspend: 1 Teb: 7edec000 Unfrozen
   8  Id: 1c98.1044 Suspend: 1 Teb: 7ede3000 Unfrozen
   9  Id: 1c98.bf4 Suspend: 1 Teb: 7ede0000 Unfrozen
  10  Id: 1c98.b3c Suspend: 1 Teb: 7efd7000 Unfrozen

Furthermore, the handle is invalid:

0:000> !handle 91c
Could not duplicate handle 91c, error 6

That suggests that the object has already been destroyed. But that the TThreadList hasn’t.

And sure enough, when I looked at the destructor for TAnatomyDiagramTileLoadThread, we clear the TThreadList, but we never free it!

Now, another way we could have caught this was to turn on leak detection. But leak detection is not always perfect, especially when you get some libraries that *cough* have a lot of false positives. And of course while we could have switched on heap leak detection, that involves rebuilding and restarting the process and losing the context along the way, with no guarantee we’ll be able to reproduce it again!

While this approach does feel a little tedious, and we did have some luck in this instance with freed objects not being overwritten, and the values we were searching for being relatively unique, it does nevertheless feel pretty deterministic, which must be better than the old “try-it-and-hope” debugging technique.

Introducing Mesmeride

October 26, 2013Computing, Cycling, Development, mashups, StravaMarc Durdin

So I recently had some holidays. Weird, I know. I took two whole weeks off and only had to go into the office twice during that time. My first week had unseasonably nice weather, so I spent some time on my bike making the most of it.

In the second week, the weather soured, so I took the opportunity to learn something of Ruby on Rails with the great Rails tutorial. I am not generally a big fan of tutorials but this particular one covered a lot of bases, and was well organised. Equally excellent were Railscasts.

OH: ruby is for people who think their programming language should make them happy. Python is for ppl who don't understand what that means

— Joel Spolsky (@spolsky) October 20, 2013

After working through the first few chapters of the tutorial, I was comfortable enough to start on my own project to test my newly acquired knowledge.

Enter Mesmeride. With this project, I had two objectives:

Get a functional and “useful” Ruby on Rails site live in a week.
Get my Strava gradient rendering code running again with the new v3 Strava API.

Mesmeride allows you to take any Strava activity or segment, and graph it out in a number of different styles. You can add waypoints and control the length, height and size of the presentation, making it suitable for print or web. After tweaking the style of the graph to perfection, you can share the result on Twitter or Facebook, embed the image on your blog, or save it for printing or offline sharing.

Waypoints

Any ride of a reasonable length will have points of interest. The Giro renderer will draw these onto the profile. You can add and delete waypoints, move them along the ride, and change their names in the left hand box in the controls section.

Mountains or Molehills?

The most popular or remarked-upon feature is the ability to make any of your rides, even the most flat and featureless, look like a day attacking the biggest climbs of the Alps. You can control the mountainosity of your ride with the Netherlands-Switzerlands slider (also called the Molehills-Mountain slider).

Size and Length

To help you adjust the dimensions of the graphic, for print or for web, you can rescale the entire ride graphic with the “Teensy – Ginormous slider”, or make the ride appear longer or shorter with the “Shopping Trip – Grand Tour” slider.

Sharing

What good is a graphic without eyes to look at it? Mesmeride has tools to share any of the graphics you create on Twitter, Facebook or even by embedding them in your blog. Or of course you can save the image and download it. The images are stored on Amazon S3, and you can save up to 3 for any given route.

I even drew the logo myself. Can you tell?

Mesmeride will save the design you create as well, and you can come back later and change it round into many other styles.

In the future I may add mapping, additional gradient styles, and more controls and waypoint types to existing styles.

Here are a few examples from my race last weekend, via Strava. No, I didn’t do well, but never mind 😉 The screenshots above show the editor in action; what you see below are the resulting files. I even fixed a bug in Mesmeride when preparing this…

Hell of the South, full route profile, with the Mesmeride “Giro” Renderer. The waypoints are fully customisable!

Hell of the South Climb 1 — The Gardiner’s Bay Climb at the start of Hell of the South. Presented with the Mesmeride “Le Tour” segment renderer

Hell of the South Kettering Climb — The climb out of Kettering, presented in the “Le Tour” rendering style. This is the climb I came unstuck on…

Nicholl's Rivulet Climb — The Nicholl’s Rivulet Climb, a lovely, smooth winding climb which I suffered greatly on. Off the back… 🙂

To finish with, the whole ride again, in another style.

HotS "Hobart 10,000 Banner" Style — HotS “Hobart 10,000 Banner” Style

Almost Caving, and Tourist Caving

October 11, 2013Family, Family|Bible|More, UncategorizedMarc Durdin

The forecast was bleak — 10°C and rain showers. So we abandoned our original plans and decided to go bush instead. Last night, Hannah prepared an itinerary, including Mystery Creek Cave, Thermal Springs, Hastings Cave and Surprise Destination (at my suggestion). This morning, we executed that carefully shaped itinerary, nearly perfectly. Apart from wet clothes all day. That was a bit of a dampener.

On the way down, we stopped in Geeveston to buy some food. And happened to park right outside a huge lolly shop. What a huge lolly shop is doing in a tiny little place like Geeveston, I don’t know. Philosophical considerations were not top of my daughters’ minds. They happily accepted the existence of this incongruous edifice, and came away with — I’m sorry, Joey! — lollies.

Bellies charged, we made our way to our first objective: Almost Caving. I knew we were regrettably not adequately prepared for Real Caving, so instead we were aiming for Almost Caving. That is, looking into the cave, shining torches around the entrance and wondering what was around the next corner.

Almost was still an adventure. The track was muddy. Very muddy. Wind had brought many trees down. Climbing over wet branches and landing in big mud puddles was a great introduction to the day. Instantly soggy pants. The rain kept coming down, soaking from above, while the puddles soaked from below.

Cleaning boots at the start of the track

We found a midden of abandoned boots and bottles. Presumably collected and deposited there over time. Creepy, according to the girls.

Then we arrived at an obstacle.

Initially, I did not think we would be able to cross: the river was flowing fast and I was not keen on getting even more soaked than we already were. But then I found some exposed rocks and a fallen tree. So crossed we did! Apologies for the quality of the pictures and video. They were taken on SoggyiPhoneCam.

Shortly thereafter, we arrived triumphantly at a quarry. The rain decided to come down just a bit more heavily. That didn’t stop the girls from checking out what lay beyond the rock piles.

But on to the cave. A couple minutes more trekking through mud and puddles, and we spotted it. The river was not (yet) high. So we were able to approach the entrance of the cave, and look into what lay beyond. It was also dry, which was a nice change.

Looking at the cliff face above the cave

Lolly snakes all round, and back to the car. With the crossing of the raging torrent, now a little higher. But not more than any adventurous (nearly) 7 or 10 year old could handle.

Back at the dry and cosy car, there was unanimous agreement that we would head to the Thermal Springs for some additional warmth and recovery. With car heater on full bore, your merry correspondent and his soggy herd proceeded thence forthwith. And a fun time was had by all, steaming in front of open log fires, swimming, in the rain, in the thermal pools (much warmer than the rain), but cut short all too soon by the need for sustenance, and tourist cave booking.

Perhaps Newdegate Cave was just too tame, after the morning expedition. Or maybe we were too cold and tired. But the review of the cave, as expressed by Hannah, and not contested by the rest of us, was “it wasn’t as good as I expected”. Still, we “did” the tour, saw a solitary glowworm, many stalactites and -gmites, shawls and curtains and straws, holes and flowstone and more, and, then more than a little cold, made it back to the car.

Amazingly, just enough time for the final Surprise Destination. This Surprise Destination was augmented by a multi-dollar budget for each of the girls. A Gemstone Shop! The initial joy rapidly gave way to serious perturbation. This was difficult. So many pretty rocks. So much choice. Yet, finally, both girls came away satisfied, thanks to a friendly shopowner, who was willing to mix and match presorted bags to her appreciative and choosy customers’ satisfaction. Amethysts, moonstone, mookaite and more. And, from there we made our tired, soggy, smelly way home, stopping only once to buy drinks.

I expected the girls to collapse on arrival home, but they bounced in the door, greeted baby Peter, and exuberantly leapt into the bath. And while I was off preparing Peter for bed, they set the table for a fancy meal, including candles, wine glasses (no wine, more’s the pity) and neatly arranged place settings. Maybe lollies are good for them, after all?

The case of the stack trace that wouldn’t (trace)

September 18, 2013Computing, Development, Tools, WindowsMarc Durdin

One of my favourite Windows tools is Procmon. I pull it out regularly, often as a first port of call when diagnosing complicated and opaque problems in the software I develop. Or in anyone’s software, really.

Procmon captures a trace of key I/O activities on your computer, including file, registry and network activity, and makes it really easy to spot operations that have failed or that may be causing problems. It’s great for spotting authentication problems, sharing violations, missing files and more (… malware). Procmon goes as far as recording a stack trace for nearly every operation it captures!

Today, we were trying to diagnose a problem with a process that was taking 15 seconds or longer to start on a Windows XP computer. The normal start time for that process should have been 1-2 seconds. None of the usual culprits came forward and admitted fault, so it was time to pull out Procmon again.

We quickly spotted a big fat delay in the trace. Note the time stamps in the two selected rows in the screen capture below.

A big time gap between 4:10:04 and 4:10:17

Now it was time to try and find out what was causing this. So we examined the stack trace for each entry, except … there were no symbols. Easy enough to fix — copy dbghelp.dll from a version of Microsoft’s Debugging Tools for Windows onto the system temporarily, fixup the symbol path in Procmon’s options, and … nope, still no symbols. Now this is one area where Procmon falls down a little bit. If symbol loading fails, it just silently fails. No warnings, errors or hints as to what might be going on.

This issue was occurring on a client’s computer, so it was time to take the investigation elsewhere for examination. Before we could really examine the captured trace, we needed to get symbols going. But how?

Procmon to the rescue!

That’s right, we realised we can use Procmon to diagnose itself! I booted up a clean new Windows XP virtual machine, loaded Procmon onto it, ran a basic capture of some random events.

Even after configuring symbols, they still silently failed to load. So I stopped the capture, saved it and immediately opened the saved capture, to stop this instance of Procmon from capturing events on the local computer. I then started a second instance of Procmon, removed the Procmon exclusion from the filtering, and instead, added a filter to include Procmon (I also filtered specifically for the PID of the original Procmon, later):

Then I started the trace, switched back to the first Procmon, and tried to examine the stack. Of course, still no symbols, but now it was time to switch back to the second, active Procmon process and see what we found.

And what we found was that dbghelp.dll was looking for symsrv.dll in order to download its symbols. So we copied that also into the folder with procmon.exe and suddenly everything worked!

The undecorated stack for the symsrv load request

Update 19 Sep 2013: Oops. Forgot to attach the decorated stack (sorry!):

The call stack with symbols loaded. Note how the function names differ from the original stack.

So that’s the first takeaway from this story: when you want symbols, copy both dbghelp.dll and symsrv.dll from your copy of Debugging Tools for Windows. We found no other dependencies, even with the latest version of these files.

A diversion

One curious anomaly we spotted: Procmon (or possibly Dbghelp) is looking in some strange places for debug symbols, including appending a SRV*path*url style symbol path to procmon’s parent path, and looking there, without much success:

I leave that one for you to solve.

Backtrack to the stack (trace)

Back to the original trace. We loaded up the saved trace, and found that we now got kernel mode symbols just fine, but no user mode symbols would load. In fact, Procmon doesn’t even appear to be looking for symbols for these user mode frames — either on the local drive or on the network. And this time Procmon isn’t able to give us any more detail. However, when we debugged the call that Procmon made to SymFindFileInPath when viewing a call stack in this log vs another new log, we found that Procmon wasn’t even providing the necessary identifying information.

What information is this? The identifying information that the symbol servers use is the TimeDateStamp and the SizeOfImage fields from the PE header of the executable file (slightly different for .pdb files).

I surmise that this identifying information is missing from our original trace because this trace was captured before we copied version 6.0 or later of dbghelp.dll onto the client’s computer — meaning that the version that Procmon used when capturing the trace did not record this identifying information.

Therefore, the second takeaway of the story is: always copy a recent version of dbghelp.dll and symsrv.dll into the folder with procmon.exe, before starting a trace. Even if you intend to analyze the trace later, you’ll find that without these, you won’t get full stack traces.

(Dear Microsoft, please can you consider including these in the Procmon and Procexp downloads, given that you now own Sysinternals? Saves a lot of hassle!)

Giro-Your-Strava now does Strava Routes as well

September 5, 2013Computing, Cycling, mashups, StravaMarc Durdin

Update May 2014: As Strava’s ride pages have changed format significantly, Giro-Your-Strava no longer works. The good news is that Mesmeride does rides — but unfortunately not routes as yet.

After a comment from Mike on my previous Giro-Your-Strava post asking if the bookmarklet could support Strava’s new Routes feature, I took a few minutes over breakfast to spelunk and found it wouldn’t be too hard.

Firebug’s DOM explorer actually made the task much, much simpler. My approach was simply to look for Great Big Arrays Of Numbers. I soon found, amongst all the Google, jQuery, Modernizr, Optimizely and other objects, a pageView object, which contained a number of juicy functions, including pageView.chartContext().dataContext().elevationStream() and pageView.pageContext().routeSegments(), which gave me all the information I needed. The data structures for routes are somewhat different to those for rides, so I opted to massage the Great Big Arrays those functions returned into the same basic structure as the ride data already used by the bookmarklet, rather than touch the rendering code at all.

So… after that overly detailed introduction, here ’tis. I’ve updated the bookmarklet to draw Giro-style elevation profiles for Strava Routes as well as for Rides. And of course, Le Tour-style elevation profiles for segments still work, within individual ride pages.

Note: this still won’t work in IE10, as Github returns the wrong Content-Type for the Javascript and IE gets a little panicky about it, and, well, just use a different browser.

1. Install the bookmarklet.

Here’s the bookmarklet. Just drag it to your Bookmarks toolbar to install it:

Giro-Your-Strava

2. Load a favourite Strava Route and click the bookmark.

(It’s best to wait for the page to load completely before clicking the bookmark.)

Here’s one of my favourite Strava Routes — Hobart 10,000 Day 1:

After a few seconds the gradient graphic may refresh with the correct font — this takes a second or two to download.

That’s it! This new bookmarklet still works with rides (so delete the old one!)

The source is still all on GitHub. Again, if you improve the code, or accidentally hurt yourself while reading it, please do share with a comment on this post.

And as DC Rainmaker says, thanks for reading!

Our collective wrongs

September 3, 2013Bible, Family|Bible|More, PontificatingMarc Durdin

It has been said that every generation and every culture harbour a collective wrong. When we read through history, it is easy to find cultural attitudes and actions which violate the rights of a group of people. These attitudes are often couched in pseudo-scientific terms, and dehumanising phrases are commonly used by the proponents of these attitudes. Societies’ leaders would use moral imperatives in support of their arguments. In some cases, these wrongs were even perpetuated by well-meaning people with a sadly misguided world view.

Those who spoke out against what they saw as grievous wrongs were marginalised, or even hunted down.

First they came for the communists,
and I didn’t speak out because I wasn’t a communist.

Then they came for the socialists,
and I didn’t speak out because I wasn’t a socialist.

Then they came for the trade unionists,
and I didn’t speak out because I wasn’t a trade unionist.

Then they came for me,
and there was no one left to speak for me.

– Martin Niemöller

These injustices have ranged from the tragic (stolen children) through to the terrible (slavery), right through to the monstrous (Nazi genocides).

It is easy to look back at our ancestors and condemn their actions. But are we as a society, today, ready to examine our own motives, understandings, and prejudices?

What are our collective wrongs? What injustices do we not see, or see, and not speak out against?

I am happy to announce that the missing characters have been found!

August 1, 2013Bugs, Computing, Delphi, Development, WindowsMarc Durdin

Last year I published a blog post about characters missing from print jobs with Internet Explorer 9 and my adventures in tracing and reporting the issue to Microsoft.

We saw situations where a letter would be printed with random letters missing, as per the following example:

Here’s how it should have been printed (oh yes, that’s a completely fictional name we used for testing non-English Latin script letters in printing):

Today, I was notified that Microsoft have finally publicly published a hotfix! We received the hotfix from them a few weeks ago, before it was publicly available, and it certainly solved the problem on our test machines and end user computers.

Download the hotfix from:

http://support.microsoft.com/kb/2853777 (Windows 7 SP1, Windows Server 2008 SP1)

http://support.microsoft.com/kb/2855336 (Windows 8, Windows Server 2012)

(The first link notes that the hotfix is included in the rollup in the second link, though the second link doesn’t mention the hotfix!)

Sadly, they’ve only published a solution for Windows 7 Service Pack 1 onwards, and not for Vista, which could be an issue for some users.

It was interesting to see that the issue was indeed a race condition (two threads ending up with the same random seed, because, and I quote:

This issue occurs because a conflict causes the text that uses the font to become corrupted when the two threads try to install the font at the same time. The name of the font is generated by the RAND function together with a SEEDvalue whose time value is set to srand(time(NULL)). When the two documents are printed at the same time, the SEEDvalue for the font is the same in both documents. Therefore, the conflict occurs.

So not related to the threading model, which was a little bit of misdirection on my part 😉 That’s par for the course though when debugging complex issues without source…

Fantastic to finally have the fix 🙂

Giro-Your-Strava updated to give Le Tour treatment to the climbs!

July 13, 2013Computing, Cycling, Development, JavaScript, mashups, StravaMarc Durdin

Update May 2014: As Strava’s ride pages have changed format significantly, Giro-Your-Strava no longer works. The good news is that Mesmeride does the same and more!

After the Strava API debacle, my little Tour Segment Gradient tool no longer works, which is sad. I’d put together a number of other Strava API-based widgets, but this was the only one which was really at all popular. Yesterday, DC Rainmaker himself mentioned (thank you Ray!) the Giro-Your-Strava elevation graph tool (which does still work) on his blog, so what better time to update the Tour Segment Gradient tool?

In short, what I have done is to dump both the Giro and Le Tour gradient mashups into the same bookmarklet. One click and you get beautiful isometric graphs for your ride (in Giro style) and your efforts (in Le Tour style). Yes, I get the inconsistency, but what would life be without idiosyncracies?

1. Install the bookmarklet.

Here’s the bookmarklet. Just drag it to your Bookmarks toolbar to install it:

Giro-Your-Strava

2. Load a favourite Strava ride and click the bookmark.

(It’s best to wait for the page to load completely before clicking the bookmark.)

Presto, you’ll get two spiffy new buttons, one for your ride:

And one for the segment view:

So go ahead and click the Giro button, and you’ll see:

Click the Le Tour button for the new elevation profile for a segment:

Have fun!

One danger with bookmarklets that fiddle with an existing site in this way is that they will tend to break when the site updates. There are no stability guarantees that APIs (ususally!) provide, so YMMV. However, if the Strava site layout changes, it’s probably only a simple tweak to the code to get it working again.

There’s nothing beautiful about the code on the backend. It really needs rewriting and modularisation etc etc etc but hey, it works 😉 Do what you want with the code, just share it with us all if you improve things!

Strava have decided to abandon their developer community

July 3, 2013Cycling, Griping, mashups, StravaMarc Durdin

I received an email from Strava today which was very disappointing.

Hello Marc,

As of July 1, 2013, V1/V2 API endpoints have been retired. Libraries, sites, and applications using these endpoints will cease to work.

In previous blog updates, we’ve discussed status and access to V3 of our API. As mentioned then, we had to make difficult decisions this year about where to invest time and resources – feature development or a full-fledged API program. We have chosen to focus on feature development at this time and so access to V3 of our API is extremely limited.

Any developer who has been granted access to V3 of the API has been contacted. We will revisit our API program and applications from time to time, but for the time being, we have no plans to grant further access in 2013.

If you have questions or comments, please send an email to [email protected]. Given our limited resources, you should not expect an immediate response.

Thanks for your understanding,

Your Friends at Strava

The highlights (or lowlights) from this email:

Strava have no plans to allow anyone else to access their new API for at least six months
It seems Strava aren’t interested in dialogue with their community about this decision.

I’ve been using the Strava V1 and V2 APIs now for several years, and have written quite a few blog posts about how to use it. The V1 and V2 APIs were always fairly experimental, but that was okay. We all understood that and it was part of the fun of working with Strava.

Discontinuing the V1 and V2 APIs was in the pipeline, and again we Strava API users could understand limiting the V3 API beta program to a small number of developers. However, the abrupt announcement today that not only are the V1 and V2 APIs no longer available, but now Strava won’t be making the V3 API available to anyone else for an Internet Eternity (that is, at least 6 months) either is completely unexpected.

The only Windows Phone app that integrates with Strava no longer works. My apps no longer work. And there appears to be no future for any of these apps.

Now, I’ve been a Strava Ambassador since before they started their official Ambassador program, and have had a lot of fun promoting Strava. I’ve developed popular tools that integrate with the Strava API. It is a very big disappointment to me that they’ve decided to throw away all that good-will with their developer community. This is not the way to treat the most dedicated and enthusiastic portion of your user base, Strava!

Giro-style elevation graphs for Strava

May 24, 2013Computing, Cycling, Development, Hobart's Top Ten Climbs, JavaScript, mashups, StravaMarc Durdin

Update May 2014: As Strava’s ride pages have changed format significantly, Giro-Your-Strava no longer works. The good news is that Mesmeride does the same and more!
Update 13 July 2013: An updated version of Giro-Your-Strava is now available here.

Just in time for the last few days of the Giro, I’ve finished a little after-hours Strava mashup project that builds on the segment graphs that I originally created for the Hobart 10,000.

I’m sure you’ve seen some of the Giro elevation graphs. Here’s one, from Stage 11:

Now, here’s a bookmarklet. Drag it to your bookmarks toolbar, load up a favourite hilly ride on Strava, and click the bookmarklet.

Giro-Your-Strava

A mysterious new button will appear in the graph menu. Go, on click it!

And up pops an elevation graph that makes it look like you’ve been riding the Giro!

The algorithm picks the categorised climbs from your ride (and tries to figure out the most appropriate segment where multiple segments finish at the top of a hill). Non-categorised segments are currently ignored. The whole project is published on GitHub, so you can tweak it and improve it to your heart’s content. Your first step should be to tidy up the mess I’ve left you 🙂

Marc Durdin's Blog

Thoughts on computing, Bible, cycling, family, my stories and more

Debugging a stalled Delphi process with Windbg and memory searches

Introducing Mesmeride

Waypoints

Mountains or Molehills?

Size and Length

Sharing

Almost Caving, and Tourist Caving

The case of the stack trace that wouldn’t (trace)

Procmon to the rescue!

A diversion

Backtrack to the stack (trace)

Giro-Your-Strava now does Strava Routes as well

1. Install the bookmarklet.

2. Load a favourite Strava Route and click the bookmark.

Our collective wrongs

I am happy to announce that the missing characters have been found!

Giro-Your-Strava updated to give Le Tour treatment to the climbs!

1. Install the bookmarklet.

2. Load a favourite Strava ride and click the bookmark.

Strava have decided to abandon their developer community

Giro-style elevation graphs for Strava