Category Archives: Development

dynamic import on node.js with circular dependencies leads to interesting failure modes

So, given these three ES modules below, what output do you expect to get when you run node one.js?

one.js:

import { two } from './two.js';

await two();

export function one() {
  console.log('one');
}

two.js:


export async function two() {
  console.log('two');
  const { three } = await import('./three.js');
  three();
}

three.js:

import { one } from './one.js';
export function three() {
  console.log('three');
  one();
}

Following through the calls, you’d expect to get:

two
three
one

But instead you get just the following output:

two

And node exits with exit code 13, which means “Unfinished Top-Level Awaitawait was used outside of a function in the top-level code, but the passed Promise never resolved.” There are no messages reported to the console, which is very disconcerting!

It turns out that the await import() causes this, because of the circular dependency introduced in module three.js, back to one.js. Now, if there is no async involved, then this circular dependency just works; that is, instead of using import('./three.js') you use import { three } from './three.js':

two
three
one

Not sure yet how to move forward with this. Tested in Node 20.9.0 and 18.14.1. Should I be raising it as a bug? It’s certainly not a documented behaviour that I’ve been able to find (node documentationmdn documentation).

Debugging a Windows Service

This is a set of notes on how to debug a Windows service starting up, mostly for my reference. Building on https://www.sysadmins.lv/retired-msft-blogs/alejacma/how-to-debug-windows-services-with-windbg.aspx with command line steps where possible.

In this example, we’ll be debugging mycool.exe, which has the service name mycoolservice.

Enabling debugging

  1. Find the path to cdb.exe, windbg.exe, gflags.exe. (e.g. C:\Program Files (x86)\Windows Kits\10\Debuggers\x86).

  2. Start an elevated command prompt. Set the service to manual start (and stop it if it is currently running, … duh):

    sc config mycoolservice start=demand
    sc stop mycoolservice
    
  3. Find the short path for cdb.exe (pasting the path from point 1 as appropriate):

    for %A in ("C:\Program Files (x86)\Windows Kits\10\Debuggers\x86\cdb.exe") do @echo %~sA
    
  4. Enable the debug hook for the service, using gflags, replacing the path as necessary:

    C:\PROGRA~2\WI3CF2~1\10\DEBUGG~1\x86\gflags /p /enable mycool.exe /debug "C:\PROGRA~2\WI3CF2~1\10\DEBUGG~1\x86\cdb.exe -server tcp:port=9999"
    
  5. Change the service startup timeout to 1 hour to avoid Windows killing the service on startup:

    reg add HKLM\System\CurrentControlSet\Control /v ServicesPipeTimeout /t REG_DWORD /d 3600000
    
  6. Reboot, start an elevated command prompt again.

  7. Start the service, which will appear to hang:

    sc start mycoolservice
    
  8. Open Windbg, Ctrl+R tcp:server=localhost,port=9999

  9. Go forth and debug.

Disable debugging

  1. Start an elevated command prompt, and enter the following commands:

    C:\PROGRA~2\WI3CF2~1\10\DEBUGG~1\x86\gflags /p /disable mycool.exe
    reg delete HKLM\System\CurrentControlSet\Control /v ServicesPipeTimeout
    
  2. Reset the service startup parameters to your preferred startup type.

  3. Reboot to reset the service control timeout.

location.hash can be re-entrant on Safari Mobile

So Josh and I were observing some really strange behaviour in the Keyman for iOS beta. When typing rapidly on the touch keyboard, we would sometimes get the wrong character emitted. We could not see anything immediately wrong in the code. So Josh added some logging. Then things got really weird: we would get the start of a touch event, then before the touch event handler finished, we’d get logging that indicated another touch event was received.

Whoa. In JavaScript, that shouldn’t be possible, right?

Each message is processed completely before any other message is processed. This offers some nice properties when reasoning about your program, including the fact that whenever a function runs, it cannot be pre-empted and will run entirely before any other code runs (and can modify data the function manipulates). This differs from C, for instance, where if a function runs in a thread, it can be stopped at any point to run some other code in another thread.

— https://developer.mozilla.org/en-US/docs/Web/JavaScript/EventLoop

After we dug in further, we found that it was indeed possible for a new touch event (and perhaps others) to be received when location.hash was set from the JavaScript code, in Safari for iOS.

Here’s a fairly minimal repro. Load this up on your iPhone, and start rapidly touching the Whack div. You’ll probably need to use two fingers repeatedly to trigger the event (I can usually get it to happen with about 50-100 rapid touches). When it happens, you’ll get a log message with a call stack showing how the whackIt() function has apparently called itself!

<!doctype html>
<html>
  <head>
    <meta charset='utf8'>
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no" /> 
    <title>location.hash re-entrancy</title>
    <style>
      #whack { font-size: 64pt }
      #log, #stack { font-family: courier }
    </style>
  </head>
  <body>
    <div id=whack>Whack</div>
    
    <div id=log></div>
    
    <div id=stack></div>
    
    <script>
    
      var tick = 0, inWhack = false;
      
      function whackIt(e) {
        e.preventDefault();

        tick++;
        
        if(inWhack) {
          stack.innerHTML = 'Re-entrant event: '+(new Error()).stack;
          return false;
        }
        
        var localTick = tick;
        log.innerHTML = tick;

        inWhack = true;

        // Setting location.hash seemingly can cause the 
        // event queue to be polled, resulting in a 
        // re-entrant touch event
        location.hash = '#'+localTick;

        inWhack = false;
        
        if(localTick != tick) {
          // alert('We had a re-entrant event');
        }
        
        return false;
      }
      
      whack.addEventListener('touchstart', whackIt, false);
      whack.addEventListener('touchend', whackIt, false);
      
    </script>
  </body>
</html>

We’ll be reporting this to Apple… Just a heads up.

The case for Keyman

Recent podcast

I was recently privileged to be a guest on Scott Hanselman’s Hanselminutes podcast. It was great to talk about the history of Keyman and how it solved real problems for us 20+ years ago. But afterwards I realised I hadn’t really described what sorts of problems Keyman solves today and why it is still relevant.

So this post is that story.

Note: I’ve put this on my personal blog rather than the Keyman blog because it’s full of my own personal opinions. There’s lots of useful content on the Keyman blog as well!

The elevator pitch – and why I always struggled with it

A common story around venture capitalists and tech startups is that you have to have an amazing elevator pitch, the idea that within about 20-30 seconds, you can sell a problem and its solution to a person just begging to give you great wads of cash. And if you can’t describe the problem in 20 seconds then you probably don’t understand it yourself.

With all due respect, that might be fine for selling package drones or IoT nerf guns, but there are plenty of hard problems out there that cannot be understood in 20 seconds.

I think keyboard input for the world’s writing systems is one of those problems. For many people growing up with limited contact with Asian writing systems, they will look pretty and exotic. But that’s about the extent of it.

So, let’s imagine you and I are in an elevator, and just as you ask me to explain the benefit of Keyman to you, the power goes out and the elevator lurches to a halt. Now you are stuck with me and I can give you the extended elevator pitch!

An illustration from Khmer

Khmer is the national language of Cambodia, a small nation in South East Asia, home of one of the world’s ancient wonders, Angkor Wat.

Photo by Marc Durdin

Here’s a couple of samples of Khmer text. The first is a photo of an inscription from Angkor Wat.

This second image shows two common styles for writing Khmer, as computer fonts:

In a little bit, I’ll dig into how the writing system is used on computer. The script is beautiful, and it is complex, and it has a long and interesting history.

But first, I would like to tell you a story. I recently learned about an effort by a group of Cambodian linguists to revise and prepare a new edition of a Khmer language dictionary. The current “gold standard” Khmer dictionary, compiled by Samdech Porthinhean Chuon Nath, has not been updated since 1967. So a new edition is very much needed.

A number of different people have been involved in typing up entries for the dictionary. In the process, they discovered a problem: some entries which looked identical on screen were actually encoded differently in the computer. This meant that they were unable to consistently search for words, or even sort their dictionary correctly.

How had this happened? This happened because it is possible to type Khmer words in a number of different ways and get identical output on the screen. And I don’t mean by using different input methods, either. I mean with the standard Khmer keyboard layouts supplied with all major operating systems, desktop and mobile.

The Khmer Script

In this post, I won’t get too technical with lots of references to Unicode encodings. Instead, I’m trying to illustrate the issues from the point of view of a user, who shouldn’t need to worry about those details.

How would we go about typing a Khmer word? I’ve picked a famous Khmer word for us to start with:

This is the word ខ្មែរ /kmae/, Khmer, which is the name of the language and the people.

So, with the standard Khmer keyboard, how do you type this word?

Let’s imagine you had this word on a piece of paper and a standard Khmer keyboard in front of you. We can tell that it’s made up of a group of characters and some sort of diacritic marks.

Start with the first shape. It looks like a   symbol and a  diacritic mark. But the two marks belong together and make up a vowel sound, which we can find on the [SHIFT]+[E] key:

Let’s go with that.

Next we have the  character and the diacritic mark. The first bit is easy enough to find, on the [X] key.

But that second bit? Okay, you won’t find that printed on a key cap. Fortunately, those using the Khmer keyboard tend to know their writing system. The character is a form of the  /m/ consonant used when it is the second letter in a consonant cluster. It is called a ជើង /cəəŋ/ letter in Khmer. This means leg or foot, base or support. (Diversion: The Unicode charts call them COENG consonants. I bet you read that as CO-ENG, didn’t you. I know I used to!)

Okay, that’s all very interesting but how does one type these cəəŋ consonants? Well, the Unicode encoding standard added a special control character that’s not part of the writing system, called a KHMER SIGN COENG (U+17D2), just to support these letters. This is typed with a [J] key on the standard keyboard. Now we’re getting somewhere. So we’ll need to type [J] [M] to get that  appearing under the .

Now before we go any further, this magic sign is a problem. It’s a problem because now the user has to know something about the way their text is encoded in order to type it. It’s not a particularly hard thing to learn, but it isn’t obvious. And this is one of the easier things that a native Khmer speaker needs to learn in order to type their own language.

The last letter is easy enough. Here it is on the [R] key.

The full sequence, first try

So the full sequence is: [SHIFT]+[E] [X] [J] [M] [R].

Let’s go ahead and type it.

Ugh. Where did that dotted circle come from?

The dotted circle is a visual hint. It tells us that the vowel  is written before the consonant it is attached to, but that the encoding places it after.

Second try

Okay, so let’s change it to [X] [SHIFT]+[E] [J] [M] [R].

Okay, that looks correct now.

Well, okay, it looks fine … but it’s not. In order to meet the Unicode ordering rules, the vowel has to be placed after the entire consonant cluster, just as if it was spoken.

Third time lucky

Let’s go back and fix that again. Here’s our final sequence. [X] [J] [M] [SHIFT]+[E] [R].

This is the correct sequence.

Now from a linguistic point of view most of this seems fairly logical, and it generally makes sense to users, once they learn it. But I hope this illustrates how some of these things are not obvious. If you haven’t been taught these rules, or if you do accidentally mis-type something, you won’t ever know about it.

It’s not hard to find examples that have a heap of different ways you could type them. For example:

At first glance that doesn’t look a lot more complex, right? However, we’ve found 14 different ways to encode this in Unicode, all of which look correct on Android devices! And if you repeat diacritic marks, then you can expand that to over 35 different combinations, all displayed identically! My colleagues and I wrote a paper on the problem. Can you imagine searching an online dictionary with these kinds of problems?

Other parts of the solution

Now I know some of you are going to be saying that this isn’t a problem that is unique to Khmer. That it can be solved with post-processing and normalisation. And you are correct, this problem extends to many scripts. And some problems can be solved with normalisation. But I think Unicode normalisation is an appropriate solution only with correctly encoded data. Our goal is for data to be encoded correctly at the time it enters the system.

You might argue that Autocorrect-type technologies could help with this, and they certainly can. But again, Autocorrect is really targeted at spelling and word completion, not avoiding encoding issues. (The inconsistently available implementations of Autocorrect don’t help either).

Improved rendering engines could also give helpful visual feedback to users, alerting them to mistyped sequences.

But none of these negate the importance of what Keyman brings to the party.

Keyman: A Comprehensive Input Method Solution

So what does Keyman bring to this party? None of the system supplied Khmer keyboards make any attempt to quality control the input – generally they just output a single character for a single key.

Given we have well defined, structured linguistic and encoding rules, we can leverage those in the input method and ensure that sequences that are invalid according to the specification just don’t make it into the text store.

The power of Keyman

Keyman can automatically reorder and replace input as you type. For our example, if you type a vowel before typing a KHMER COENG SIGN and a consonant, Keyman will transparently and instantly fix the order in your document, and you won’t even know.

You can try a Khmer keyboard which automatically solves these encoding issues online at https://keymanweb.com/?tier=alpha#km,Keyboard_khmer_angkor

That, in a nutshell, is the power of Keyman.

Keyman makes possible amazingly intuitive input for complex writing systems.

Keyman is open source and completely free.

Keyman keyboard layouts are defined with a clear and easy to understand keyboard grammar, so that anyone can write a keyboard layout for their language. The development tools include visual editors, interactive debuggers and automated testing to help you develop sophisticated keyboard layouts.

Keyman runs everywhere. A keyboard layout can deployed to Windows, macOS, iOS, Android, Linux and even as a Javascript web keyboard. The Keyman project has a repository of over 700 existing keyboard layouts, and more are added every week.

Version 10 of Keyman is about to hit Beta. Would you like to be involved?

Oh look, the power is back on. Elevator pitch over. Oh by the way, aren’t writing systems cool?

Working around Delphi’s default grid scrolling behaviour

Delphi’s T*Grid components have an annoying little feature whereby they will scroll the cell into view if you click on a partially visible cell at the right or the bottom of the window. Then, this couples with a timer that causes the scroll to continue as long as the mouse button is held down and the cell it is over is partially visible. This typically means that if a user clicks on a partially visible cell, they end up selecting a cell several rows or columns away from where they intended to click.

In my view, this is a bug that should be fixed in Delphi. I’m not the only person who thinks this. I’ve reported it to Embarcadero at RSP-18542.

In the meantime, here’s a little unit that works around the issue.

{
  Stop scroll on mousedown on bottom row of grid when bottom row
  is a partial cell: have to block both initial scroll and timer-
  based scroll.

  This code is pretty dependent on the implementation in Vcl.Grids.pas,
  so it should be checked if we upgrade to new version of Delphi.
}

{$IFNDEF VER320}
{$MESSAGE ERROR 'Check that this fix is still applicable for a new version of Delphi. Checked against Delphi 10.2' }
{$ENDIF}

unit ScrollFixedStringGrid;

interface

uses
  System.Classes,
  Vcl.Controls,
  Vcl.Grids,
  Winapi.Windows;

type
  TScrollFixedStringGrid = class(TStringGrid)
  private
    TimerStarted: Boolean;
    HackedMousedown: Boolean;
  protected
    procedure MouseDown(Button: TMouseButton; Shift: TShiftState; X: Integer;
      Y: Integer); override;
    procedure MouseMove(Shift: TShiftState; X: Integer; Y: Integer); override;
    function SelectCell(ACol, ARow: Longint): Boolean; override;
  end;

implementation

{ TScrollFixedStringGrid }

procedure TScrollFixedStringGrid.MouseDown(Button: TMouseButton;
  Shift: TShiftState; X, Y: Integer);
begin
  // When we first mouse-down, we know the grid has
  // no active scroll timer
  TimerStarted := False;

  // Call the inherited event, blocking the default MoveCurrent
  // behaviour that scrolls the cell into view
  HackedMouseDown := True;
  try
    inherited;
  finally
    HackedMouseDown := False;
  end;

  // Cancel scrolling timer started by the mousedown event for selecting
  if FGridState = gsSelecting then
    KillTimer(Handle, 1);
end;

procedure TScrollFixedStringGrid.MouseMove(Shift: TShiftState; X, Y: Integer);
begin
  // Start the scroll timer if we are selecting and mouse
  // button is down, on our first movement with mouse down
  if not TimerStarted and (FGridState = gsSelecting) then
  begin
    SetTimer(Handle, 1, 60, nil);
    TimerStarted := True;
  end;
  inherited;
end;


function TScrollFixedStringGrid.SelectCell(ACol, ARow: Longint): Boolean;
begin
  Result := inherited;
  if Result and HackedMousedown then
  begin
    // MoveColRow calls MoveCurrent, which
    // calls SelectCell. If SelectCell returns False, then
    // movement is blocked. But we fake it by re-calling with Show=False
    // to get the behaviour we want
    HackedMouseDown := False;
    try
      MoveColRow(ACol, ARow, True, False);
    finally
      HackedMouseDown := True;
    end;
    Result := False;
  end;
end;

end.

 

Substituted layouts in Text Services Framework

There is a tantalising parameter in the ITfInputProcessorProfileMgr::RegisterProfile function called hklSubstitute.  The description (used to) read “The substitute keyboard layout of this profile.   When this profile is active, the keyboard will use this keyboard layout instead of the default keyboard layout.  This can be NULL if your text service doesn’t change the keyboard layout.”

This functionality also surfaces in ITfInputProcessorProfileSubstituteLayout::GetSubstituteKeyboardLayout and ITfInputProcessorProfiles::SubstituteKeyboardLayout.

From the minimal description in MSDN, this sounds like a feature that we want to use in Keyman Desktop (allowing us to, for instance, use kbdus.dll as a base layout for some Keyman keyboard layouts).  However, we have been unable to get it to work.

  1. It seems the parameter type in the documentation is not quite right.  This parameter is not a HKL that you can obtain from LoadKeyboardLayout or some such.  Instead it needs to be a KLID, a keyboard layout ID, such as 0x00010409 for US Dvorak.
  2. The base layout is limited to one linked to the language the TIP is being installed for.  This means you cannot, for instance, substitute German QWERTZ for a French TIP, even if you wanted to. (And we want to!)
  3. Despite these restrictions, the RegisterProfile function does not check the validity of the parameter and happily tells you that the profile registered OK.
  4. While GetSubstituteKeyboardLayout returns the exact value that you registered, ITfInputProcessorProfileMgr::GetProfile returns 0 for the hklSubstitute member unless you meet the preconditions described above.  Furthermore, hklSubstitute, when available, is not a HKL but a KLID.
  5. Finally, even after all this, it seems that the substitute keyboard layout does not apply.

I received some clarification from Microsoft (edited slightly, thank you Vladimir Smirnov):

In fact, the feature is not that generic as you apparently expect. Its whole purpose was to enable smooth deprecation of IMM32 IMEs and their replacement (substitution) with TSF IMEs (aka TIPs). It doesn’t support substituting an arbitrary plain keyboard layout with another layout or IME (let alone of a different language).

The hklSubstitute is supposed to be a real HKL, but for IMM32 IMEs it actually matches the IME’s KLID. There’s old verification code there which looks for a matching KLID in HKLM for a given hklSubstitute and ignores the latter if there’s no match. That’s why you get 0 from GetProfile for your invalid or unsupported hklSubstitute.

Generic substitution was just never meant to be supported in TSF.

It’s a shame, but we work with what we’ve got. So for Keyman Desktop 9, we wrote a mnemonic layout recompiler that recompiles a Keyman mnemonic layout keyboard, merging it with a given Windows system base layout, at keyboard installation time in Keyman Desktop.

The KeymanWeb Hello World app

A couple of days ago I helped some developers put together a very basic web page that uses KeymanWeb to showcase their keyboard layout. The idea is that a visitor to the page will immediately be able to try out their keyboard layout without any fiddling around.

Another requirement was that the page work well on desktop and mobile devices. This requires a few little tricks, mostly because KeymanWeb has some special requirements to work smoothly on mobiles and tablets.

I’ve presented snippets of the code, in the order they appear in the document.

And here are screenshots just for posterity.

The usual HTML 5 boilerplate is thrown in at the top of the page. Our first trick is to put the page into an appropriate zoom and scroll mode for mobile and tablet devices with a viewport meta tag. This is done with the following code:

<!-- at present, KMW requires this for mobile -->
<meta name="viewport" content="width=device-width, user-scalable=no">

At the time of writing, KeymanWeb (build 408) requires the viewport meta tag in order to render correctly on a variety of devices. Both clauses in the content attribute are required, and the keyboard will not scale correctly without them.

Next, I add a link to the stylesheet for the on screen keyboard. This is optional, because KeymanWeb will inject the link itself anyway, but it does prevent a Flash Of Unstyled Content.

<!-- this is optional but eliminates a flash of unstyled content -->
<link href='https://s.keyman.com/kmw/engine/408/src/osk/kmwosk.css' rel='stylesheet' type='text/css'>

I throw in some basic styling for the desktop and touch devices. You’ll see a selector of body.is-desktop on a couple of the style selectors. I use this instead of a @media query because this is differentiating between viewport-controlled devices and plain old boring desktop browsers. There may be a better way of doing this now, but I haven’t found it yet. The code for setting the is-desktop class is found further on in this post.

/* Styles for the page */

* {
  font-family: Tahoma, sans-serif;
}

body.is-desktop {
  padding: 22px 2px 0 2px;
  width: 900px;
  margin: auto;
}

h1 {
  font-size: 24pt;
  font-weight: bold;
  margin: 0 0 24px 0;
  padding: 0;
}

body.is-desktop h1 {  
  font-size: 48pt;
}

/* Style the text area */

textarea {
  width: 100%;
  height: 250px;
  font-family: "Khmer OS";
  font-size: 24px;
}

I need to talk a bit more about the On Screen Keyboard (OSK) styling. While I could use the default styling (as shown below), this looks a little dated and I wanted to show how the keyboard could be styled as you like. There is some complexity to the OSK styling as it has many moving parts, around sizing, scaling and positioning of each individual key. I don’t want to throw away that baby, so instead I keep all the bathwater and just add some bubbles to jazz up the display the way I want it. That’s a mixed metaphor, but it was late when I wrote this.

The default KeymanWeb keyboard style

First, this CSS rule-set removes the thick red border and dark background, and adjusts the padding around the sides to compensate.

/* Style the on screen keyboard */

body .desktop .kmw-osk-inner-frame {
  background: white;  
  border: solid 1px #404040;
  padding-bottom: 2px;
  padding-top: 8px;
}

Next, I hide the header and footer for the OSK. I don’t need them for this demo.


body .desktop .kmw-title-bar,
body .desktop .kmw-footer {
  display: none;
}

I tweak the spacing between keys and rows and make the keys slightly less rounded.

body .desktop .kmw-key-square {
  border-radius: 2px;
  padding-bottom: 8px;
}
body .desktop .kmw-key {
  border-radius: 2px;
  border: solid 1px #404040;
  background: #f2f4f6;
}
body .desktop .kmw-key:hover {
  background: #c0c8cf;
}

And that’s it for the CSS changes. It’s really pretty easy to restyle the keyboard without losing the benefits of a scalable, cross-platform keyboard. You’ll note that sneaky .desktop selector creeping in there. That’s because I’ve opted to style only the desktop keyboard; the touch layouts are already pretty nice and I’ll keep them as is.

I load the KeymanWeb code from the KeymanWeb CDN. You can load from the KeymanWeb CDN (running on Azure) or keep a locally hosted copy, of course. I’ve commented out the source version and because we have only a single keyboard, I have elected not to include a menu for switching languages.

<!-- Uncomment these lines if you want the source version of KeymanWeb (and remove the keymanweb.js reference)
<script src="https://s.keyman.com/kmw/engine/408/src/kmwstring.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwbase.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/keymanweb.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwosk.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwnative.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwcallback.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwkeymaps.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwlayout.js"></script>
<script src="https://s.keyman.com/kmw/engine/408/src/kmwinit.js"></script>
-->

<script src="https://s.keyman.com/kmw/engine/408/keymanweb.js"></script>

<!-- Uncomment if you want the user to be able to switch keyboards with a toggle popup ...
or you can create a custom switch if you prefer
<script src='https://s.keyman.com/kmw/engine/408/kmwuitoggle.js'></script>
-->

Now, no self-respecting web developer is going to include inline script, except in a Hello World demo. So let’s call this the KeymanWeb Hello World demo, so I can keep my self respect. Or something.

I use a simple page load listener. You could use an earlier listener, such as DOMContentLoaded, but window.onload has a rich and nearly honourable history.

// This should really be in a separate file. 
// But for now in one file is easier to understand
       
// After everything has loaded
window.addEventListener('load', function() {

Next some voodoo. Allows me to style the touch and desktop versions differently, as I touched on earlier. Why? Because KeymanWeb works very differently on touch devices. If you think about this, it must be so. A touch device has its own soft keyboard, which KeymanWeb must, by slightly convoluted means, hide, and replace with its own soft keyboard. Whereas, on a desktop device, KeymanWeb can show a utility soft keyboard, but does most of its interaction by getting in there between the hardware keyboard and the input fields.

Don’t ask me about Windows touch devices. Does anyone actually use those? (Okay, I’m sure they do, but it’s a rocky path for us poor keyboard devs to tread!)

if(!tavultesoft.keymanweb.util.isTouchDevice()) {
  document.body.className += 'is-desktop';
}

Next, make sure KeymanWeb is initialised. If I don’t do this myself, KeymanWeb will do so when things are ready, but I need KeymanWeb to be ready in order to load keyboards and attach events, and so on.

tavultesoft.keymanweb.init();

I could add another keyboard, or a stock keyboard from the repository. I haven’t, but this shows you how.

//tavultesoft.keymanweb.addKeyboards('@eng'); // Loads default English keyboard from Keyman Cloud (CDN) if you want it

In this case, I have a custom keyboard, developed by Lyheng Phuoy. This is an early beta version of the keyboard but already it’s very impressive.

// Add our custom keyboard        
tavultesoft.keymanweb.addKeyboards({
  name: 'Khmerism',       // Display name of the keyboard
  id: 'lyhengkeyboard',   // ID of the keyboard for reference in code
  filename: './lyhengkeyboard-1.0.js',  // source of the keyboard for dynamic load
  version: '1.0',         // version of the keyboard, optional
  language: [{
    name: 'Khmer',        // language name for UI elements
    id: 'khm',            // language ID for tagging text
    region: 'as'          // region of the language, for UI elements, optional
  }]
});

In this section, I want to control the size, position and flexibility of the keyboard. I don’t want it to be resizable or movable. So I set nomove and nosize accordingly.

var ta = document.getElementsByTagName('textarea')[0];

// Watch for the keyboard being shown and set its position
        
tavultesoft.keymanweb.osk.addEventListener('show', function() {
  var rectParams = {
    left: ta.offsetLeft, 
    top: ta.offsetTop + ta.offsetHeight + 8, // a small gap below the text area
    width: ta.offsetWidth, 
    height: ta.offsetWidth * 0.4, // a pleasing aspect ratio
    nomove: true,
    nosize: true
  };
  tavultesoft.keymanweb.osk.setRect(rectParams);
});
      
// Focus text area after everything loads
        
ta.focus(); 

Finally, in the body element, I set a special magic class, osk-always-visible, so that KeymanWeb doesn’t hide its on screen keyboard after displaying it the first time. And then we have basically the world’s simplest page with a textarea.

<!-- The osk-always-visible class tells KeymanWeb not to hide the osk on blur, after it is
     shown the first time -->
<body class="osk-always-visible">
  <h1>Typing Khmer</h1>
  <textarea rows="6" cols="60"></textarea>	
</body>

And that’s it! I’d love to see what you do with KeymanWeb on your own sites.

Here’s the full keyboard source, and that link to the demo page again, including Lyheng’s keyboard (with his permission).

Don’t forget to navigate to about:blank when embedding IWebBrowser2

Today I spent several hours trying to figure out why an embedded web browser component (in this case TEmbeddedWB) in a Delphi test app never received the appropriate IHttpSecurity and IWindowForBindingUI QueryService requests.

I was doing this in order to provide more nuanced handling of self-signed certificates in an intranet context. We all do this, right? Here the term “nuanced” means “Of course I trust self signed certificates on my intranet, don’t you?” Feel free to rant and rave on this. 😉

But no matter what I did, what incantations I tried, or what StackOverflow posts I perused, I was unable to find an answer. Until finally I stumbled on a side comment in a thread from 2010. Igor Tandetnik notes that:

Right after creating the control, navigate it to about:blank. Right after that, navigate it to the page you wanted to go to. It’s a known problem that IServiceProvider doesn’t work for the very first navigation.

And this was something that I kinda knew in the back of my head, but of course had forgotten. Thank you Igor.

This post would not be complete without some splendiferous code. Just for reference, it’s so simple if you don’t blank out and forget about:blank.

unit InsecureBrowser;

interface

uses
  Winapi.Windows,
  Winapi.Messages,
  Winapi.Urlmon,
  Winapi.WinInet,
  System.SysUtils,
  System.Variants,
  System.Classes,
  Vcl.Graphics,
  Vcl.Controls,
  Vcl.Forms,
  Vcl.Dialogs,
  Vcl.OleCtrls,
  Vcl.StdCtrls,
  SHDocVw_EWB,
  EwbCore,
  EmbeddedWB;

type
  TInsecureBrowserForm = class(TForm, IHttpSecurity, IWindowForBindingUI)
    web: TEmbeddedWB;
    cmdGoInsecure: TButton;
    procedure webQueryService(Sender: TObject; const [Ref] rsid,
      iid: TGUID; var Obj: IInterface);
    procedure FormCreate(Sender: TObject);
    procedure cmdGoInsecureClick(Sender: TObject);
  private
    { IWindowForBindingUI }
    function GetWindow(const guidReason: TGUID; out hwnd): HRESULT; stdcall;

    { IHttpSecurity }
    function OnSecurityProblem(dwProblem: Cardinal): HRESULT; stdcall;
  end;

var
  InsecureBrowserForm: TInsecureBrowserForm;

implementation

{$R *.dfm}

function TInsecureBrowserForm.GetWindow(const guidReason: TGUID;
  out hwnd): HRESULT;
begin
  Result := S_FALSE;
end;

function TInsecureBrowserForm.OnSecurityProblem(dwProblem: Cardinal): HRESULT;
begin
  if (dwProblem = ERROR_INTERNET_INVALID_CA) or
     (dwProblem = ERROR_INTERNET_SEC_CERT_CN_INVALID)
    then Result := S_OK
    else Result := E_ABORT;
end;

procedure TInsecureBrowserForm.webQueryService(Sender: TObject;
  const [Ref] rsid, iid: TGUID; var Obj: IInterface);
begin
  if IsEqualGUID(IID_IWindowForBindingUI, iid) then
    Obj := Self as IWindowForBindingUI
  else if IsEqualGUID(IID_IHttpSecurity, iid) then
    Obj := Self as IHttpSecurity;
end;

procedure TInsecureBrowserForm.cmdGoInsecureClick(Sender: TObject);
begin
  web.Navigate('https://evil.intranet.site/');
end;

procedure TInsecureBrowserForm.FormCreate(Sender: TObject);
begin
  web.Navigate('about:blank');
end;

end.

 

My favourite debugging story

This was some years ago, when I was living in Vientiane, the capital of the Lao Peoples’ Democratic Republic. It was 1994 or thereabouts – just prior to the release of Windows 95. I had written a piece of software called “Keyman” which was being increasingly used to type in Lao in Windows 3.1, overloading characters in the 128-255 range of the standard US English character set at the time (I don’t want to be too technical here). Before Unicode.

The owner of a local computer store had had reports of an issue with Keyman from one of their clients in the provincial capital of Savannakhet, about a one hour flight from Vientiane in a small plane. The technical minutiae of the problem escape me now, but it was something to do with a certain set of keystrokes which gave the wrong output in some applications – I think Excel. The report had been communicated over the telephone to the computer shop technical staff, and then translated into English for my benefit – as my Lao was probably not good enough to really get the detail. So as you can imagine, Chinese Whispers is a good way to describe the final report I received.

I tried to diagnose the problem from the description, and tried to reproduce it on my computer, but could not figure it out.

Now it is important to remember that Laos in 1994 was still pretty much unknown to the outside world. There were few tourists; it was (and is) a communist country, at least in principle. Things there didn’t work quite the same as in Australia. There was no Internet access in Laos at that time, telephones were unreliable and the use of modems was technically illegal. This meant that remote diagnosis was only possible by means of telephoned conversations over noisy phone lines, by fax, or with posted letters. The post often took weeks, even in-country.

So after a few days of fruitless telephoning back and forth, the owner of the computer shop suggested I accompany him on a trip down to Savannakhet. (From memory, he was already planning to visit). I was but a callow youth, of 17, and so this was a fantastic opportunity!

When we met up at the airport, the first thing I remember was standing in the security line behind a Lao businessman who caused a bit of a ruckus at the hand luggage screening, because his briefcase had two pistols in it. This seemed a little unusual, even in Laos. After some discussion, his pistols were removed from his hand luggage and given to a guard, who told him he could not take them on the plane because there was not a separate luggage hold. I don’t know what happened to them after that as we were ushered through the security.

When our plane started boarding, a second problem arose. It appeared that the flight had been overbooked, or perhaps they’d substituted a smaller aircraft. The plane we could see was a Xian Y-7, a Chinese clone of a Russian Antonov An-24. The Wikipedia page linked above shows a picture, coincidentally enough of a Lao Aviation plane, perhaps the very plane we were to fly on (they only had 4).

But, as I said, the plane was overbooked, and we ended up in the group of about 10 that didn’t get onto the plane. Pistol-man was in the group that boarded the Y-7, and we didn’t see him again.

Fortunately for us, Lao Aviation had a solution to the problem. They simply rolled a second plane out of the hanger, a Harbin Y-12 this time, and fired it up.

Well, they tried to fire it up. It coughed and spluttered, and lots of black smoke poured out of the engines, but it didn’t start. Boh pen nyang. They pushed it back into the hanger, and rolled out yet another Y-12.

At this point I was feeling a little nervous.

The thired plane coughed and spluttered, poured out lots of black smoke, but it started! After a minute, they shut off the engines and asked us to board.

You can see in the picture below how part of the engine cowling is painted black. You can also see, if you look closely, how there are black smudge marks around that black painted area. Yeah. Smoke. I guess that the smoke mustn’t be a big problem, but it wasn’t inspiring at the time.

Lao_Aviation_Harbin_Y-12_Sibille-1

https://upload.wikimedia.org/wikipedia/commons/4/4b/Lao_Aviation_Harbin_Y-12_Sibille-1.jpg

Image © 2000 Regis Sibille, used under CCSA. (Another picture: http://www.airliners.net/photo/Iran—Revolutionary/Harbin-Y12-II/1503896/L/)

We rolled out and took off moments after the larger first plane. For a while, we could see the larger plane ahead and slightly above us – I don’t know why they didn’t go straight up to cruising altitude as the Y-7 is a lot faster than the Y-12. But eventually the Y-7 was out of sight. The scenery was in places spectacular. As I recall, the plane stayed in Lao airspace for the whole flight, despite this making the flight significantly longer.

Arriving in Savannakhet, we first travelled to the house of a friend of the computer store owner. This man happened to be one of the richest men in southern Laos. He had a beautiful house, filled with beautifully carved tables, paintings and collected antiques. After a brief meeting there, we were escorted by this man to a café in the city for a coffee. Well, some of us drank coffee; I didn’t. I was but 17 and at that age drank far too much Pepsi.

At this sidewalk café, an interesting encounter occurred, which has stuck with me. A street sweeper stopped and ordered a drink, and sat at the same table as this very wealthy man, and they struck up conversation. For some time, all at the table talked. The friendly interactions between two very different social classes was remarkable to me at the time – especially coming from Thailand where the social strata were clearly delineated.

Finally, social requirements met, we made our way to the computer with the problem. It was about 3 or 4 flights up dusty stairs in one of the tallest buildings in the city. There was a lift, but use of it was definitely not recommended. The problem was demonstrated to me, and I was able to observe there what had flummoxed me from afar. After just a few minutes, I realised what the problem was, and had enough information to fix it. I didn’t have my laptop with me so I had to write down some notes – and then we left them, a little sad that we couldn’t fix the problem immediately.

What was the problem? I actually don’t remember the detail. I just remember how we got there and back again!

We took a ferry across the river to Thailand, and took a bus to Nakhon Phanom. There were two reasons: first, my friend the computer shop owner wanted to visit some relatives there, and second, the roads in Laos were at the time in such poor condition that travel on them was best avoided if an alternative was available. Thai roads were busy but generally in excellent condition.

Once in Nakhon Phanom, it was a short motorcycle taxi ride to the relatives’ house by the river. We stopped there for a couple of hours and drank tea with them (yes, even I, Mr Pepsi Boy drank tea).

But when we tootled back to the bus stop, we found our bus had left a few minutes earlier than we had planned!

This was not a big problem. A bystander offered to chase down the bus in his pickup truck. This pickup was sparkling clean, had bright alloy wheels with a thin slick of rubber spray painted on, and a lowered chassis, so much so that the wheels would have been superfluous if they hadn’t been required for their locomotive capability.

pickup

It wasn’t this car, but it could well have been its older brother. Now in Thailand, the buses moved fast. Especially once they got out onto the open road. I’ve been on Thai buses doing 150 km/h or more.

So our intrepid young driver started chasing down the bus, flying down the city streets at over double the speed limit, braking hard for corners and easily avoiding the (fortunately) light traffic, and once he got out onto the open road he was eager to show us what his car would do. So here we were, flying down a Thai highway in a stranger’s car doing a ridiculous speed, chasing down a bus driver that didn’t know we existed. I’m pretty sure my parents would not have been pleased.

At the speed we were doing, we caught up to the bus pretty easily. After some vigorous flashing of the headlights, the bus’s left indicator came on, and it slowed and finally stopped. My friend paid the pickup race driver a token of our appreciation, and with slightly wobbly legs we climbed onto the bus, and off we went to Nong Khai.

We arrived in Nong Khai well after midnight, and ended up at a noisy Thai transit hotel, in a room without windows a couple of floors above the nightclub. One bed: a queen bed.

Now, remember, I was a callow 17 year old youth. The idea of sharing a bed with any man was pretty terrifying. But my friend, who I guess was in his 40s at that stage, kindly noted my abject and unwarranted fear, and gave me the whole bed – I think he sat in the chair and dozed. Not fair, I know.

At about 4am the nightclub finally quietened down, and at about 6am we were awoken by the daily noise that accompanies the start of every day – car doors slamming, shouts, trucks reversing. So we gave up on sleep, got up and made our way back across the border to Vientiane.

Upon returning to Vientiane, I fixed the problem in the Keyman code in a few minutes, prepared a new version on a floppy disk, and rode my bicycle over to the computer shop to deliver it. The computer shop sent the disk on to the user in Savannakhet, and as far as I know, that was that.

That’s how debugging used to work in the nearly olden days. None of these fancy remote desktop VPN SSH thingies. It was a lot more fun.