We ran into a problem recently with the MSHTML editor where empty paragraphs would collapse when the user saved or printed the document.  If  the user loaded the document again, the empty elements seem to disappear entirely.  Logically, this makes sense: an empty element has 0x0 dimensions, so will take up no space.  But if the user adds a blank line, they would not expect it to disappear after saving: MSHTML gives these new empty elements dimensions as if they contained a non breaking space, until the document is saved and reloaded.

Let’s look at that visually.  We’d type ONETWOTHREE into the editor, and all would be fine, as shown on the left.  After reloading, it would display as shown on the right:

Source HTML editor and collapsing HTML result

So what is going on, and what’s the solution?

The basic solution is to add an   entity (non-breaking space) to empty elements to give them a non-zero width and height.  MSHTML does this.  But there’s a heap of complexity around this.  From my analysis of the problem, it seems that the core issue is that element.innerHTML or domnode.childNodes.length is returning “” or 0 respectively when viewing the full source shows that there is actually an   in the element.  This happens only when the MSHTML editor is active.

The complexity arises when one looks at the different edit modes and methods of loading documents, because each mode has slightly different symptoms.  Whilst juggling the tangle of symptoms that these issues present, we also need to consider the following requirements:

1. A document may contain both empty

   elements and

    

   elements, and the editor must not conflate the two when loading.

2. When the user inserts a blank line, it must not collapse.  Nevertheless, the user does not want to learn about non-breaking spaces, so the editor must transparently manage this. Ideally, the non-breaking space would be hidden from the user but managed in the back end.  I must say that MSHTML is very close to this ideal.

A diversion: this is not a new problem, and many solutions have been proposed. One solution suggested in various forums online is to use
to break lines instead of the paragraph model with

or

.  This is not a great answer: it means the whole document has a single paragraph style.  I do note however that this is what Blogger and some other blog editors do, but then they do dynamically insert a

when the user changes the paragraph style.  Still not pretty.

Now where I talk about

elements, feel free to imagine that I am talking about

elements.  The behaviour appears to be much the same, and there’s just a flag to switch between the two elements. Also, I’ll be using Delphi for the code samples because it hides a lot of the necessary COM guff and makes the examples much easier to read.

When a new blank line is inserted into the editor, behind the scenes MSHTML will add an   entity to prevent the element from collapsing.  When you type the first letter, the   is deleted.  MSHTML also makes the   itself invisible to the user.  This is great.  It’s exactly what we want.

So let’s look at the activation of the editor and what is happening there. It turns out that there are three ways of making a document editable — four if you include the undocumented IDM_EDITMODE command that some editor component wrappers use.  So what are these four methods?

1. Set document.designMode to “On”.

     D := WebBrowser.Document as IHTMLDocument2;
     D.designMode := ‘On’;
    

2. Set document.body.contentEditable to “true” (or anyElement.contentEditable).

     D := WebBrowser.Document as IHTMLDocument2;
     (D.body as IHTMLElement3).contentEditable := ‘true’;
    

3. The DISPID_AMBIENT_USERMODE ambient property.  See the link for an example.
    
4. The aforementioned IDM_EDITMODE command ID.  I’m not condoning this method, just documenting it because some editor wrappers use it.

     D := WebBrowser.Document as IHTMLDocument2;
     (D as IOleWindow).GetWindow(hwnd_);
     SendMessage(hwnd_, WM_COMMAND, IDM_EDITMODE, 0);

To make things even more complicated, there are different ways of loading content into the HTML editor, and different methods have different outcomes.  The three methods we explored were using Navigate, document.write, and IPersistFile.

1. Using editor.Navigate to load either a local or remote document.

     WebBrowser.Navigate(DocFileName);
    

2. Using document.write to write a complete document.

     D := WebBrowser.Document as IHTMLDocument2;
     VarArray := VarArrayCreate([0, 0], varVariant);
     VarArray[0] := DocText;
     D.write(PSafeArray(TVarData(VarArray).VArray));
     D.close;
    

3. Accessing the editor’s IPersistFile interface to load a document.

     D := WebBrowser.Document;
     PersistFile := D as IPersistFile;
     PersistFile.Load(PWideChar(DocFileName), 0);

It turns out that if you use either Navigate or contentEditable, then MSHTML will not hide   from the end user for elements already in the document.  New empty elements typed by the user will still have the default behaviour described previously.  This is inconsistent and confusing to both me (the poor developer) and the end user.

The following table shows how   are treated in otherwise empty elements when loaded in the various ways:

	Navigate	document.write	IPersistFile
designMode	visible	invisible	invisible
contentEditable	visible	visible	visible
IDM_EDITMODE	visible	invisible	invisible
DISPID_AMBIENT_USERMODE	not tested

Now, it turns out that the visible results on that matrix are not going to do what we are looking for.  That’s because, as mentioned, any new empty elements typed by the user will still have the invisible behaviour for that empty element  .

Here’s a final example to clarify the situation.  Given the following document loaded into the MSHTML editor using document.write, and designMode = “On”.

 

 

With this result, we get the following results.

Code	Result
doc.getElementById(“a”).innerHTML
doc.getElementById(“a”).outerHTML
doc.getElementById(“a”).parentElement.innerHTML
doc.getElementById(“a”).parentElement.outerHTML
doc.documentElement.outerHTML

So. How do we determine if an element is empty and collapsed, or is a blank line?  MSHTML isn’t consistent with its innerHTML, outerHTML or DOM text node properties of the element.

But wait, all hope is not lost!  It turns out that IHTMLElement3 has a little buried property called inflateBlock.  This property tells you whether or not an empty element will be ‘inflated’ to appear as though it has content.  This little known property (I found no discussions or blogs about it!) should solve our problem neatly:

isElementTrulyEmpty := (element.innerHTML = ”) and not (element as IHTMLElement3).inflateBlock;

isElementJustABlankLine := (element.innerHTML = ‘ ‘) or ((element.innerHTML = ”) and (element as IHTMLElement3).inflateBlock);

Now I just have to push this fix into the HTML editor component wrapper we are using.  At least I’ve already written the documentation around the fix!

Final note: even the Blogger editor that I’m using to write this post has trouble with consistency with new lines.  Here’s an example — look at the spacing around the code samples.

A screenshot of this blog post, in the editor (Firefox)

A screenshot of this blog post, previewing (Firefox)

The print engine that we use for one product I work with is based on Internet Explorer’s custom print templates functionality.  This actually works really well, and gives us lots of flexibility and generally is pretty straightforward to use.  Unfortunately, we have run into a bit of a problem recently when trying to print documents in IE8 standards compliance mode: multiple page documents print blank pages, are missing content or sometimes even fail to start printing.

I have been able to reproduce this issue using Microsoft’s own printtemplates.exe example (link in Introduction of article), with only a couple of minor changes to trigger the problem.

Symptoms
The issue arises when all of the following specific conditions are met:

1. The document to be printed is in IE8 standards compliance mode.  This is mostly easily forced with the X-UA-Compatible META element:

   <META HTTP-EQUIV='X-UA-Compatible' CONTENT='IE=8' />
   

2. The document to be printed is greater than 1 page long.
3. The print template uses LAYOUTRECT elements of differing sizes.  This is a common requirement for letters which may have an address section or expanded letterhead at the top of the first page, but will have a much smaller letterhead on subsequent pages.

The Baseline
We’ll use the printtemplates.exe sample provided by Microsoft.  The sample Template3.htm in the example shows how to dynamically create LAYOUTRECT elements.  Here’s what happens before we make any changes to the example:

Print Template sample application, working with defaults

Print Preview, defaults, page 1

Print Preview, defaults, page 2

As shown in the screen shots, the print preview displays as expected.

Reproducing the Problem

With Internet Explorer 8, the problem can be reproduced as follows:

1. Start printtemplates.exe, and select Template3.htm.
2. Click the Page Source button.  In the HTML file that is opened, add the META element as the first child of the HEAD element.  This forces the page into IE8 standards compliant mode.  Save the file as sample.htm:

   <HTML>
     <HEAD>
       <META HTTP-EQUIV='X-UA-Compatible' CONTENT='IE=8' />
       <TITLE>Print Template Samples</TITLE>
     </HEAD>
   

3. Back in printtemplates, click the Template Source button.  In the HTML file that is opened, make the highlighted change to the OnRectComplete function.  This small change means pages other than the first page have a smaller LAYOUTRECT than the initial page.  Save this file as template.htm.

   function OnRectComplete()
   {
       if (event.contentOverflow == true)
       {
           document.all("layoutrect" + (iNextPageToCreate - 1)).onlayoutcomplete = null;
   
           newHTML  = "";
           newHTML += "STYLE='height:5.5in'/>";
           newHTML += "";
   
           pagecontainer.insertAdjacentHTML("beforeEnd", newHTML);
           iNextPageToCreate++;
       }
   }

4. Enter the full paths of sample.htm and template.htm into the respective fields in the print template sample, then press ENTER in the page field to load the modified sample.htm.

When you press Print Preview now, the print preview window will show blank pages instead of the expected content, and will have only 2 pages instead of the 5 or more that were shown previously:

The print template application with modified page and template ready to roll

Failing print preview, page 1

Failing print preview, page 2

If either of the changes are removed from the sample.htm and template.htm, the problem does not occur.  In Internet Explorer 9, the problem also occurs but appears to be resolved when using IE=9 in the META tag.  However, as Internet Explorer 9 is not available for Windows XP, this is not a viable solution for us.

Workarounds
We’ve found a few workarounds.  None of these are very viable for us but I list them for completeness:

1. Don’t use LAYOUTRECT elements of differing sizes.  This is a non-starter for us.
2. Don’t use IE8 standards compliant mode.  This would mean rewriting a number of reports and some complicated CSS to workaround bugs in IE7’s standards compliant mode, but may be a way forward.  Or use quirks mode with all the joy that brings.
3. Use IE9.

I’m writing up these notes in order to document what has been a long and painful process, involving much spelunking through MSHTML.DLL and IEFRAME.DLL to try and understand what Internet Explorer (or more accurately, the WebBrowser control) is doing and how to correctly use the semi-documented interfaces to provide full control over a print job.

The original requirement for this mini-project was to provide tray, collation, and duplex control for a HTML print job using IHTMLDocument2.execCommand(IDM_PRINT), with a custom print template.  These functions had been supported through a 3rd party ActiveX component, but this component proved to be incompatible with Internet Explorer 9 (causing a blue screen would you believe!), and the company providing the component was defunct, so it fell to me to re-engineer the solution.

After considerable research, I found some sparse documentation on MSDN suggesting that one could pass a HTMLDLG_PRINT_TEMPLATE flag to ShowHTMLDialogExand thereby duplicate and extend the functionality of the print template.  In particular, the __IE_CMD_Printer_Devmode property that could somehow be passed into this function would give us the ability to control anything we liked in terms of the printer settings.

Too easy.  Much too easy.  The first stumbling block was trying to discover the type of the pvarArgIn parameter to ShowHTMLDialogEx. A variant array seemed sensible but did not work.  It turns out that this needs to be an IHTMLEventObj, which can be created with IHTMLDocument4.CreateEventObject.  You can then use IHTMLEventObj2.setAttribute to set the various attributes for the object.

Then there were questions about what IMoniker magic was needed for the pMk parameter.  And more questions about the most appropriate set of flags.  Diving into the debugger to examine what Microsoft did answered both of these questions — it was a simple CreateURLMonikerEx call, no need to bind the moniker or other magic, and the flags that Microsoft used were HTMLDLG_ALLOW_UNKNOWN_THREAD or HTMLDLG_NOUI or HTMLDLG_MODELESS or HTMLDLG_PRINT_TEMPLATE for a print job, or HTMLDLG_ALLOW_UNKNOWN_THREAD or HTMLDLG_MODAL or HTMLDLG_MODELESS or HTMLDLG_PRINT_TEMPLATE for a print preview job.  Yes, that is both HTMLDLG_MODAL and HTMLDLG_MODELESS!

Next, what variant type should the __IE_BrowseDocument attribute be?  VT_DISPATCH or VT_UNKNOWN?  The answer is VT_UNKNOWN — things just won’t work if you pass a VT_DISPATCH.  I also came unstuck on the __IE_PrinterCmd_DevMode and __IE_PrinterCmd_DevNames attributes.  These need to be a VT_I4 containing an unlocked HGLOBAL that references a DEVMODEW structure.  I’ll leave the setup of the DEVMODEW structure to you: there are a lot of examples of that online.

However, even after overcoming these hurdles (with copious debugging to understand what MSHTML.DLL and IEFRAME.DLL were doing), there were other issues.  First, the print template was unable to access the dialogArguments.__IE_BrowseDocument property, with an Access Denied error thrown.  Also, HTC behaviors would fail to load as the WebBrowser component believed that they were being referenced in an insecure, cross-domain manner.  And finally, JavaScript in the page being printed was failing to run — and this JavaScript was required to render some of the details of the page.

I knew that Microsoft actually pass a reference to a temporary file for printing in the __IE_ContentDocumentURL attribute.  So I saved the file to a temporary file, which also required adding a BASE element to the header so that relative URLs in the document would resolve.  But the problems had not gone away.

All three of these problems in reality stemmed from the same root cause.  The security IDs for the various elements — the print template, the document being printed, and the HTC components — were not matching.  So I embarked on an attempt to find out why.  At first I wondered if we needed to bind the moniker to a bind context or storage.  That was a no-go.  Then I looked at the IInternetSecurityManager interface, which a developer can implement to provide custom security IDs, zones and more.  Sounds logical, right?  Only problem is that the ShowHTMLDialogEx function provides its own IInternetSecurityManager implementation, which you cannot override (and its GetSecurityID just returns INET_E_DEFAULTACTION for the relevant URLs).  Yikes.

I was starting to run out of options.  As far as I could tell, we were duplicating Microsoft’s functionality essentially identically, and I could not see any calls which changed the security for the document so that it would match security contexts.

Finally I noticed an undocumented attribute had been added to the HTML element in the temporary copy of the page: __IE_DisplayURL.  And as soon as I added that to my file, referencing the original URL of the document, everything worked!

Now, this is all fun (and sounds straightforward in hindsight), but without some code it’s probably not terribly helpful.  So here’s some code (in Delphi, translate to your favourite language as required).  It all looks pretty straightforward now(!), but nearly every line involved blood, sweat and tears!  This is really not a complete example and hence does not compile but just covers the bits necessary to complement the better documented aspects of custom printing with MSHTML.  Please note that this example uses the TEmbeddedWB component for Delphi, and that temporary file cleanup has been excluded.

procedure THTMLPrintController.StartPrint(FPrint: Boolean);
var
  FDeviceW, FDriverW, FPortW: WideString;
  FDevModeHandle, FDevNamesHandle: HGLOBAL;
  pEventObj2: IHTMLEventObj2;
    procedure SetTempFileName;
    begin
      FTempFileName := GetTempFileName('', '.htm');
    end;
    { SaveToFile: Saves the current web document to a temporary file, adding the required BASE and HTML properties }
    procedure SaveToFile;
    var
      FElementCollection: IHTMLElementCollection;
      FHTMLElement: IHTMLElement;
      FBaseElement: IHTMLBaseElement;
      FString: WideString;
    begin
      FElementCollection := webBrowser.Doc3.getElementsByTagName('base');
      if FElementCollection.length = 0 then
      begin
        FBaseElement := webBrowser.Doc2.createElement('base') as IHTMLBaseElement;
        FBaseElement.href := webBrowser.LocationURL;
        (webBrowser.Doc3.getElementsByTagName('head').item(0,0) as IHTMLElement2).insertAdjacentElement('afterBegin', FBaseElement as IHTMLElement);
      end
      else
      begin
        FBaseElement := FElementCollection.item(0,0) as IHTMLBaseElement;
        if FBaseElement.href = '' then FBaseElement.href := webBrowser.LocationURL;
      end;
      FElementCollection := webBrowser.Doc3.getElementsByTagName('html');
      if FElementCollection.length > 0 then
      begin
        FHTMLElement := FElementCollection.item(0,0) as IHTMLElement;
        FHTMLElement.setAttribute( '__IE_DisplayURL', webBrowser.LocationURL, 0);
      end;
      with TFileStream.Create(FTempFileName, fmCreate) do
      try
        if webBrowser.Doc5.compatMode = 'CSS1Compat' then
        begin
          FString := '';
          Write(PWideChar(FString)^, Length(FString)*2);
        end;
        FString := webBrowser.Doc3.documentElement.outerHTML;
        Write(PWideChar(FString)^, Length(FString)*2);
      finally
        Free;
      end;
    end;
    { Configured the printer, assuming we've already been passed an ANSI handle }
    procedure ConfigurePrinter;
    var
      FDevice, FDriver, FPort: array[0..255] of char;
      FDevModeHandle_Ansi: HGLOBAL;
      FPrinterHandle: THandle;
      FDevMode: PDeviceModeW;
      FDevNames: PDevNames;
      FSucceeded: Boolean;
      sz: Integer;
      Offset: PChar;
    begin
      Printer.GetPrinter(FDevice, FDriver, FPort, FDevModeHandle_Ansi);
      if FDevModeHandle_Ansi = 0 then
        RaiseLastOSError;
      FDeviceW := FDevice;
      FDriverW := FDriver;
      FPortW := FPort;
      { Setup the DEVMODE structure }
      FSucceeded := False;
      if not OpenPrinterW(PWideChar(FDeviceW), FPrinterHandle, nil) then
        RaiseLastOSError;
      try
        sz := DocumentPropertiesW(0, FPrinterHandle, PWideChar(FDeviceW), nil, nil, 0);
        if sz < 0 then RaiseLastOSError;
        FDevModeHandle := GlobalAlloc(GHND, sz);
        if FDevModeHandle = 0 then RaiseLastOSError;
        try
          FDevMode := GlobalLock(FDevModeHandle);
          if FDevMode = nil then
            RaiseLastOSError;
          try
            if DocumentPropertiesW(0, FPrinterHandle, PWidechar(FDeviceW), FDevMode, nil, DM_OUT_BUFFER) < 0 then
              RaiseLastOSError;
            FDevMode.dmFields := FDevMode.dmFields or DM_DEFAULTSOURCE or DM_DUPLEX or DM_COLLATE;
            FDevMode.dmDefaultSource := FTrayNumber;
            if FDuplex
              then FDevMode.dmDuplex := DMDUP_VERTICAL
              else FDevMode.dmDuplex := DMDUP_SIMPLEX;
            if FCollate
              then FDevMode.dmCollate := DMCOLLATE_TRUE
              else FDevMode.dmCollate := DMCOLLATE_FALSE;
            if DocumentPropertiesW(0, FPrinterHandle, PWideChar(FDeviceW), FDevMode, FDevMode, DM_OUT_BUFFER or DM_IN_BUFFER) < 0 then
              RaiseLastOSError;
            FSucceeded := True;
          finally
            GlobalUnlock(FDevModeHandle);
          end;
        finally
          if not FSucceeded then GlobalFree(FDevModeHandle);
        end;
      finally
        ClosePrinter(FPrinterHandle);
      end;
      Assert(FSucceeded);
      { Setup up the DEVNAMES structure }
      FSucceeded := False;
      FDevNamesHandle := GlobalAlloc(GHND, SizeOf(TDevNames) +
       (Length(FDeviceW) + Length(FDriverW) + Length(FPortW) + 3) * 2);
      if FDevNamesHandle = 0 then RaiseLastOSError;
      try
        FDevNames := PDevNames(GlobalLock(FDevNamesHandle));
        if FDevNames = nil then RaiseLastOSError;
        try
          Offset := PChar(FDevNames) + SizeOf(TDevnames);
          with FDevNames^ do
          begin
            wDriverOffset := (Longint(Offset) - Longint(FDevNames)) div 2;
            Move(PWideChar(FDriverW)^, Offset^, Length(FDriverW) * 2 + 2);
            Inc(Offset, Length(FDriverW) * 2 + 2);
            wDeviceOffset := (Longint(Offset) - Longint(FDevNames)) div 2;
            Move(PWideChar(FDeviceW)^, Offset^, Length(FDeviceW) * 2 + 2);
            Inc(Offset, Length(FDeviceW) * 2 + 2);
            wOutputOffset := (Longint(Offset) - Longint(FDevNames)) div 2;
            Move(PWideChar(FPortW)^, Offset^, Length(FPortW) * 2 + 2);
          end;
          FSucceeded := True;
        finally
          GlobalUnlock(FDevNamesHandle);
        end;
      finally
        if not FSucceeded then GlobalFree(FDevNamesHandle);
      end;
      Assert(FSucceeded);
    end;
  { Creates the IHTMLEventObj2 and populates the attributes for printing }
  procedure CreateEventObject;
  var
    v: OleVariant;
    FShortFileName: WideString;
    FShortFileNameBuf: array[0..260] of widechar;
  begin
    v := EmptyParam;
    pEventObj2 := webBrowser.Doc4.CreateEventObject(v) as IHTMLEventObj2;
    pEventObj2.setAttribute('__IE_BaseLineScale', 2, 0);
    GetShortPathNameW(PWideChar(FTempFileName), FShortFileNameBuf, 260); FShortFileName := FShortFileNameBuf;

    v := webBrowser.Document as IUnknown;
    pEventObj2.setAttribute('__IE_BrowseDocument', v, 0);
    pEventObj2.setAttribute('__IE_ContentDocumentUrl', FShortFileName, 0);
    pEventObj2.setAttribute('__IE_ContentSelectionUrl', '', 0);  // Empty as we never print selections
    pEventObj2.setAttribute('__IE_FooterString', '', 0);
    pEventObj2.setAttribute('__IE_HeaderString', '', 0);
    pEventObj2.setAttribute('__IE_ActiveFrame', 0, 0);
    pEventObj2.setAttribute('__IE_OutlookHeader', '', 0);
    pEventObj2.setAttribute('__IE_PrinterCMD_Device', FDeviceW, 0);
    pEventObj2.setAttribute('__IE_PrinterCMD_Port', FPortW, 0);
    pEventObj2.setAttribute('__IE_PrinterCMD_Printer', FDriverW, 0);
    pEventObj2.setAttribute('__IE_PrinterCmd_DevMode', FDevModeHandle, 0);
    pEventObj2.setAttribute('__IE_PrinterCmd_DevNames', FDevNamesHandle, 0);
    if FPrint
      then pEventObj2.setAttribute('__IE_PrintType', 'NoPrompt', 0)
      else pEventObj2.setAttribute('__IE_PrintType', 'Preview', 0);
    pEventObj2.setAttribute('__IE_TemplateUrl', GetPrintTemplateURL, 0);
    pEventObj2.setAttribute('__IE_uPrintFlags', 0, 0);
    v := VarArrayOf([FShortFileName]);
    pEventObj2.setAttribute('__IE_TemporaryFiles', v, 0);
    pEventObj2.setAttribute('__IE_ParentHWND', 0, 0);
    pEventObj2.setAttribute('__IE_HeaderString', webBrowser.Doc2.title, 0);
    pEventObj2.setAttribute('__IE_DisplayURL', webBrowser.LocationURL, 0);
  end;
  procedure InstantiateDialog;
  var
    FWindowParams, FMonikerURL: WideString;
    FMoniker: IMoniker;
    FDialogFlags: DWord;
    varArgIn, varArgOut: OleVariant;
    res: HRESULT;
  begin
    varArgIn := pEventObj2 as IUnknown;
    varArgOut := Null;
    FMonikerURL := GetPrintTemplateURL;
    OleCheck(CreateURLMonikerEx(nil, PWideChar(FMonikerURL), FMoniker, URL_MK_UNIFORM));
    if FPrint then
    begin
      FWindowParams := '';
      FDialogFlags := HTMLDLG_ALLOW_UNKNOWN_THREAD or HTMLDLG_NOUI or HTMLDLG_MODELESS or HTMLDLG_PRINT_TEMPLATE;
    end
    else
    begin
      FWindowParams := 'resizable=yes;';
      FDialogFlags := HTMLDLG_ALLOW_UNKNOWN_THREAD or HTMLDLG_MODAL or HTMLDLG_MODELESS or HTMLDLG_PRINT_TEMPLATE;
    end;
    res := ShowHTMLDialogEx(0, FMoniker, FDialogFlags, varArgIn, PWideChar(FWindowParams), varArgOut);
    if res <> S_OK then raise EOSError.Create(SysErrorMessage(res));
  end;
begin
  SetTempFileName;
  SaveToFile;
  ConfigurePrinter;
  CreateEventObject;
  InstantiateDialog;
end;

Update 14 July: This code is not our production code: I’ve stripped out bits and pieces and tried to keep the bits that are somewhat relevant. Don’t worry too much about the ConfigurePrinter details — the takeaway is the HGLOBAL. I must also apologise for the atrocity that is the SaveToFile function. That’s what you get when working with legacy versions of software. Internet Explorer also won’t reliably work with non-ASCII content there unless you toss a BOM into the start of the stream.