Under the Hood: Find in the Word Web App Viewer

As an engineer, I find myself frustrated with product blogs that spend all their time discussing the end-user experience while ignoring specific implementation details. For me, the most interesting classes in college were always the ones where guest lecturers would discuss specific challenges in developing their application, how difficulties were overcome, and why specific design decisions were made. With that in mind, I decided to talk a little about developing the Find feature in the Word Web App viewer.

Using Find

As explained in the following section, we do not render Word documents using plain HTML. As a result, your browser's native find is unhelpful when viewing documents. Rather than leave our customers without the ability to search the document they are currently reading, we chose to write a find implementation based on Word 2010’s new find experience. Using find works the same in both the Silverlight and non-Silverlight versions of the Word Web App viewer. Click the “Find” button in the toolbar, press CTRL+F, or press the F3 key to open the “Find in Document” pane. We will attempt to automatically place your insertion point in the search box. Enter the word (or sentence) which should be located in the document and press the enter key or click on the magnifying glass. Your document will be searched to find words (or phrases) that match, and the results (along with some context) will be given. You can navigate through the results with the up and down arrow keys or by clicking on the result you wish to navigate to. If a result which is not currently in view is selected, we will scroll to it. Hit highlights will be placed over the result in the document, with the currently selected result being given a different color to make it easily locatable.

Compare Find (Word 2010 vs. Word Web App viewer)

Before posting any pictures, I’d just like to take a moment and remind our readers, that images posted on this blog are from internal builds and the final appearance of the any product shown (including the Word Web App viewer) may change before being released.

The Word Web App is on the left and Word 2010 is on the right…

Behind the Scenes Data

As mentioned in previous posts, the primary goal of the Word Web App viewing experience is to make sure the document looks exactly as intended when viewed in the browser. To achieve this high level of fidelity, the contents of the document are displayed using XAML (if you have Silverlight installed) or images (if you don’t), rather than plain HTML. Unfortunately, by using XAML/images instead of HTML, your browser's native find does not work.

In addition to sending down the content of the document (XAML or images), we also send down some metadata. This metadata, sent in the form of a XML document, contains the bounding box for every line on each page, as well as that line’s text. We can then search this text and highlight the appropriate areas on the image/Silverlight canvas. As a side effect, this XML is often already downloaded for selection, allowing us to find search results without making an additional round-trip to the server.

To allow Silverlight to render documents identically to Word, each glyph is manually positioned on a page. This provides the added benefit of allowing perfect hit highlighting. In the non-Silverlight experience, though, hit highlighting is done by estimating the result’s position based on the match’s location in your line of text, and the line’s bounding box. This is why Gareth mentioned that Silverlight provides “improved accuracy of hit highlighting in Find.”

Compare Hit Highlighting (with and without Silverlight)

Find Without Silverlight…

Find With Silverlight…

Challenges in Highlighting Results

There was an internal discussion about whether or not we should improve the non-Silverlight experience by also sending down every character’s position on each page. Ultimately, we decided that since Silverlight typically required sending down fewer bytes to your browser, it did not make sense to further increase the divide solely for improved hit highlighting. At the same time, we felt that giving an approximate location of the result was better than eliminating the highlights entirely, as it attracts your eyes to roughly the correct location.

Some challenges that we faced included getting highlighting to work with text that is right-to-left , vertical, or a combination of the two. This required modification of the metadata to include the angle of the text line (0, 90, or 270), and the text's direction (left-to-right or right-to-left). In each of these cases, we are not highlighting actual text. Highlights are semi-transparent colored rectangles that float above the document.

Say we have a right-to-left text line that is rotated 90 degrees clockwise, and we know the hit covers the 2nd through 5th character in a line that is 25 characters long. Where should the highlight box go, and what should the dimensions be? In Silverlight, we know the widths of the characters thanks to the XAML. The height of the highlight box should be the summed width of the characters as given by the XAML. Why height? Because the text is rotated 90 degrees. We also know that the width of the highlight box is the width of the bounding box of the text line. Finally, since the text is right-to-left, the highlight box starts from the bottom. Why from the bottom? When rotating left-to-right text 90 degrees right, the first character will appear at the top. Because our line is right-to-left, the highlighting starts from the bottom.

The non-Silverlight experience is similar, only for this case the height is calculated differently. Because we don't know individual character widths, we treat the bounding box as containing 25 equally wide characters. Therefore the height is 4/25 times the height of the bounding box.

(I could talk about the multiple-characters-to-a-single-glyph issue, but that may get too technical...)

Challenges in Capturing Hot Keys

To enable the find experience described above, certain key combinations need to be overloaded. For example, if CTRL+F opened the browser’s find box instead of our custom find pane, customers would be unable to locate their query in the document. For this reason, we attempt to capture and suppress the find hotkeys before they reach your browser. Unfortunately, Safari 4.0.3 does not allow the CTRL+F combination to be easily suppressed. At the time of writing, we succeed in detecting when this key combination is pressed and opening our find pane, but we are unable to prevent the browser find from opening as well. Furthermore, Safari places the browser’s focus in its own find toolbar, so you’ll need to manually set focus in our application’s find to begin searching documents in the Word Web App viewer. I hope that further investigation into this area will allow us to succeed in preventing Safari's find toolbar from opening.

I hope this gave you a better idea of some of the challenges we faced when implementing the Find feature.

Robert Rolnick and Carrie Butler
Software Development Engineers, Office Web Apps