Back
News

Using the new Office with touch

Editor’s note: Windows
8 provides a number of platform capabilities for enabling highly responsive
touch support in applications, ranging from hardware accelerated graphics and
improved touch targeting to the a new app platform that makes it easy to build
touch-optimized Windows 8-style apps. The new Office takes advantage of these
to deliver great new touch-based experiences on Windows 8. Clint Covington, a lead program manager for our User Experience
team authored this post. We’re hearing reports of IE10 users not successfully viewing the videos. If you are unable to play the videos, please visit the Office Next page on Facebook. http://www.facebook.com/MicrosoftOfficeNext . We’ve also updated the post to point to higher quality videos on Office channel on YouTube, and these are compatible with IE10 (both modes).

Clint Covington

On Monday in San Francisco we took the wraps off of the new
Office’s touch experience designed for Windows 8. We showed the new touch-optimized
Windows 8-style app for OneNote, and we showed how we’ve touch-enabled Word,
Excel, PowerPoint, Outlook, and other apps on the desktop. The new Office is
designed for a great experience whether you’re sitting on a couch with a
tablet, or at a desk with a mouse and keyboard. It makes common tasks fast,
fluid, and intuitive, while still enabling the rich capabilities required to
create high-quality documents.  In this
post I’ll walk you through the thinking, engineering process and design
framework we used to reimagine these experiences for touch.

 

Our vision for touch

When we started planning the next version of Office, it was
clear that touch was going to be a big part of how people used Office in the
future, and we were excited about the opportunity to add a delightful new
dimension to our apps. Hardware was evolving rapidly to enable new types of PCs
and mobile devices, and along with them new styles of interaction, new physical
postures, and new usage locations. We believed people would continue to use
Office at their desks, and type on physical keyboards at a desktop or laptop,
but would also start to use tablets and hybrid laptop/tablet devices in a
variety of other situations: leaning back in a chair, leaning forward on a
train, bus, or plane, sitting on a couch, lying in bed, standing and holding
with two hands, or walking down the hall.

We analyzed what kinds of tasks were comfortable to do in
each of the postures, and what kinds of input were best for different kinds of
tasks. For example, a physical keyboard is optimal for large amounts of typing
(still significantly faster than an on-screen keyboard for most people). A
mouse is optimal for precise targeting, and touch is great for broad strokes
such as scrolling and zooming. In some postures, such as standing, touch is the
preferred input, while the efficiency of typing at a desk is hard to beat. Each
posture and input characteristic is great for some jobs and not as good for
others. We wanted to make sure Office apps felt intuitive, natural and
comfortable as across different postures and different kinds of input.

In parallel, Windows was introducing deep investments in
touch, ranging from new touch digitizer requirements for touch-enabled PC
hardware, through new touch drivers, new platform APIs, and a new user
experience. This platform provided us with the core capabilities to deliver a
natural experience across our applications and the new usage postures. (As a
side note, many of Office’s new touch capabilities will work on touch machines
running Windows 7, but improvements in the underlying platform in Windows 8
make the experience substantially better, improving everything from the touch
targeting accuracy – making it easier to tap the button you meant to tap – to the
button sizes on the screen, to the speed and responsiveness of the touch
feedback.)

We’ve invested in touch in two ways for the next version of
Office. The first is that we’ve begun to build new versions of some of our apps
(OneNote and Lync) specifically tailored for the new experience in Windows 8.
For these, we deliberately started from scratch to design new versions that
were touch-first. The second is that we’ve touch-enabled the familiar desktop
versions of all of the Office apps. These continue to be designed first and
foremost for mouse & keyboard, and to preserve the features & layout
you’re used to, but now support touch throughout the experience. This dual
approach allows you to immediately get the full power of Office on a touch
machine with no relearning required, along with new experiences that embrace
touch from the ground up and deeply integrate with the new experiences in
Windows 8.

The scope of our vision meant we would be making broad changes
across all of the Office applications. We started by identifying the heavily
traveled paths through each app and using these to focus our efforts.  Some examples were working with mail, sending
an instant message, reading a document, editing a spreadsheet, and presenting a
slide deck. We wanted to ensure that each of these felt great end-to-end using
only touch, or a mixture of touch, mouse, and keyboard.

Guidelines for touch

The next step was to develop a common framework for how we
would enable touch across Office. After comparing our current experiences with
where we wanted to go, we came up with the following areas of investment:

  • Touch responsiveness
  • Touch targeting
  • Selecting text & objects
  • Typing
  • Commanding

We wrote detailed guidelines for each of these areas and
established minimum bar expectations for how the work should come together. We
worked closely with people across the company to make sure the internal touch
language is consistent across Microsoft experiences and cultivate learning
transfer. We built out scorecards for each scenario and identified areas of
work that needed to happen to achieve a consistent, responsive and delightful
experience that leaves people touching with confidence. Here is an example of
one of our early scorecards for the work with mail scenario:

Let’s take a look at each of these guidelines.

Touch responsiveness

When you touch something, it needs to immediately respond. Content needs to “stick to your finger” when
you pan and zoom. The UI needs to respond with inertia and bounce when flicked
or reach the top/end of sections. Pictures need to immediately resize as you
drag your finger, and text needs to reflow around it to give the user a sense
of physical realness to the action.

To build an experience where content sticks to a finger, we
had to refactor much of the document surface to use a compositor where content
is rendered into images and moved around on the screen with animations. This
required us to migrate away from GDI to modern hardware accelerated graphics
services. The result of this work is an experience that feels natural and
responsive. Here are some examples:

 

 

Targeting

Targeting refers to successfully touching the thing you’re
trying to touch. Targeting is almost entirely about raw physical size. Fingers
are much bigger than mouse pointers or pen tips, so things need to be
physically larger on the screen in order to be comfortable to touch. The
Windows team did extensive research to develop guidelines for hit target sizes,
which we used throughout our designs.

The new Windows 8-style apps for OneNote and Lync were
designed from scratch, so we were able to ensure that every part of the
experience was easy to touch right out of the gate. In the desktop apps, many
of the typical hit targets designed for mouse and keyboard did not meet these
guidelines, so we had to figure out how to resize them. We identified two types
user interface elements that needed to be adjusted larger:

  • Fixed parts
    of the user interface that are always visible, such as the Quick Access Toolbar (QAT), status bar, ribbon, and folders in
    Outlook
  • Contextual
    parts of the user interface that appear
    in response to a user action, such as context menus and the Minibar

 

Fixed user interface elements

Many fixed user interface elements fell well below the
minimum size to be touchable. It was not possible to reliably touch common
commands like the Save button in the QAT. So we added a new feature called
Touch Mode that increases the size of fixed UI. Touch Mode is automatically
turned on for properly configured tablet machines. It increases the size of the
QAT, Ribbon tabs, adds spacing around small buttons in the ribbon, increases
the height of the status bar, turns on the Outlook touch triage action bar and
adds space to expanded folders in Outlook. Here is an example of a portion of
the Word ribbon with Touch Mode on and off:

Touch Mode Off:

Ribbon with Touch Mode Off

 

Touch Mode On:

Ribbon with Touch Mode On

You can turn on Touch Mode by clicking on the QAT overflow
and selecting the Touch Mode icon.
The icon then will show up in the QAT and can easily toggle Touch Mode on and
off.

 

 

Contextual user interface elements

A lot of Office’s contextual UI (menus, context menus, color
pickers, mini-tool bar, etc) contains small hit targets as well. Since
contextual UI isn’t visible until you take an action, we can decide whether to
show a touch or mouse-optimized version based on whether the action was a touch
or a mouse click. So contextual UI can always be the right size regardless of
whether you’re in touch mode. For example, if a context menu was invoked through
touch, we spaced items in the menu to make it more comfortable to touch. Here
is an example of how the color picker renders differently for touch verses the
mouse:

Mouse Color Picker       Touch Color Picker

For shapes, charts and pictures we increased the size of the
grab handles and used Windows touch targeting APIs to make it easier for users
to select and resize objects.

Mouse Grab Handles    Touch Grab Handles

New Windows 8 APIs tell us the shape of a finger on the
screen and the first contact point. If the first contact point is not inside
the grab handle, we have the opportunity to detect if a finger overlapped it
and ensure the selection is successful. The end result is a finger feels more
precise and people miss-tap less. These touch targeting APIs are used across
Office to help users touch with confidence through more accurate determination
of user intent for sloppy or imprecise touches. See the Touch
hardware and Windows 8
post on the Windows
8 Engineering blog
for more detail.

Positioning an object by dragging

One special case of targeting is grabbing and object and
dragging it. When you are simply dragging something onto another object, for
example, dragging an e-mail onto a folder in Outlook, the difficulty of
targeting the folder while dragging isn’t much different from trying to simply
tap on it. (It’s actually a bit easier.) However, when you’re dragging an
object in order to position it, it’s hard to get the same precision you get
with a mouse. To help with this, we’ve added guidelines and snap points that
appear automatically when moving or resizing pictures, shapes and other
objects. This makes it easy to line them up with margins or other objects.

 

 

Text and object selection

One of the most fundamental actions in Office is selection
of text and objects. Mice and keyboards make selection easy with precise
targeting & modifiers such as the shift and control keys. A finger lacks
the precision required to do these operations efficiently, so we worked with
the Windows team to develop a series of selection patterns for text and
objects.

We added new text selection handles to Word, PowerPoint,
Excel, OneNote, Visio, Lync and messages in Outlook. Here are a couple of examples
from Word and Excel:

These are the same text selection handles you see across
Windows 8 in apps like the new modern browser.

To select multiple objects, we followed the Windows
guidelines for cross-slide to select multiple slides and drag and drop to
rearrange. You can easily select multiple objects in PowerPoint by keeping one
finger down and tapping on other shapes and pictures.

 

 

Typing

Typing is a part of many scenarios in Office. Windows 8 made
a large investment in its on-screen or “virtual” keyboard. In the new Windows
8-style apps for OneNote and Lync, the virtual keyboard appears and disappears
automatically, leveraging APIs provided by Windows.

In the desktop apps, we needed to do some more work:

  • The keyboard didn’t appear automatically when
    you needed it
  • It sometimes covered what you were typing
  • You had little space left to work while it was visible

Invoking the keyboard

With Office 2010 and Windows 7, people have to manually
invoke the keyboard through clicking on the keyboard icon in the Windows task
bar. Windows 8 provides new desktop APIs that allow Office to hide and show the
virtual keyboard automatically.

Keyboard covers what you’re typing

Few things cause more frustration in touch than when the
virtual keyboard shows up and overlaps content or the area where you are
typing. It is even more frustrating when as you type, the cursor scrolls under
the keyboard. Windows 8 includes new desktop APIs that inform our applications
when the keyboard is coming and going, as well as where it is on the screen.
This lets our applications scroll content out of the way and keep the cursor in
view as you type.

Working space when the keyboard is visible

We found there was little space to work with content when
the ribbon is expanded and the virtual keyboard is up. This leaves users
feeling cramped and frustrated. To solve this problem, we default the ribbon to
collapsed mode on tablet machines. This gives users more space to work. We
introduced a number of subtle refinements to the interaction model of the
ribbon to make it work even better in collapsed mode. You will notice that it
now is possible to execute multiple commands at a time (such as Bold, Italic
and Underline). Yet, when you execute a command that is highly unlikely to be
followed up by another ribbon click, it goes away (e.g. Insert Table).

We also build a full screen view that allows users to hide
the ribbon and status bar so they can focus on content and see even more lines
of text or rows of cells without the chrome taking away from valuable content
space. We will provide more details about full screen view in a subsequent
post.

Minimize the need to type overall

The virtual keyboard is not as efficient for typing text as
a physical keyboard. Many of our scenarios and new postures required users to
use the virtual keyboard to complete a job, and we wanted to reduce traffic to
the virtual keyboard where ever possible.

Common patterns include suggestion menus and most recently
used (MRU) lists that remember previous input. For many years, applications
remember paths to recently opened documents. This used to work well, as people
primarily used one computer. With the use of Smart Phones, Tablets and the
cloud, people need to access documents across different machines, at home and
at work. If you are connected to the Office service, your document and common
locations MRU will roam across all of your devices. This significantly reduces
traffic to the file open and file save dialogs which invariably involves typing
in hard to remember, archaic file paths.

Excel’s new Flash Fill is another feature that saves you
time because Office does the typing for you. Here’s how it works:

 

 

Commanding

The final area of focus for touch is the command experience.

In the desktop apps, it was nearly impossible to complete
any of our scenarios without using the Ribbon, Backstage, Context Menus or MiniBar.
Additionally, we know from the Customer Experience Improvement Program that
many commands are executed through the keyboard. Some of the most common
commands like Cut, Copy, Paste, and Delete are most frequently executed through
the keyboard and not the application’s command interface. Here are a few of the
guiding principles we came up with as we refined the commanding experience:

  • Common commands are easy to find and fast to use
  • Users locate commands in consistent places
    between touch and mouse/keyboard experiences
  • Frequent keyboard actions
    (cut/copy/paste/delete) are easily accessible
  • The user experiences feels familiar and
    touchable

As we evaluated our scenarios, we found that most would be
accomplished with a small subset of commands. When people use Office with touch,
we expect them to work on different tasks than sitting at a desk with a mouse
and keyboard. We also found for many contexts (selected text, picture, shape,
table, etc) a small number of commands make up the majority of actions. As
mentioned earlier, the vertical real estate becomes a scarce commodity. All of
these insights lead us to want to provide users with quick access to common
commands while keeping enough vertical real estate to work with their content.

In addition, in designing our new apps for Windows 8, we
aimed to be consistent with the goals and design language of all of the other
Windows 8 apps that are being developed and deliver an experience optimized to
help you get your work done. Most importantly, we expect you to be able to
focus on your work and we want the Office user interface to get out of the way.
Windows 8 is designed so that apps have control of every bit of real estate on
the screen and we want to pass that real estate on to you.  This gives you
the room you need to work – to read, to type, to move around and see your
content.  When you do need to take an action, then the commands that you
need are just a click, tap, or swipe away.

MiniBar

Office 2007 introduced the concept of a MiniBar
-
a small strip of commands that show up when you select text or certain objects.
The minibar was designed to provide efficient access to frequent commands. For
Office 15, we extended the minibar to support our touch scenarios.

In the desktop apps, we changed it to show up when users tap
on selected items like text, cells, images and shapes. We added common keyboard
commands such as Paste, Cut, Copy and Delete to decrease traffic to the virtual
keyboard and help people accomplish scenarios much faster. Here the MiniBar for
a picture:

For second-tier commands, there is also a contextual menu
drop down from the MiniBar. These are the less frequent commands, but makes the
common right click actions with a mouse and keyboard also available with touch.
When the contextual menu shows, it does not repeat commands already on the
minibar (such as cut, copy and paste). Here is the same picture with the
context menu expanded:

In-Canvas Commands

Communicate with others is one of our most important
scenarios for tablets because we expect users to spend lots of time in Outlook
triaging mail. We found for mail there were a small set of commands that
dominate usage. These frequent commands include Reply, Reply All, Forward, Delete,
Move to Folder, Flag and Mark as Unread.

Originally we exposed these options in the mail message list.
As we started using Outlook on tablets, it became clear the posture of the
usage was different. It was extremely difficult to target a delete button in
the middle of the screen when in the preferred lean forward posture while
holding the device with two hands. In this position it is much easier to use
thumbs to target real estate along the edges.
Here is how the thumb bar works in Outlook:

 

Radial menus

Your tools at your fingertips. That’s the goal behind
the radial menus that
are available while you are typing and editing in the new OneNote app for
Windows 8. The commands that we expect you to use most while editing, we
make available next to your content.  While
you are working, your commands stay tucked behind the menu.  But when you
need to format your text, for example, just click or tap on the radial menu and
the commands you need appear:

 

The radial menu is designed so that the most common commands
you need are available immediately.  Then if you need more commands, you
can just swipe your finger out from the center of the menu or tap on any of the
arrows around the menu.  For instance, more formatting commands are
available right there next to bold:

 

You’ll also notice that when the commands are tucked away,
the radial menu changes based on what you are doing.  For example, if you
are typing and need to insert a table or a bulleted list, you’ll see:

However, if you are working with images, you’ll see:

Under each menu are the commands or tools that you’ll need
at that time. And you don’t even need to expand the menu to select the 8 most
common actions. You can just swipe the radial menu in the direction of the
command you want. For example, if you have some text selected, swiping right on
the collapsed radial menu will bold text, swiping up will copy, and swiping left
will undo. They’re like keyboard shortcuts for touch. These quick swipes are just
one more way radial menus can make touch interactions more fast and fluid,
without getting in the way of your content.

 

App Bar

Finally, like all other Windows 8 style apps, we expose the
commands that you need to navigate within Lync and OneNote in an app bar at the
bottom of the screen.  Just swipe in from
the edge of the screen or right click (or press the Windows key + Z) and an app
bar slides in:

In this screen shot from OneNote, you can see that you can
move back and forward through your notes, bring up your list of notebooks or
pages, add a new page, or give us feedback on the app (please do!).

The app bar will also change based on what you right clicked
on.  For example, if you right click on a section within an OneNote
notebook, additional commands will appear on the left side of the app bar.
Now you can also rename or delete the section or copy a link to that section:

 

Summary

We are excited about the opportunity to enable a whole new
set of usage postures for Office, from the bus to the plane to the couch to the
hallway. We are excited how with touch common tasks are fast, fluid and
intuitive while still enabling the rich capabilities required to create
high-quality documents. We hope you enjoy using the new Office as much as we
did building it. There is still a bit of touch polish work left but I encourage
you to download the preview, install
OneNote MX
, and please give us feedback. We look forward to your comments!
Thanks

 

 

 

 

Join the conversation

29 comments
  1. he I would like to know whether Microsoft has enabled the capability of embedding documents / presentations (especially presentations) to websites. with html5 it is not an bigdeal right?

    • If I understand your comment, you can already do that today using SkyDrive / Office web apps… at least for Excel & PowerPoint. In the next major update you will be able to embed Word documents as well. And I imagine there will be some enhancements to the current embed capabilities. In Excel there is a new Form capability so that could be very interesting to some websites depending on how it is implemented (ex. can we just display the form on the website without redirect users to SkyDrive).

  2. Is there an ability to build custom radial menus? XML, VBA? Nice new interface element!

    • The radial menu doesn’t not support customization in this version.

  3. Touch Mode should be the only interface mode available. It looks like a wonderful interface to use with either a mouse or your fingers. It looks cleaner and friendlier.

    Users don’t want or need an extra layer of complexity in the interface; they don’t want to have to choose between two different Office interfaces that are nearly identical anyway. Don’t complicate the experience of using Office with that weird toggle interface mode button which no one is going to understand or use properly. Please Microsoft, simplify!

    • Glad you like touch mode Donald. The one downside for touch mode on non-slate machines is that it takes up even more vertical real estate (especially when the ribbon is pinned). There is a trade off from a user experience perspective that we are currently discussing. For large monitors that might be okay but for laptops the benefit is less clear if users primarily use mouse and keyboard. that said, we are looking at a couple refinements and your feedback is helpful.

  4. So will this also be available for iPad? And if so, when? You know, it’s the touch-based device most people use in the office… ;-)

    • For now. There should be an iOS and Android Office coming sometime after Office 2013 but it certainly won’t anywhere near this powerful or capable.

  5. I pinned desktop OneNote to the right side, and I want to scroll up using the scroll bar in the webbrowser, but noway because the width of the zone for resizing onenote is the same as the width of the scroll bar… still a problem, through

  6. I think its moving in that direction but just like ribbon, Microsoft is going to gather some real world numbers before pursuing that full steam.

  7. It’s a shame that the menu tab titles are now all upper case (as is the task bar text). Putting text into upper case destroys the word shape and makes it far harder to read, particulalrly for dyslexics (As somebody that used to teach dyslexic students I can vouch for this). Generally the Metro style uses mixed case and the Office 2010 menu tab titles were mixed case, so why the change?

  8. Touch and so is nice if you have a tablet. At work, I use Office all the time and 10x more important to me is, that you still can use a real keyboard and a mouse ! All this touch is the future stuff is somehow insane, cause it isnt. It is a part of a consumer experice, but it is a drag if you have to work with this stuff. With Win8 = touch and Office 2013 = touch, I really think MS lives in a different reality, where you use this stuff only for fun and to play around or so… Honestly create some usefull stuff, not this hey it looks like wp7 stuff.

    • Moritz,
      Touch is just one part of the overall work we did in the new Office. We definitely agree that a lot of people will continue to use mouse and keyboard configs with desktop machines.

    • What element of touch in Office 2013 inhibits your if you’re on a desktop with a keyboard and mouse?

    • I do not see how new interface makes using office on a desktop harder. I use both tablet PC and desktop everyday and a lot. And I do not see any conflict here. Actually planning to install both Windows 8 and new office on my desktop.

  9. This is something we will look at. We are thrilled people like the radial menu. As for making it the default we want to be very thoughtful. People develop deeply ingrained behaviors for things like context menus. Definitely something we will consider.

  10. Nice article, and nice work on the new office.

    My concerns are following: why did you not enable touch by default on tablet? (I mean, I have office installed on my Samsung slate with Windows 8… And if didn’t read your article, i had not notice about the touch mode.

    The other concerns is about the article it-self, and the video. I read it on my slate (so IE10, metro) and I had to open chrome to be able to play the videos. Maybe you better have to talk to Steven about how they did the videos for the Building Windows 8 blog :)

    Except this two points, very great work on office 2013! and on this article…

    • Cedric,
      There is an OEM setting that manufactures can set that will turn touch mode on by default. We are working with OEMs to make sure this gets configured correctly. It should happen by default when Windows 8 slates hit the market this fall.

      We did just move the videos over to the Office channel on YouTube so they should now work with IE 10.

  11. I love the radial menu. If it remembers the colour you used, shouldn’t it also remember that I applied italic, not bold? So that swiping right becomes last format used. Or at least is there a way to change that, so that I can make swiping right apply italic?

    • There isn’t a way to change the behavior but that is good feedback. I will pass along your feedback to the owner.

  12. Cleartype in Office 2013 is a problem. The text effect in PowerPoint Presentation is no longer clear as 2010 did. Need improvement.

  13. None of the videos show the Ribbon Bar? As this is a really big part of the Windows UI, I was very much hoping to see how well it works with touch. Any chance chance of adding a video that shows it being used?

  14. I just posted my thoughts on my first week using a Tablet that is running Office Preview. I linked back to this post in mine – http://blogs.sharepoint911.com/blogs/jennifer/Lists/Posts/Post.aspx?ID=110 I am really hoping that some of the items you mention in this post get fully baked into the final release. I had some issues, especially in Outlook around the touch elements and not being able to see what I was typing. Overall experience has been good…just hoping some final things change with the release. Keep up the good work!

  15. This is phenomenal. Great job on both high level research and implementation. As a user of Asus EP121 for full year, and now Samsung Slate for the last 6 months I can more than appreciate all of the mentioned improvements. All of them are very important! Great job!

  16. So, it is obvious that in the long-term, the rest of the Office apps will have MX counterparts like OneNote does.in the meanwhile, however, here are a few suggestions to make the desktop apps more palatable on tablets:

    1. Allow OneNote MX to break the WinRT sandbox & share local storage with OneNote desktop; currently both apps waste time uploading & downloading from Skydrive even if edits are made on the same machine. Otherwise, allow them to mark their notebook files in the Skydrive MX or desktop apps as cacheable for longer periods of time.

    2. Add full stylus pen/highlighter/eraser options to the radial menu in OneNote MX & allow activating it by tap & holding with the stylus as well.

    3. Autohide the taskbar & switch to touch mode in the Office apps when Windows detects touch capability at first run or when the user touches the screen, & ask the user if they’d like Office to switch it off whenever there is mouse activity.

    4. Add a shim to allow the Office apps to show up individually in the system-wide app switcher (the swipe from left or Win+Tab activated switcher). Right now, they only show up within the Alt+Tab switcher in the Windows Desktop "app" regardless of how many of them are open. Also dehydrate & rehydrate them like the Metro apps. I want none of these "out of memory" error messages on my PC with 4 GB RAM when the 2010 apps have no such problems.

    5. Add a shim for Outlook’s tile to be a Live Tile which shows unread messages/pending tasks/reminders/people updates. You could do this locally or through a companion app that simply syncs the notifications & opens Outlook.

    6. Make the Windows Reader metro app capable of reading (not editing) Office documents & ePub files in addition to PDFs. I’d prioritize .docx & .pptx support if there are time/resource constraints. So, a tablet user who doesn’t need to edit these files can choose to never use the desktop.

    7. Some of the setup screens & options dialogs need to be optimized for touch.

    Overall Windows itself would do well to replace all the most commonly seen desktop icons with the Office 2013 style ones. And add some of the editing capabilities from MovieMaker & PhotoGallery into the new Metro apps or build separate apps for the same. Skype needs to do a Metro app fast.

Comments are closed.