Skip to content

Cues from the Microsoft Productivity Future Vision 2011 Short

Microsoft’s Envisioning team in the Office division puts out a video every few years (20082009, 2011) highlighting ideas and concepts they feel are coming in the next 10 years or so. Besides being extremely high quality and intriguing, these videos also give us a glimpse into what different individuals inside Microsoft are working on, researching, and conceiving. I’ve taken an especially keen interest in their most recent 2011 Productivity Future Vision short because of its ties with communication and email. Watch it before continuing (and perhaps revisit it as you read along so as to keep up):

Pretty cool, huh? There are some fantastic concepts in there (and very cool technology as well). I’ve watched the video many times, and I was even able to get some commentary on it directly from one of the Microsoft employees who helped create the video. Let’s take a closer look at some of the more pertinent and interesting scenes.

Short-form Messages and Integrated Calendaring

Near the beginning of the video, we see businesswoman Ayla reading an email while traveling in the cab to her hotel. You can see the email and the integrated response in the image below (click for full size).

A short message and integrated calendaring with the response.

First thing to note is the length of the original message. It’s short! Far too few messages are that short in real life. The sender, Qin, manages to get in a salutation, a comment about the flight, and a request in five lines, leaving enough space for a big, beautiful picture of him. We then see Ayla dictate a response that indicates she will get to the review “first thing tomorrow.” The device recognizes that a temporal event has just been mentioned and determines what the temporal event was (“Review Proposal”), checks her calendar, and inserts it, asking Ayla for confirmation.

What is wonderful about this proposed interaction is that it allows Ayla to use a “fire and forget” strategy because the task has been moved from her inbox straight to her calendar. Ayla does not need to flag the message, mark it as unread (as many do), or hope that she sees it tomorrow morning to remember to actually perform the review. Probably one of the most common strategies of coping with numerous emails and tasks is to leave the message in the inbox (Whittaker, 2005), often marking it as unread “so it will be visually distinctive and continue to catch [the user’s] attention” (Wattenberg, Rohall, Gruen, & Kerr, 2005). However, Whittaker notes that “visual reminding is compromised when there are too many inbox messages” because “older outstanding items are overlooked when processing new incoming messages” (2005). And for busy executives receiving tens or hundres of messages a day, this can happen quickly.

Furthermore, when Ayla has the device actually create the calendar appointment automatically, it saves her from having to transpose the information from her email to her calendar, which is a step that prevents many people from actually completing a task (Blandford & Green, 2001). Later in the video we see Ayla attend to the task the next morning, and the review file is automatically associated with the appointment so Ayla doesn’t have to dig around in her emails or on a server to retrieve the file. This again is a huge step in reducing overload because it does not require Ayla to transpose any information or documents to the task.

Some email clients do currently attempt to pick out metadata like this from emails (Mail.app will look for dates and times; Outlook will too), but no client does it as effectively as is shown in the video, nor does any client do it from your reply, only from the original message, to my knowledge. This is certainly a technique worth implementing for those creating email clients.

5 Minute Focus

5 Minute Focus screen on Qin's device

One of the concepts reiterated several times throughout the video is that of a device automatically displaying tasks for a user to complete based on the situation. David Jones, the Microsoft program manager who was kind enough to give me a few thoughts on the video, said the following:

“The bigger idea we tried to get across is that we can surface the right emails for you to look at based on your current context – Ayla in the car, Qin with 5 minutes to spare in the subway, and Ayla in the Hotel room based on the fact that Shannon was worried.”

Qin replying via an integrated app to a quick questionQin might not have been able to write an employee review while waiting for the subway, but he certainly could respond to a quick one-line question. Somehow the device is aware of two things: the current context (the fact that Qin is waiting for a specific subway train), and the estimated time required to complete a request in his voicemail inbox (automatically transcribed, something that I think the latest version of Exchange Server will do for you, as will Google Voice). As a side note, I also like the nice touch of giving the option of grabbing something at the café as well, a little geolocation magic!

This also brings up the concept of an integrated mini-app that is displayed instead of a typical reply field. No written response or call is required by Qin because he approves the order on the spot from within the original message. Who remembers Google Wave? This type of ability was built in, where you could conduct polls, attendance, etc., in-message. That wasn’t the first time this idea had been proposed, however. Duchenaut and Watts found that the first time this was suggested was in 1976 by Anderson and Gillogly with “computational mail,” where when “the message is opened, the program is executed” (2005). Now obviously we would have to be careful to avoid the email viruses of yesteryear, and this would mostly likely be a very custom development job inside of each company, but it would certainly also be a leap forward in usability.

Conversations from Anywhere

Another powerful idea developed in the Productivity Vision film is that content, no matter what or where, is all linked, and people can have a conversation about any piece of content, augmented by context. At the end of the video we are introduced to Shannon, Ayla’s daughter. Shannon is trying to find a recipe to make for school, and writes and then video calls her mother.

Shannon initiating a video call from within her message

I was wondering just how that video call was started. Here is how David elucidates this scene:

“Shannon initiated the call. She wrote a handwritten note, and then initiated a call from “within” it. The idea here is that you can have a conversation from within any piece of content. This is similar to how Ayla leaves Jeff a comment in the presentation she is working on in the Office scene. And how Jeff leaves Qin a comment in the document that we see in 5 minute mode in the Subway scene. Unfortunately we don’t show Ayla actually answering the call—we were re-using footage and didn’t have a shot of her pressing anything on the phone. It goes by so fast that  no one notices though.”

So Shannon started by writing a note, but then used that note to start a video call. How many times have you been writing a reply to a colleague, or been in the middle of composing a report you are both working on, and realized that just wanted to talk to them? Or how often have you been reviewing some work done by a co-worker and thought of leaving a comment, but didn’t want to compose a whole new email or add the comment in-file and then resend it?

Similarly, we see other examples of a related concept elsewhere in the video. For example, when Jeff is writing the report, two sets of words are underlined for him, with the computer offering to insert supplemental information about those objects into the report for him. Arrows have been added to show what the computer offered in the video.

The computer automatically offering to insert supplemental content.

I’m not so sure that I would find this type of feature handy very often, but the idea that related information should be surfaced is extremely powerful, and also very pertinent to information overload. It is time intensive to open attachments or find previously saved data when conducting business over email, and according to the most current research, the biggest contributor to email overload are messages/tasks that require lots of interdependent, qualitatively complex messages (Bellotti, Ducheneaut, & Howard, 2005). Stopping mid-sentence during composition to go and find an Excel document that you know has a figure in it related to what you are writing incurs a large task-switching cost. If the computer can automatically determine and retrieve relevant information, documents, and links, then this task-switching cost can be avoided and the complexity of those messages greatly reduced. Likewise, pulling people’s schedules and locations when discussing them in an email would also be very helpful, avoiding an extra trip to the often-separate calendar. Both Bellotti et al. and researchers at IBM (Kerr & Wilcox, 2004) have experimented with making related information in email clients “first-class citizens,” with very positive results. Implementing such a feature would obviously require very complex natural language processing and metadata handling capabilities, but it would be a huge boon to reducing complex and overloading emails.

Note the pink bar indicating Shannon's emotionAs a bonus, David also pointed out something that he said most people miss: the fact that Shannon was worried about finding a recipe, and therefore gets a pink bar above her as an indication to her mother of her emotion. “Tried to work in some emotional reco here,” said David. This is quite advanced, but makes a lot of sense. How would it apply in the business world? We are often unaware of the current workload of the people we are sending messages to, and are often unaware of how much work we have delegated them in the past. Sending emails is “cheap,” monetarily of course but also cognitively, and we get in the trap of sending too many to be useful. Could some sort of indicator be included marking how “overloaded” the individual is (this is measurable, in fact; see Hogan & Fisher, 2006; or Jonathan Spira of Basex), or how much we have delegated to them in the past? Indicators such as these may help senders to temper or change the workload they are sending out via email.

Conclusion

Whew, there’s a ton here and I haven’t even covered the half of it! Let’s review the major ideas and concepts that spring from this short that would be useful in the fight against email and information overload.

  • Shorter, more complete messages: Sounds simple, but it would be a big help for many people to retrain ourselves to write more clearly and succinctly (especially me; as you can tell, brevity is not my forté). Could we write email client plugins that evaluate writing style and give us helpful hints?
  • Automatic information transposition and task creation: Huge because it removes several steps that many people simply don’t take the time for (transposing information) and removes the burden of task management from your inbox, organizing it more effectively elsewhere.
  • Context-aware task surfacing: Do you have five minutes to spare? Your email client should give you tasks that it knows you can complete in that time frame so that you don’t have to hunt and peck for them.
  • Integrated mini-apps: Similar to Google Wave widgets, business-related application components that allow non-text actions to be taken from emails.
  • Conversations from and in any content: Any and all content is up for use as a conversation starter, and doing so allows the conversation to always be context aware.
  • Context-aware data surfacing: By pulling up relevant documents or information as you are working, the computer can greatly reduce task-switching costs associated with searching for needed data. This would be an enormous boon to productivity.
  • Emotional/historical indicators: Is someone overloaded because you’ve unconsciously sent them too many tasks recently? Indicators could help us track how someone else is doing, and we can adjust our behavior as senders based on that.

A big thanks to David Jones for taking the time to respond to my questions about the video and for the information he provided. I’m hoping that we can take the concepts presented here and turn them into reality and see the benefits we can reap from it.

Sources

Bellotti, V., Ducheneaut, N., & Howard, M. (2005). Quality versus quantity: E-mail-centric task management and its relation with overload. Human-Computer Interaction, 20, 89-138. Retrieved from http://portal.acm.org/citation.cfm?id=1466574

Blandford, A., & Green, T. (2001). Group and individual time management tools: What you get is not what you need. Personal and Ubiquitous Computing, 5, 213–230.

Ducheneaut, N., & Watts, L. A. (2005). In search of coherence: a review of e-mail research. Human-Computer Interaction, 20, 11-48. Retrieved from http://portal.acm.org/citation.cfm?id=1466572

Hogan, B., & Fisher, D. (2006). A scale for measuring email overload. Microsoft Research, 7-9. Retrieved from http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:A+Scale+for+Measuring+Email+Overload#0

Kerr, B., & Wilcox, E. (2004). Designing remail. Extended abstracts of the 2004 conference on Human factors and computing systems – CHI  ’04 (Vol. 23127, p. 837). New York, New York, USA: ACM Press. doi:10.1145/985921.985944

Wattenberg, M., Rohall, S. L., Gruen, D., & Kerr, B. (2005). E-mail research: targeting the enterprise. Human-Computer Interaction, 20(1), 139–162. L. Erlbaum Associates Inc. Retrieved from http://portal.acm.org/citation.cfm?id=1466575

Whittaker, S., Bellotti, V., & Moody, P. (2005). Introduction to This Special Issue on Revisiting and Reinventing E-Mail. Human-Computer Interaction, 20(1), 1-9. doi:10.1207/s15327051hci2001&2_1

Published inEmail Research

One Comment

  1. Interesting post, Josh. During my previous viewings of the Microsoft video I had focused wholly on the interface design which I found unimaginative and boring. I am glad you dug deeper to uncover the implications of advances in information processing and sharing. My favorite innovation is the use of the emotional/historical indicators. That might partly be due to the fact that I often do not feel comfortable sharing how I feel on my own, and would love for technology to do it for me. Also, I would love for technology to use such indicators to help manage and prioritize messages from coworkers and family.

Leave a Reply

Your email address will not be published. Required fields are marked *