Ubiquitous Computing

This entry is in response to a question asked through my LinkedIn network; specifically, "What are the implications of devices like the Apple iPhone™ and Microsoft Surface™?"

Well, it depends on exactly what you mean by the question. There is the general question of what applications will look like in the future; and the more specific question of how technology like Surface will influence that future.

Ubiquity is the first answer

Ubiquity is a primary motivator of the first aspect of the question. Ubiquitous display devices, mobile devices, connectivity, and computing power will change the way that users interact with applications and one another. Devices like the Apple iPhone™ and Microsoft Surface™ evoke the future by showing that general purpose content display and application interaction can be embedded in furniture and appliances, and can interact with personal mobile devices. They are the thin edge of the wedge; precursors of what will evolve as we learn how to leverage this technology.

Ubiquity requires context

In a ubiquitous computing environment, the many distributed components that store, compute, apply, and display information need some way to determine the identity of the current user or users, and access context associated with those identities. Context is the who (identity), what (activity), where (location), when (current, historical), and why (role) of an interaction.

Problems with current technology

Unfortunately, the Surface interfaces as shown in Microsoft demo videos are focused on the devices themselves, rather than the deeper context. For example, photos are exchanged on the table in one video by pulling them out of one device, onto the table for review and sorting, then dragging selected copies to another device. An interface that focused on the content (perhaps displaying content areas owned by each device's owner directly on the table, and dragging between those areas) would allow for deeper collaboration. The device would still be integral as an identity and presence provider, but the content could actually exist anywhere in the owner's context.

Appliances don't have to be smart, just connected

An appliance could also provide a display surface; although most just contribute information and services to the local context, with display and control available within the environment. The surface of your refrigerator might default to a status display, grocery list, or even a live view into the interior; eliminating the need to open the door five times looking for the mayonnaise. It could just as easily be manipulated to display any accessible content within the context. The context of the refigerator is content associated with the persons associated with nearby personal devices (not just the content on the device), other household systems, neighborhood and community resources, family calendar, etc.

For a more traditional feel, you might have it display to-do lists, the kid's artwork (pulled from their own contexts, which might also be shared with the school context), recent photos, and the Little League schedule (pulled from that context, and tailored to the local members). Important messages can be flashed to the recipient(s) when they are nearby. This progression reflects the digitization of our information and media; requiring more convenient places to display that information than a dedicated computer screen can provide.

Think about what you see on most people's refigerators, and imagine that content accessed through the wireless network in its digital form, and displayed on the surface of the appliance. Children's artwork, report cards, reminders, postcards, or even logos of places and inspirational or humorous sayings (think: refigerator magnets).

 

The (d)evolution of the wireless phone

Wireless phones that use a local Wi-fi connection hint at the effects of ubiquitous connectivity, as they eliminate the need for the phone to be a "phone", and make it just a peripheral hub for the ubiquitous environment (audio, video, pointing, and whatever other sensors can be tacked on once all that "phone" overhead is gone (including keypads, a la iPhone).

The next generation of mobile devices will serve as presence interfaces, with connection to the shared environment, location, identity and authentication, etc.

Collaboration is the second answer

The second motivator is collaboration as the dominant usage paradigm. The current dominant paradigm for computing is an environment that allows the execution of processes that used and created information, maintained in storage which can be accessed via a presentation. Once upon a time, only a few people were allowed to operate a computer. The execution of processes was centrally scheduled by these few; as only one process could run at one time, and the resources were expensive. Punched cards provided input and storage capabilities; presentation was printed reports.

Multiprocessing allowed for multiple processes to run at once, but the basic paridigm remained the same. With the introduction of online storage (disk drives), timesharing was introduced; everyone logged in to a timesharing system, and worked within the same environment. That environment allowed multiple users to appear to have simultaneous access to information and the sceduling of processes. Since all of the users had access to the same resources, sharing context was natural, while subject to appropriate access restrictions. Networks were introduced as a way to exchange information and context between systems.

As a consequence of the PC revolution, work moved into isolated islands, as each user maintained their own data and applications, and worked alone to create and modify information. Only by passing around files on media (sneakernet) or networks (email, shared volumes) could there be an exchange of information. Content creation tools and applications were (and still are, mostly) poorly integrated.

This will have to change, as users engage in collaborative content creation in a shared environment. A collaborative application consists of local agents (identity, multimedia input and display) for each participant, the application functionality (flow, commands, rules, events, etc.), and content that is within the context of the participant (perhaps scoped down to a particular role/task). There is much more to be explored here, but space does not permit a full expedition.

Content management as the dominant paradigm

Finally, another primary motivator is the emergence of content creation, content subscription and filtering, and content search as key use cases for computing technology. Cross connecting these functions becomes even more interesting when you consider the use of incoming content to support the creation of new information. This could be manual (a content consumer could comment on (creating new, linked content), aggregate the information (creating a new context for the content), or using information to derive new content by combining it with information in the local context (creating derived or enriched content). As an example of the latter, incoming stock quote updates can be used to calculate part of a portfolio value that is published as content for consumption of further derivation (like comparing the values to setpoints and creating new "alert" content).

Summary

To summarize: Technologies like Surface and iPhone are precursors of what is coming at us from new interface technologies. These innovations will drive changes in user expectations, and increase the diversity of those expectations. New technologies will challenge the assumptions that are common in application design, and software design in general. The new world will be centered on content (creation, consumption, management), context (who is doing what to which, where, when, and how), policy (who can do what to which, where, when, and how), and collaboration (multiple people (who), doing multiple things (what), to the same thing (which), at the same time (when)).

Copyright 2008, Christopher P. Caserio