User Interaction with the Semantic Web

Academics and practitioners from the semantic web community are seeking the involvement of usability professionals as they define the "next generation web." Recently, workshops at various conferences have focused on user needs and user interaction.

Over the course of 2005-2007, I began collecting examples of current innovations related to user interaction/design for the semantic web. This page serves as a reference for usability/design practitioners as they explore the subject and consider the role of semantic web opportunities and offerings in their work. Presentation slides (pdf) are available from talks at the Usability Professionals' Association, June 2006, and Computer Human Interaction (ACM SIGCHI), May 2007. Examples and interesting web sites are listed further on in this paper.

This paper includes:

Background on the semantic web
Interaction challenges
Questions for the usability community to consider
List of semantic web interaction workshops and resources
Examples and applications

Background

The semantic web has been defined as "an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation" (Tim Berners-Lee, James Hendler, Ora Lassila in Scientific American, 2001).

Is it time for the usability community to look further at the specifics of the semantic web? Conversations with researchers, semantic practitioners, and government officials over the last few years confirm that there is a very strong demand for usability input and involvement in the development of the semantic web.

The tools for the "next generation web" are becoming available now and will be increasingly used by web developers, so it is important for user-centered designers and usability professionals to understand what is coming into greater use.

If we think of the challenges and limitations we have when designing currently to meet user needs, then it appears that the semantic web is dangling some enticing carrots in front of us:

Greater user context available to make application/site behavior more user-centered
More flexible and adaptive language parsing and "understanding" to make the conversation with the computer more effective
Better visualizations and representations of complex data and interactions
Availability of agents that can manage routine background tasks and reduce the need for users to hunt and search among thousands, millions, or billions of items of information without adequate support
Sharable ontologies and vocabularies, allowing all the applications and sites that a user sees to speak a "similar language" and reducing the demand on the user to interpret language when trying to complete a task that requires more than one site or application at a time
Logical inferencing that can help synthesize information from disparate sources, compare and organize that information, then present it to the user in a more seamless way, reducing the cognitive burden

While it entices us, it also challenges us to consider where and how "semantic enabling" user interactions can facilitate increased usability of the web and software applications. It challenges us to consider the risks and complications that arise from these technologies, and the role we need to play as user advocates and representatives in the ongoing development of these technologies. Some of these challenges are explored in further detail in our 2004 paper: the Usability Imperative Inherent in the Semantic Web.

return to top of page

Where to focus? Interaction challenges for the Semantic Web

Below are some broad categories of challenges that need both research and practice input from the usability community. A growing list of challenges is being developed by the Semantic Web User Interaction (SWUI) mailing list community, and can be found on the SWUI web site.

Context:

Capturing context in a simple, usable, yet rich way -- ideally, as a by-product of other activities
Facilitating the "Context Conversation" -- how simple can we make it for applications to understand a user’s specific context?

Process:

Involving the user throughout analysis and design, as well as during evaluation
Refining user-centered methods to support design of richer contextual/personal applications

Navigation, views, and data interpretation:

Scalability
Focusing on the relationships (the lines) in visualizations, representations, and queries
Maintaining ontologies -- language evolution, collaboration, relationships, rules

Authority and permission:

Interacting with agents -- defining your relationship with them, answering their questions, communicating changes
Avoiding "pestering" while maintaining oversight
Understanding trust, privacy

return to top of page

Questions for the usability community to consider

What aspects of the semantic web have an interface? Even if there is no interface, what are the interactions?
How can we be sure our interactions with data capture/management tools are so easy that anyone can -- and would -- describe themselves and the associated context?
How do users and technologies work together to build the semantic connections? How are they expanded and enhanced when users collaborate with each other electronically? How are old connections removed?
How will seeking and reasoning applications be forgiving of differences in language and meaning, and be clear/respectful of semantic ‘shades of gray’?
How do users remain (and feel) in control, when so much is going on ‘behind the curtain’?
The more that applications are built from shared, distributed components, how much harder is it to shape the overall user experience?
What are the fundamental implications of things like agents, reasoning engines, component technologies and web services - building blocks for web-based applications and interfaces?
How are user personas, preferences, experiences and interests modeled? Once these connections are built, then how are they changed as the user’s expectations and needs change?
How can we create better-managed semantic environments to help us avoid drowning in the ocean of metadata?
How does a user’s interaction with an application help filter out the myriad of possibilities, hone in on what is relevant, and make sure that the interpretations being made in the background are suitable to a particular situation that day, and not a situation faced last week or last month?
How much of this can be done without requiring many hours of my time informing my machines/appliances of my desires, circumstances and vocabulary preferences?
What interactions are required so that semantics can support online conversations and interactions? Can semantic technologies enable person-to-person and group interaction, and make the idea of online community more practical? What facilities are required to make that easier?
How do we keep the language from becoming stale - or, put another way, how do we maintain the semantic web?

return to top of page

Workshops and resources

Mailing list for SWUI discussion: public-semweb-ui@w3.org. (list archive is available on the web)

Workshops:

Semantic Web User Interaction. The Third International Workshop on Interaction Design and the Semantic Web, held in conjunction with ISWC2006. Athens, GA, USA. November 6, 2006. http://swui.semanticweb.org/.
End User Semantic Interaction. The Second International Workshop on Interaction Design and the Semantic Web, held in conjunction with the 4th International Semantic Web Conference, Galway, Ireland. November 7, 2005. http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-172/.
Beyond the Click: Interaction Design and the Semantic Web. The First International Workshop on Interaction Design and the Semantic Web, held in conjunction with the World Wide Web Conference 2004, NY, NY. May 18, 2004. http://interaction.ecs.soton.ac.uk/idsw04/.
User Aspects of the Semantic Web. Held in conjunction with the 2nd Annual European Semantic Web Conference, Heraklion, Crete. May 29, 2005. http://kmi.open.ac.uk/events/usersweb/.
Promoting User Experience in the Semantic Web. Held in conjunction with the Usability Professionals’ Association conference, June 16, 2005. Montréal, Québec, Canada.

Additional references:

W3C Semantic Web activity area. The reference site for current standards-setting work. http://www.w3.org/2001/sw/
Annual Semantic Web Challenge. Further information and demonstrations may be drawn from this showcase of innovative applications, sites and methodologies that illustrate the potential of the semantic web. http://challenge.semanticweb.org/.
Passin, Thomas. Explorer’s Guide to the Semantic Web. Greenwich, CT: Manning Publications, 2004. Online reference: http://www.manning.com/passin.

return to top of page

Examples and Applications

The following list of applications is not exhaustive or definitive -- it was created to show a range of possibilities and issues for consideration from a usability perspective. Not all the examples are strictly "semantic web" applications, but they serve to illustrate interactions that are -- or could be -- adopted by semantic web applications.