Creating a Usability Ontology

Developing a sharable, reusable ontology of usability and user-centered design terms and relationships is an idea whose time has come. There is an increasing need to organize resources consistently in this discipline, as well as the need to support cross-reference between concepts. Our workshop at the 2007 Usability Professionals' Association conference begins this work.

Increasingly, we see categorization and metadata appearing in all aspects of user experience, because it helps users find relevant information and understand the context of the tasks they are performing. Categorization is appearing in places such as:

• Search engines -- Not only is Google moving toward more complex and personalized versions (based on categorization), but there are increasingly useful search engine competitors such as Yahoo, Clusty, Ask.com, Kartoo, etc. Their goal is to get a user closer to what they want to see more quickly.
• Portals -- the "My [provider name here]" services all classify their service offerings, news/sports feeds, etc. by standardized categorization.
• Social sites -- del.icio.us and flickr may be the best known, but they are not the only sites that offer users the ability to create and use meaningful terminology and classification.
• e-Commerce -- most shopping sites now have faceted browsing, based on product categories and features, as a standard element of the shopping experience.

Benefits of a usability ontology

The usability ontology that we create must be robust enough to cover the broad territory of our professional field, and simple enough to be useful. It must contain language that is recognized by practitioners, and ideally by people from related disciplines.

This workshop will benefit UPA, its members, and workshop participants.

How UPA will benefit: UPA needs a good ontology that covers the breadth of topics in the field of usability, user experience, and user-centered design, to help people search effectively across their growing library of conference proceedings, articles, research papers, and reference materials/sites. It makes sense to establish one primary source for this vocabulary to enhance the quality of description and support integrating materials.

How UPA members will benefit: All members and readers of the UPA publications will benefit through improved ease of finding related materials. Many individual members, both practitioners and academics, may also benefit from the ability to borrow and reuse a standard usability ontology. For example, at the Usability BoK Special Interest Group (SIG) meeting during the UPA 2006 conference, many attendees talked about work they had already done or planned to do in their own organizations to compile and distribute usability information.

How workshop participants will benefit: Workshop participants who are involved in any of UPA’s publications, the Body of Knowledge project, or who are creating repositories of usability information for other projects or clients, will benefit through helping to create an ontology that meets their needs. In addition, participants will gain direct experience collaborating to build an ontology or controlled vocabulary, which is increasingly in demand in many organizations.

Previous usability terminology work

There are several good starting points -- existing usability terminology -- that workshop participants can draw on for creating a usability ontology:

• The Usability BoK project has defined a preliminary categorization of topics for use in collecting information, as well as a preliminary set of glossary terms.
• The UPA conference has a categorization of topics for use in the description of submissions and in the review process.
• The new Journal of Usability Studies (JUS) has keywords that could be drawn from its published articles.
• ISO 18529 (doc) and ISO 13427 contain terminology for high-level activities in a human-centered design process.
• UsabilityNet provides a categorization of some usability methods based on where they occur in the software development lifecycle.
• The ACM digital library keywords are associated with each article. For example, usability-related articles are tagged with keywords such as "business strategy, concept design, ethnography/ethnographic studies, experience design, human-centered design, information architecture, interaction design, marketing/market research, process improvement, product design, usability research, user experience, user interface, user research."
• Usable Web topics list has a hierarchy of topics tied to the publications reference list.
• The STC Usability SIG has an index of topics in usability.
• The IxDA web site has a categorization scheme focused on the types of resources in its library, although unfortunately not focused on the content of those resources.
• Scott Henninger of the University of Nebraska is currently developing an ontology for usability design patterns (an article describing this ontology is part of the proceedings for the Semantic Web User Interaction workshop at ISWC in November 2006.

While there are many useful, and even somewhat authoritative, terminology sets for the usability subject domain, we have not been able to find much evidence of previous, comprehensive, published efforts to create a usability ontology. This is particularly true in the semantic web space, where many earlier ontologies are now becoming available. For example:

• SWOOGLE, a search engine for finding RDF ontologies in the semantic web, does not produce meaningful results for existing RDF/OWL ontologies when using terms such as "usability" and "HCI" -- and very few useful data sets containing "interface."
• The Ontoworld wiki has the term "usability" as a classification for articles, but does not appear to have any lower level of classification detail for that subject area.

Del.icio.us, which takes the "folksonomy" approach to collecting keywords for tagging content, has a handful of keywords related to usability. Again, these keywords are only at a high level (e.g. "usability" and "accessibility"), suggesting that the grassroots approach alone will not generate a vocabulary at the level of detail that would be needed in order to be useful to our professional community.

We have found some very initial descriptions relating to possible ontology development for the SWEBOK (Software Engineering Body of Knowledge) at Lehigh University, but have not yet located further progress on that work.

The workshop organizers plan to draw on published works about creating ontologies in order to help participants with this activity.

This workshop will launch a completely new effort, drawing upon best practices for ontology creation, to establish a usability ontology based on starting points such as the ones listed above.

PARTICIPANT SELECTION CRITERIA

We are looking for a group of experienced practitioners with broad knowledge of the field. It is important that participants also have an open mind and willingness to challenge assumptions that may be embedded in existing terminology!

The requirements for submission are somewhat different than for other workshops, as we are not asking for a full "position paper." Instead, every participant will need to provide a contribution of a proposed or existing set of terminology for the group to work with. We then need a minimum of background information to better understand each participant’s usability experience.

The outline for the participant submission would be:

1. Background and experience. Participants will be asked to describe their educational background, work experience and particular interests in the field of usability/user experience/HCI.

2. Goals. Participants will be asked to briefly describe why they want to participate in the creation of a usability ontology and what they hope to get out of the workshop. This can include description of a particular need that could be addressed through the creation of a usability ontology.

3. Any specific experience with creating or using vocabulary sets or ontologies. Participants will be asked to describe any activities that they have participated in which are related to the creation or use of ontologies in any subject domain. (Lack of experience in this area is not a barrier, but we would like to encourage participants who do have this experience.)

4. Input to usability ontology. Participants will be asked to collect and submit (or reference) raw materials for use as starting points in the creation of a usability ontology. Information on rights of use for these materials (i.e. are they public or proprietary?) should be provided as well. These contributions may include:

   • Vocabulary lists
   • Glossaries
   • Indexes
   • Categorization schemes
   • Taxonomies

View the sample position paper for an example of the types of information you can contribute.

Deadline for position papers: May 1, 2007. (Papers submitted after this date may be accepted at the discretion of the facilitators.)

PRE-WORKSHOP PARTICIPANT ACTIVITIES

Before the workshop, participants will be asked to:

• Review a short reading list on ontology development and controlled vocabularies.
• Review the submitted papers/vocabulary lists from the other workshop participants.

PRE-WORKSHOP FACILITATOR ACTIVITIES

Before the workshop, the facilitators will:

• Prepare a short reading list on ontology development and controlled vocabularies.
• Review the inputs submitted by the participants.
• Prepare a list of sources of vocabulary (based on what is provided by attendees, along with additional research by the workshop facilitators) as a handout for the workshop.
• Use the input to create cards or sticky notes for categorization activities during the workshop.

WORKSHOP ACTIVITIES

Timeline / Duration	Topic or Event
8:30-9:00	Introductions of participants and facilitators
9:00-9:15	Facilitator discussion of ontology and approach to the day
9:15-10:00	Needs assessment (in small groups if enough participants)
10:00-10:30	Break
10:30-11:00	Participants briefly describe their contributions
11:00-12:00	Strawman usability ontology (breakout session)
12:00-1:30	Lunch
1:30-3:00	Strawman usability ontology (breakout session)
3:00-3:30	Break
3:30-4:00	Review of strawman usability ontology
4:00-5:00	Discussion of next steps and ongoing activities

Workshop activities description

1. Introductions of participants and facilitators

All participants and facilitators will introduce themselves briefly at the beginning of the workshop.

2. Facilitator discussion of ontology and approach to the day

The facilitators will provide a brief introduction to the concept of an ontology, and discuss relationships between concepts such as keywords, controlled vocabulary, taxonomy, folksonomy, and ontology.

The facilitators will also give a preview of the approaches that will be used during the day to organize, extend, and refine the raw materials in order to create our first strawman ontology.

3. Needs assessment

This 45-minute brainstorming session will clarify the need for a usability ontology from the perspective of different groups that would use it. For example, how would the conference use it? How would practitioners use it? How would universities use it? This quick user needs analysis may be done by dividing the participants into small groups based on their area of interest, asking the small groups to brainstorm about their needs and then report back to the full group.

4. Participants briefly describe their contributions

This 30-minute (max) session will provide an opportunity for each participant to introduce the raw materials that they contributed as starting points for the workshop. This description can include what it is, where it came from, the purpose for which it was developed, and anything the group should know about its use.

5. Strawman usability ontology (breakout session)

In this breakout session, small groups will take a first pass at organizing terminology into a rough framework for a strawman usability ontology. This session will be highly interactive and will have the flavor of an affinity diagramming session.  

6. Strawman usability ontology (breakout session)

After lunch, the participants will be reassigned to different small groups to continue work on the strawman usability ontology. Switching the groups will provide an opportunity for the participants to mingle and talk with different teams, stimulating new ideas.

7. Review of strawman usability ontology

We will spend 30 minutes reviewing and discussing the progress made on the strawman usability ontology. The facilitators will list questions and issues that surfaced during the breakout sessions that will need further investigation.

8. Discussion of next steps and ongoing activities

The last hour of the day will be spent organizing ongoing activities to continue building and refining the ontology. As a full group, we will discuss methods for collecting further contributions through future working sessions and online collaboration. Issues to be discussed will include:

• Should this workshop serve as the opener for a series of workshops on developing a usability ontology? If so, what procedures, protocol, approaches, venues, should be considered for subsequent workshops?

• What type of online collaboration space would be needed to support follow-on activities?

  • How can people suggest terms for the ontology?
  • What collaborative tools could we use?
  • How will it be managed?
  • How will it be reported?
  • Will visualization help?

• How will we encourage members of our community to participate and collaborate in such an effort?

• How will disagreement be handled?

• How can we facilitate translation of this ontology into multiple languages?

POST-CONFERENCE ACTIVITIES

To continue momentum, we can:

• Create a draft agenda for ongoing work.
• Create an online venue for continued collaboration to refine the ontology -- this could be done within the construct of the BoK draft wiki environment, or possibly as a separate collaborative environment.

To encourage use, evaluation, and comment on the ontology, we can:

• Publish guidelines for use (for example, how to create html meta tags in usability-oriented web pages, how to access and embed RDF, how to incorporate this terminology in existing online folksonomy environments like del.icio.us or Amazon).
• Meet with UPA publications/project leaders to discuss how they might effectively begin to pilot the use of the ontology.

| Welcome to IPGems | PCD & HCI | IM & KM | Contact IPGems |