Card Sorting - What We Learned
Jun
23
As part of our site evaluation of the Web Administration site, we turned to you - our users - for help. One of the methods we decided to use was a card sort.
Card sorting is a usability testing method in which names of items are placed on index cards and participants are asked to organize those cards into similar piles. In an open card sort, the participants come up with names for the piles they create. This process is repeated multiple times and the cards are then analyzed for grouping patterns among the participants. Card sorting is helpful in organizing all kinds of things, including the information architecture of a website and the collection of information on an individual page.
Instead of actually writing on index cards, we turned to WebSort.net, an online application that works similarly to the analog card sort but allows the participant to drag cards into piles. It also does some of the grouping analysis.
Here are some of the interesting things we learned about this round of card sorting:
Things we liked:
- Participants have opinions! After the online sort, participants had the option of leaving comments about their experience. One of the things we discovered was that we had goofed on a few of the card labels.
- Using an online tool made it very easy for people to participate. Now, instead of having to have a facilitator set up an appointment and go through a testing procedure, the participant could process the card sort at his or her leisure.
- WebSort.net’s analysis tools were very helpful. It was easy to pull out a biased participant out of the group and rerun the analysis, for instance.
Things to fix the next time:
- We should have been clear that we were not asking the participants to create site navigation. We indicated that we were using this information in order to redo the site and many were thinking of the items in terms of where they should go within a site structure, with some even noting that they were frustrated that they could change the order of items.
- Some of the best information one can gain from usability testing comes from observing and listening to the participants, which we missed by using the online application. We did get some great comments from people, but often you learn things from watching the participants move cards and make comments about the sort activity. Had we tested in person, we would have learned very quickly which cards had confusing labels, instead of having to wait for one of them to comment at the end of the online sort.
If you are interested in doing your own card sorts, there are some fantastic resources available. The book listed is available for checkout from us here at Web Administration.
June 23rd, 2009 at 3:30 pm
Next time you should also consider having participants use physical cards. Why? By using an online application, you’re potentially skewing your data by oversampling a dependent variable. That is, by using an online application you are getting data from a user group that is potentially more savvy than say “regular” or “normal” users. This could cause your results to be skewed in critical ways.
June 24th, 2009 at 10:07 am
Christopher, great point!
If this sort was for a more general interest website, I would have either done an analog card sort or a mediated online sort. Since the sort was for the Web Administration site, I assumed that our normal users would have more of a technical background so the online sort wouldn’t have as much of an impact. We selected folks for testing that would likely use the site.