What is Card Sorting?

Taxonomy design is a fundamental component of any information management initiative. It provides a structure to label and organize content, whether it is for a website, intranet, or digital library. Card sorting is a technique for discovering how people categorize information and consequently for validating a taxonomy design. The results of a card sorting exercise provide valuable information to confirm whether a taxonomy is intuitive enough for end users or to identify certain categories that need adjustments based on input from the participants.

At EK, one card sorting tool we use to test and validate the taxonomies we design for our clients is OptimalSort. We start by selecting a group of end users and stakeholders that will be invited to participate in the card sorting exercise. Participants receive an email with a link to access the exercise, which typically takes only 10 to 15 minutes to complete. The exercise consists in organizing individual cards into certain categories assuming that the labels for these categories are descriptive enough and make sense to them. Otherwise, participants are able to suggest new categories.

Card Sorting Types

Types of card sort include open, closed, and hybrid. In an open card sort, the participants are able to create and name their own categories, while in a closed card sort, categories are given to participants. In a hybrid card sort, participants can use either existing categories or create new ones that make sense to them. The selection of the type of card sort depends on the purpose of your study. At EK, we apply the hybrid card sorting method to validate topical taxonomies that we design during taxonomy workshops with our clients. By conducting this exercise, we are able to confirm whether the topic taxonomy we designed is intuitive enough and meets the needs of users and stakeholders or whether we need to add or modify some topics. Even though card sorting is a great way to validate a taxonomy, it is not the last word to complete a taxonomy. After completing the card sort, it still necessary to discuss the results of the exercise with the stakeholders or with the taxonomy governance team, if existing, for decision making.

A Practical Example

Last year, our EK team conducted a card sort to validate some categories of a topical taxonomy for one of our client’s Intranet. The exercise consisted in asking participants to take a look at a list of document names on the left of the screen and sort them into a topic category that made sense to them on the right of the screen. They simply had to drag and drop a document name from the left into a category on the right. They also had the option to create new topic categories, if needed. At the end of the study, we analyzed the results and were able to identify a few existing categories that needed some work and also new categories that we had not initially considered and could potentially be added to the taxonomy. The results of this card sort combined with additional discussions with key stakeholders helped the team finalize the taxonomy design before implementing it in the new Intranet.

The screenshot below illustrates what the card sorting exercise looked like for the participants.

How to Analyze the Results of a Card Sorting Exercise

For each card sort study, OptimalSort provides a variety of comprehensive and insightful tables and diagrams to analyze the findings. These include a standardization grid and a similarity matrix. A standardization grid shows the number of participants that placed a card in a category. It is important to look for high-level of agreement (over 65%) for each piece of content to confirm that a category is intuitive for the majority of end users. A lower percentage indicates that that category needs to be revisited and adjusted. Below is an example of a standardization grid.

A similarity matrix clusters together the strongest card groups (taxonomy categories) along the diagonal edge. In other words, it indicates strong associations between pairs of items (document names). This analysis helps identify which pieces of content should be under the same category in the taxonomy because people found the association among them intuitive. Below is an example of a similarity matrix.

Steps to Prepare for a Card Sorting Exercise

To conduct a closed or a hybrid card sorting exercise, it is necessary to prepare the following information in advance:

1. Identify the cards that will be sorted. About 20-30 document names.

2. Choose the taxonomy categories that you want to validate. About 10-15 topic taxonomy categories.

3. Write the welcome message that participants will see when they first access the card sort. About 1-2 paragraphs explaining the purpose of the study and expected time to complete.

4. Prepare the instructions for participants to perform the exercise. If it is a hybrid card sorting, indicate that participants would be able to suggest new categories.

5. Identify a logo and color scheme. It could be your company’s logo and color scheme to customize the look and feel of the exercise.

6. Write an email invitation for participants. Remember to include the link to the card sorting exercise.

7. Choose the start date and end date for the exercise. Typically the card sort study should be available for about a week.

8. Identify pre-survey or post-survey questions. This is optional to gather additional information, if needed.

After preparing this information, launch the card sorting exercise by sending an email to the participants including the link to the study. It is always helpful to send a friendly reminder encouraging people to participate, if they have not done so yet. After getting the results and analyzing them with the similarity matrix and the standardization grid, you will have actionable feedback to make adjustments to the taxonomy.

Closing Remarks

Validating a taxonomy is an essential step in designing it. Whether you perform a manual card sort, use OptimalSort or any other online tool, card sorting is a recommended and highly proven approach to confirm with the end users or stakeholders that the taxonomy design is going in the right direction.

If you need help designing a taxonomy or conducting a card sorting exercise to validate a taxonomy, EK is here to help. Reach out to us and our consultants would be happy to assist you. To learn more about Taxonomy Design, check out the blog posts from EK’s CEO, Zach Wahl: The Art of Taxonomy Design and Taxonomy Design Best Practices.