How to Conduct a Card Sorting Study?

UX DesignArmen Ghazaryan • December 21, 2015 • 5 minutes READ

Opinions differ just like tastes. That is why when making important design decisions we need to consider what users think. We may argue whether contact information should go in the “about us” section or in the footer or elsewhere and each team member may be right in a sense, but the only opinion that matters is that of the user.

But how can you incorporate users’ ideas into the information architecture of a product?

I suggest card sorting. It’s simple, easy to conduct and analyze and it’s an informative UX design process.

Card sorting is a user study where participants organize content pieces into groups based on topic similarity. The group labels may be predefined (closed card sorting) if you already have some kind of information architecture and navigation. However, it’s even more informative to leave group labeling to the participants (open card sorting), especially if you are unsure about the category naming. This way you will get an understanding of how your users “speak” about the same topic.

Now let’s go through the process of conducting a card sorting study step by step.

Choose Your Methods

Depending on your project goals you can choose to conduct either an open or closed card sorting study. While closed card sorting results may be easier to analyze, open card sorting provides more insights into user mindsets.

Another thing to consider is the medium of the card sorting research: web/remote or traditional paper based studies. There are plenty of online services that give you every single tool you need for a remote card sorting study. Just like remote usability testing, this type of research method comes with pros and cons.

On one hand it’s easier than ever to create cards, send the test to a bunch of users and let the system analyze gathered data and give you a final category tree. On the other hand during a moderated, real life card sorting session the researcher can observe and take notes of discussions, behavior, ideas and questions, which brings a somewhat emotional touch to the raw data.

Prepare the Cards

The next step in card sorting research is to decide whether you are testing the whole IA or a part of it. For example if you are designing an e-commerce website, you may want to test the product category architecture while leaving the company related page structure as is. It is tempting to want users to sort all of the pages on your website, but be mindful not to overwhelm participants with too many tasks.

So when you have a list of pages that you want to sort, simply create a card per page and a card set per participant. Now you can either do the whole staff offline with sticky notes or you can print them out with a barcode on each card so that you can then easily digitize the data with a barcode scanner and make data input a breeze.

Recruit and Screen Participants

Again if you choose to use one of the online card sorting tools, you will take advantage of the available participant recruitment services. There are also a bunch of professional study participant recruitment services that will find the ideal target users for you to test. But this costs money and if you are short on budget you can go ahead and use all of the free means to spread the word: email lists, social media, especially LinkedIn, personal contacts, etc.

You may find it hard to recruit users free of charge so maybe some kind of incentive like a discount coupon of your product/service or a free trial would be helpful. Once you have a list of people ready to take part in the study you need to screen them to find the best matches for your target audience, then schedule and organize. And that’s it, you are ready for the card sorting session.

Run the Card Sorting Session

First, set the stage for the session. Make sure there is enough room for each participant to work with the cards, whether it is a table, a board or anything else. If you choose to run a computer-based card sorting, then each participant should be equipped with a laptop or PC. When all participants have gathered, explain in plain English how the test works.

It is important to make participants feel comfortable and at ease. They should be confident that they are not the ones being tested, but the information architecture. It would be perfect if the participants speak out loud, so that you can understand their thoughts and ideas as well, but never interrupt them yourself.

Analyze the Results

When all participants have completed their tasks, you will have a wealth of data to work with, including both actual card sorts and observations from the session. I would advise taking pictures of the whole process and especially final card groups and the way participants have organized them. This will be useful for further reference. In case of a digital card sorting you skip the data input stage and can get a dendrogram of the card sorting data analysis almost instantly. For those who prefer traditional paper-based method, data input is inevitable.

As said above it would be a bit faster to use printed cards with barcodes so that you can input the data with a simple barcode scanner. When you are done with data input you can choose to use one of the free or premium card sorting services to analyze the data for you or you can do it yourself with a quick and dirty method offered by Shanshan Ma from UXMatters. It boils down to an Excel-based table with a matrix of all cards and groups. Each card is attributed to the group where it appeared the most during the session.

Understanding and Presenting Card Sorting Results

Another challenge with a card sorting study is to actually understand the final results and be able to present them to interested parties, such as a design team, product owners or even stakeholders. The results are usually presented in the form of a dendrogram, but you are free to visualize the results the way you feel is more comprehensive. A simple hierarchic structure works well enough.

The good thing about card sorting is that it provides insight into how users navigate a website or a digital product. But for a more grounded research you need to recruit participants matching your user persona profiles which is not always easy. However, valuable data never comes easy, so go ahead and try card sorting for your IA research and judge for yourself.