Discuss: Usability Testing Demystified
by Dana Chisnell
- Editorial Comments
12 Control Testing Costs in the Planning Phase of Dev
@Bill Leonard
I come from a background in game design and have implemented some of the techniques that I learned from game development into my web development projects.
I always create a design document upfront that not only describes in text exactly what the goal of the project is, but also provides well thought out visual diagrams of how the application will work. The diagrams do not have to be an exact replication of what the user-interface will look like. Think of it as the sketch that a painter lays on a canvas before actually painting. The main goal is to be able to quickly create and modify a tangible representation of the process that the user will have to go through in order to achieve the desired goal.
This design procedure provides the ability to immediately identify unnecessary actions that the user will have to go through to navigate your application. You can also easily create several diagrams of different ways that the navigation could work. You can then have a simple pre-test by presenting the diagrams to multiple people from your user demographic and getting a consensus on which UI seems to be the most user friendly.
This will allow you to focus your development from the start before spending too much of your budget. You should always do a usability test once you’ve got a working prototype, but a good design document goes a long way in making sure that the first prototype is not a waste of time, money or resources.
posted at 04:45 am on October 14, 2009 by timmcart
13
The problem about testing the usability of early prototypes is that you have to find many more people to test it on. I realized that most of the people who tested early versions of a design do not want to spend time over again. Probably enterprises with a considerable budget do not have difficulties solving this problem, but for small projects good usability testing is challenging anyway.
posted at 09:49 am on October 15, 2009 by patrick_l
14 Early testing, prototypes and costs
Bill Leonard
timmcart
@patrick_l
Great discussion. Costs are always a concern; budget and time for gathering feedback from users must be built in to a project plan from the beginning.
For help with costs, you might want to check out a book by Mayhew and Bias called Cost Justifying Usability Second Edition.
The idea of testing early on prototypes is that the team must be testing something that it is willing to experiment and if the design being experimented on doesn’t work, then be willing to throw it away (or part of it), and start over (or revise heavily). Prototypes are trials – they’re meant to be disposable but also to be cumulative. As you learn more, each successive one gets more usable and more detailed. For excellent help with prototyping software, web app, and web site UIs, see Carolyn Snyder’s Paper Prototyping. It’s brilliant and makes the whole process easy.
To say that you need a lot of users to test a prototype isn’t quite right. You can test the first prototype with 1 or 2 or 3 users, and you’ll probably get data right away. That is, just within those few people, you will see where there are frustrations and hindrances. Then you can change the prototype and test with a few more people. Though there’s a lot of discussion about how many people to have test a design, my rule is use as many as you need to feel confident in the design decisions you make. If you observe one person having a particular problem and feel like you know just what to do about it, go ahead and change the prototype UI and test again. With luck, you’ll find different problems, or be able to refine what you did in the last round.
In any case, usability testing is never meant to be a one-shot thing. It’s part of an iterative process of improving a design. The idea is that you’re learning what you need to know from real people to help you make well-informed design decisions.
posted at 10:39 am on October 15, 2009 by Dana Chisnell
15 Sourcing, recruiting, and selecting participants
Several commenters have expressed some concern about recruiting participants for user research and usability studies.
This is the most important step to get right in user research. If you don’t have the right users, you don’t have the right data.
So, that sounds hard and expensive, but it doesn’t have to be.
When you’re just starting out with a design, you can use people close to you. People in your company, people in your family, or friends — but they still must be like your real users. And they shouldn’t know anything about what you’re trying to do. That is, they should not be people on your project. Make sense?
Usually, you can just say, “Can you give me a few minutes to try something out?” And they almost always will. Buy a cup of coffee or soda for them and thank them profusely. Not costly at all.
As the prototypes become more real and you want to do more formalized testing, you can keep costs lower by going to the participants rather than setting up a formal test in a lab (unless your company has a lab and doesn’t charge you back for using it). Visit them in their offices or wherever they would normally use the design.
This is a less expensive option than asking participants to come to you because you’re using less of their time. When you ask them to come to you, though it isn’t obvious, you’re also compensating them for travel time and the hassle of getting to you. If you visit them, you also get the benefit of bonus data because you’ll learn things by being in their environment that you won’t pick up from their being in your space.
I’ve got a lot of articles on my blog about recruiting. You might want to check those out. This link will get you to all the posts tagged with “recruiting”: http://usabilitytestinghowto.blogspot.com/search/label/recruiting
You might also want to read my article on Boxes and Arrows about how to treat study participants: http://www.boxesandarrows.com/view/why-we-call-them
posted at 10:49 am on October 15, 2009 by Dana Chisnell
16 Treating your participants appropriately
@ Dana Chisnell
Thanks for providing the great resources. The article on Boxes and Arrows is spot on. Many of the projects that I have been working on recently have extremely tight budgets. That being said, we are usually pinching pennies during development. Treating my testers well has really paid off for me.
I’ve been able to establish a good relationship with 5-6 people who will participate in tests as often as I need them. These users are excited to see how their feedback effects our applications and always willing to test the prototypes for no cost. Usually, we will provide lunch or other amenities before or after the test. That is enough to keep them happy.
These guys are great at finding bugs in our systems and declaring what changes they would like to see. I always test on a larger group once my core group of testers seem comfortable with the app. This process saves us a good amount of time and money.
posted at 04:49 pm on October 16, 2009 by timmcart
17 Most interesting
Thanks for that article, very informative.
Whenever I’m designing and considering usability, I always think of the classic example of the newly built student campus. Put the buildings in place, but not the footpaths. After a couple of months, the students will naturally choose the easiest path between buildings and wear a groove in the grass. Build the footpaths there!
posted at 02:35 pm on October 19, 2009 by deltacubed
18 Hmmm
Interesting article.
I’m a big believer in testing but sometimes I get lazy. I think the feedback idea rather than testing is a great idea.
I think content is key but visuals are important
posted at 10:04 pm on October 19, 2009 by Orlando Website Design
19 There's Always Something to be Learned
Usability testing may be an arduous process and it could very well be a luxury for some websites, but no matter what, it will always provide some insight into your website and your users.
It’s amazing to realize that you can spend weeks hammering out a quality sitemap, navigation structure, and labels for important links, only to find that the actual visitors you are targeting don’t use the same terminology you do, and therefore can’t find what they’re looking for (even if its right in front of them).
Usability testing doesn’t have to be burdensome — hold a simply focus group and ask participants to complete a “homework assignment” you’ve come up with. In the end, you can see how many were successful and give them a survey to add in their thoughts and frustrations with the project.
posted at 01:47 pm on November 17, 2009 by psuhiker
20 New Unique Remote Testing Tool
Thanks Dana, great article. Thought you might be interested in our new fast and inexpensive service.
It can be used for any type of online property (websites, website prototypes, online adverts, search processes, etc).
It has a very attractive delivery speed, and price point (24-48 hours from order, 299$ for a 5 person user experience test). The features are as follows:
Clients define a target url (their own or that of their competition or best practice)
Clients define a goal for the testers to perform (e.g.; “find product x and take it through the checkout process..”)
Clients define the demographics of the kind of testers they would like
Clients define survey questions
Within 24-48 hours clients receive a report that includes, for a minimum of 5 testers:
Web cam recordings of the testers conducting their assigned goal/task
A synchronized recording of the entire screen session during the test
ClickFlow Analysis
Contextual written “bubble” commentary on screenshots
Survey results
Other quantitative data
Visitors to the site (www.userlytics.com) can request a free 1 person sample test.
Kind regards,
Alejandro
posted at 01:36 pm on December 7, 2009 by userlytics
Got something to say?
Discuss this article. We reserve the right to delete flames, trolls, and wood nymphs.
Create a new account or sign in below if you’d like to leave a comment.
Subscribe to this article's comments: RSS (what’s this?)
11 Costs associated...
Many of us would like to do user testing, but how can resources which are typically thin be allocated? Resources meaning time and/or money.
Specifically, how can you gain meaningful input on something that s quite frankly not even to the “usable” stage in production yet, and getting it there would eat up some of the budget… only to learn from the testing that things have to be re-done? How is this reconciled?
posted at 03:21 pm on October 12, 2009 by Bill Leonard