Discuss: Usability Testing Demystified
by Dana Chisnell
- Editorial Comments
2 Spot on
Great article, Dana. Thanks for clarifying the value, purpose, and flavors of usability testing. From what I’ve seen, the bad rap seems to stem from teams working with less experienced practitioners who ask the wrong questions, leading questions, or believe they must use the formal lab approach vs. doing quick or in-context testing. Hope this helps squelch the naysayers.
posted at 09:28 am on October 6, 2009 by kwalser
3
Thanks, Dana. Very good information. I’m always hearing that some testing is better than none at all. That being said, what do you think about usertesting.com? I’m considering it as an option for quick and inexpensive testing for some of our firm’s clients.
posted at 12:03 pm on October 6, 2009 by usersrule
4 Useit.com or loseit.com
@ronansprake – there was a project to redesign one of Nielsen’s pages, you can see the results at http://www.designbyfire.com/000094.html . Nielsen does not pretend that his website is an example of good aesthetic design, he knows that it is not, but says that he does not have the skills to make it aesthetically pleasing and so he would rather leave it plain and simple than make it a mess.
Despite its looks, it is one of the most popular websites out there for web authoring and usability research, which just proves his point – if you build it well, they will come … even if it looks cruddy.
posted at 03:57 pm on October 6, 2009 by Stephen Down
5 finding participants
I’ve only done one session, but I found people didn’t like being asked to “test” something – they didn’t want to take a test. I learned a lot that time, mainly about what I should and shouldn’t say.
It’s difficult to work behind a screen everyday and then go into such a public and more than likely humiliating situation. Through humiliation comes humility though. I guess.
Next time I will try the “gathering feedback” approach.
posted at 10:45 am on October 7, 2009 by David Rodriguez
6 Nothing like fingers on a keyboard
Recently while talking to a “Usability Expert” at one of our local utility companies, she asked if I had any experience with different usability software programs that spit out usability data. I must have had a blank look on my face as I thought to myself, “how many software programs are going to be using your website?”
posted at 12:45 pm on October 7, 2009 by MikeStover
7 Participants
I’ve only done one session, but I found people didn’t like being asked to “test” something – they didn’t want to take a test. I learned a lot that time, mainly about what I should and shouldn’t say.
What you need people to understand is that they aren’t being tested. They are the examiners, your website is under the microscope. If they are having difficulty, that means that the website probably isn’t well-enough designed to meet their needs. Because even if they are clearly not demonstrating a lot of intelligence, logic or resourcefulness – and there are a lot of people out there who don’t! – they are still the people that your website is there to serve, and you’ve got a better chance of changing your website to fit what people can work with than you have of changing the way they work with your website.
Yes, sometimes it’s hard. When you’ve created a design that you think is really easy to use, and some knucklehead comes along and can’t follow simple instructions, ignores the big highlighted links and buttons that you’ve put in prime spots and clicks around randomly for a while before giving up, it’s hard not to scream and cry and ask them why they can’t see the blindingly obvious when it’s right in front of them … but you need to be able to understand their thought processes and what they are looking for/at, so that you can come up with a design that they can follow.
posted at 07:36 am on October 8, 2009 by Stephen Down
8 Jakob is not out of touch
To @romansprake and Stephen Down
Stephen is right – Jakob has said this about useit.com. Visual design is not his forte.
Okay, so useit.com looks very 1997, but so what? It’s about content. Look at craigslist. Not beautiful but tons of great content that is pretty easy to get to.
What you can learn through getting feedback from users is whether which part of design matters and how much, so you can make an informed decision about where to spend resources. Jakob has decided that, based on his audience, it’s about content, not slick visual design.
posted at 01:07 pm on October 11, 2009 by Dana Chisnell
9 RE Spot on
@kwalser Thanks much!
It’s easy to do a bad job of moderating usability test sessions or user research, but I think most people are well-intentioned about it.
I actually find it much easier to ask the right kinds of questions in quick, in the wild tests because they tend to be very focused. And the context offers so much to observe. In formal lab tests there’s a lot of pressure to do it the “right” way, and less opportunity to recover if you screw up.
But mostly it just takes practice. So I take your point about people having bad experiences with usability testing because their facilitator is not experienced.
posted at 01:12 pm on October 11, 2009 by Dana Chisnell
10 RE usertesting.com
@usersrule,
Glad you liked the piece. I am all about observing people using designs to reach their own goals. It’s about being present to witness the crazy things they say and do that you could never have predicted — it’s that “ah ha!” moment that you get when a participant creates a workaround to some hindrance in the design that gives you what you need to make good design decisions. There’s no substitute for that. And more data isn’t necessarily better. It depends on the source. Two things offend me about usertesting.com.1. What I do is get people to help me evaluate a design. It is the design that is being tested, not the user. So I hate the term “user testing.” It’s usability testing.
2. usertesting.com reduces “user testing” to a QA process. It isn’t QA. It’s a tool for the team to understand users. So, to do remote, unmoderated usability testing, you have to make sure that your test is very well designed, so there are no questions about the tasks. The other thing with QA is that it’s just running stuff through a test kit. Usability testing is about learning why users do what they do in addition to the what. So although with usertesting.com you get video of people, there’s no guarantee they’re going to think aloud, and you have no opportunity to ask follow up questions.
Short answer: I’m not a fan.
Dana
posted at 01:20 pm on October 11, 2009 by Dana Chisnell
Got something to say?
Discuss this article. We reserve the right to delete flames, trolls, and wood nymphs.
Create a new account or sign in below if you’d like to leave a comment.
Subscribe to this article's comments: RSS (what’s this?)
1
Maybe Jakob Nielsen should allow a designer to style http://www.useit.com/, since this is the site I most often hear referenced by people who associate good usability with poor, outdated design
posted at 08:23 am on October 6, 2009 by ronansprake