Profile image
By webdesignerdepot (Reporter)
Contributor profile | More stories
Story Views

Last Hour:
Last 24 Hours:

How do you know when a design fails a usability test?

Wednesday, March 1, 2017 7:24
% of readers think this story is Fact. Add your two cents.

(Before It's News)

align="left" alt="" width="100%" style=
"max-width:100%; width:auto;" /> A handy technique I learned from
the wrong job…

Years ago, I spent an awkward patch of my career as an
instructional designer, creating courses for online learning. It
was a bad fit and I moved on happily, but one part of that job has
made me a better UX designer: learning

Learning objectives are simply what you want the student to
learn by the end of the training. If there’s a test, the test
questions should be based on those objectives — otherwise, what’s
the point of the test?

The same approach comes in handy for figuring out whether a
design has passed or failed a usability test. Just remember: it’s
the design that’s being tested, not the participants.

What does the test participant need to do or say for you to feel
confident that the design has succeeded? Do they need to track
three hours of time for a particular project? Generate an invoice
to a client based on that tracked time? Send the invoice? That’s
your test criteria.

Of course usability testing is about observing how users
complete tasks, but what will you get them to do, exactly? The
beauty of these criteria is that they steer you away from vague
testing goals like, “understand how time tracking works.” How will
you know they’ve understood it? You get them to describe it. And
once they’ve described it accurately, you can say that aspect of
the design was successful.

Success criteria help you twice over: they clarify whether your
design is really successful, and they make it easier to share those

Verbs are magical

The book that taught me about learning objectives, George
Piskurich’s ""
target="_blank">Rapid Instructional Design
, offers a handy list
of behaviours to start your success criteria.

For example, the objectives for comprehension might be
“describe” or “demonstrate”. Again, “understand” is no good — you
need them to say (that is, describe) or do (that
is, demonstrate) something that proves to you that they’ve

And then, at a higher degree of difficulty, a participant might
“explain” or “organize”; at a higher level still, they might
“create” or “evaluate”.

Whatever verb you choose to start your success criteria, the
point is that you can observe whether or not a user has actually
said or done whatever constitutes task success.

“By the end of this session…”

So, when you’re planning your next usability test, and you’re
working on tasks, start by asking, “What should a user be able to
do with (or say about) this design?”

Then, you might write something like this:

By the end of the session, the participant should be able

  • track three hours of time for a particular project;
  • generate an invoice to a client based on that tracked
  • describe the difference between tracking time and logging

Now you have three success criteria and, based on those, you’ve
also got a pretty clear sense of what tasks you’ll need to give the

One caveat: success criteria aren’t quite the same as tasks.
Tasks have more context; they’re written to be read to the
participant, and might include some context about the task,
particularly if you’re steering them to find something in your
prototype. For example:

Success criteria: Generate an invoice to a
client based on that tracked time

Task: “Now that you’ve tracked three hours on
the Atlas project, show me how you would invoice Acme Products for
your time.”

Pretty similar, obviously, but success criteria are for
you and your team; the task is for the participant
in the
context of the usability session.

And you’ll notice that one of the success criteria above is
about describing something, rather than completing a task. It might
be a follow-up question to a task. These are handy for validating
whether your design’s mental model is clear to users. I’ve seen
users find their way through a task, but then describe to me a
mental model of the app which is at odds with how it was designed.
That’s task success for one participant, but more importantly
there’s an underlying problem with matching that participant’s
mental model.

So, start with your success criteria, then write your tasks and
follow-up questions based on your criteria.

Stakeholders love success criteria

Stakeholders don’t necessarily care about your process, but they
really care about the results. And if your presentation of the
results is vague, they will be rightfully irritated.

“The user managed to track a few hours, but we weren’t sure
whether she understood that tracking time isn’t the same as logging
it against a client…” Well, why aren’t you sure? Isn’t it your
job to figure this out?
You’re wasting their time, and not
giving them clear direction on how to fix the UX problems — which
is also your job, right?

Success criteria help you twice over: they clarify whether your
design is really successful, and they make it easier to share those

We’ve had some success tracking success criteria in a simple
table, and colour-coding the results. Like so:

width="500" title="" alt="" />

We whip up a colour-coded table of results (green = success,
red = failure) on our wiki. In the top row, we list participants;
in the left column, we list our success criteria. It’s ugly, but
quick and useful.

This is easy to scan, shows pretty clearly where the problems
are, and grounds the results in the experiences of actual
participants. We also list a bullet-point summary of results and a
list of usability problems and recommendations just beneath it.
We’ll zero in on those problems and iterate until we believe
they’re solved. Your process might be a little different — maybe
you’re a consultant handing over a report to a client, for
example — but the benefits are the same.


The document has moved ""

Apache Server at Port 80

height="1" width="1" alt="" />


Report abuse


Your Comments
Question   Razz  Sad   Evil  Exclaim  Smile  Redface  Biggrin  Surprised  Eek   Confused   Cool  LOL   Mad   Twisted  Rolleyes   Wink  Idea  Arrow  Neutral  Cry   Mr. Green

Top Stories
Recent Stories



Top Global


Top Alternative



Email this story
Email this story

If you really want to ban this commenter, please write down the reason:

If you really want to disable all recommended stories, click on OK button. After that, you will be redirect to your options page.