Evaluating Multimedia Teaching Software: How to Design and
Implement
Usability Studies
Sandra McIntyre, M.Ed and Sebastian Uijtdehaage,
Ph.D., David Geffen School of Medicine at UCLA; Joyce Nyhof-Young, Ph.D., University
of Toronto Faculty of Medicine Centre for Research in Education
ABSTRACT:
Usability
of software is typically described in terms of five
characeristics:
learnability, efficiency of use, memorability, error
frequency
and severity, and subjective satisfaction (Nielsen 1993).
Usability
is separate from the utility of software (i.e., whether it can
perform
some function) and from other characteristics such as reliability
and
cost. Even in the context of a
well-planned development effort, with
clearly
defined functional goals, early iterations of interactive software
inevitably
exhibit usability issues. The goal of
this full-day workshop
is to
provide concrete advice and methods that can be systematically
applied
to ensure a high degree of usability in the final version of any
teaching
software program. Whether participants
are developers involved
in an
actual courseware project currently or are managers faced with the
need to
compare different courseware programs to make institutional
decisions,
the ability to conduct simple, low-cost usability testing will
strengthen
their online offerings.
The
workshop will present the "big picture" for usability studies within
the
context of the entire development cycle, and discuss the application
of human
factors considerations and the results of research into
usability. The importance of defining user goals,
developing appropriate
heuristics,
and iterating the user interface through several rounds of
testing
will be explained.
Various practical
methods for conducting usability tests will be covered,
including
"think-aloud" sessions, focus groups, structured questionnaires,
log file
analysis, and speed of information retrieval tests. The
strengths
and limitations of each method will be discussed, along with
guidelines
for selecting a suitable combination of methods and then
implementing
them. A practical setup for a low-cost
testing laboratory
will be
described. Additional considerations
for managing a testing
effort
will be explained, including human subject protection issues,
recruitment
strategies, and how to manage various resources. We will also
discuss
triangulation strategies and methods of distilling patterns from
the
testing.
Participants
will have an opportunity to work with some of the testing
methods
in the workshop. For example, they will
practice conducting a
heuristic
evaluation of a website. A think-aloud
walkthrough will be
demonstrated
with volunteers. Dr. Nyhof-Young will
present an example of
a
complete evaluation study. Finally,
participants are encouraged to
bring the
URL or Zip/CD-resident files of programs of their choosing; they
will use
the guidelines and worksheets from the workshop to develop and
share a
plan for the usability testing of the chosen program.
BENEFIT TO PARTICIPANTS ATTENDING SESSION:
Learn to
evaluate the usability of your teaching software before you make
costly
investments of time, money, and other resources. Participants in
this
full-day workshop will learn the basics of planning, running, and
analyzing
the results of usability studies. The
workshop will compare
various
methods for assessing usability, including "think-aloud" sessions,
focus
groups, structured questionnaires, log file analysis, and speed of
information
retrieval tests, along with guidelines for pattern
distillation
and triangulation strategies.
Participants will learn how to
set up a
low-cost testing laboratory and how to manage recruitment and
resources. An example of a comprehensive evaluation
study will be
presented,
and participants will have an opportunity to develop a testing
plan for
programs they have developed or are in the process of developing.
|
Sandra
McIntyre, M.Ed. Program
Manager Health
Education Assets Library (HEAL) 60-051 Center
for the Health Sciences David Geffen
School of Medicine at UCLA Box 951722 Los Angeles,
CA 90095-1722 Phone:
310-267-2873 Fax:
310-794-7465 Email:mailto:smcintyre@mednet.ucla.edu Website: http://www.healcentral.org/ |
Sebastian
Uijtdehaage, Ph.D. Assistant Professor
of Medicine Co-Director,
HEAL 60-051 Center
for the Health Sciences David Geffen
School of Medicine at UCLA Box 951722 Los Angeles,
CA 90095-1722 Voice:
310-794-9009 Fax:
310-297-0320 Email: mailto:bas@ucla.edu Website: http://www.healcentral.org/ |
Joyce Nyhof-Young, Ph.D. Education Researcher Princess Margaret
Hospital, Room 5-312 610 University Avenue Toronto, Ontario Canada M5G 2M9 Voice: (416) 946-4501
x5838 Fax: (416) 946-4442 Email: mailto:joyce.nyhof-young@uhn.on.ca |