Product Testing: For Usability and Market Research
Corrections or additions?
These articles by Michele Alperin and Barbara Fox were prepared for the July 11, 2001
edition of U.S. 1 Newspaper. All rights reserved.
In Software Ergonomics, Usability Is Key
The test laboratory is set up like any other Princeton
Forrestal Center office, with a desk and a computer. But instead of
the standard issue corporate art a large mirror covers half the wall.
Behind this mirror, really a one-way window, is an observation room
where two dozen people are clustered to watch a software
demonstration.
They stare at an enlarged computer screen identical to the one viewed
by the test subject. Another screen captures the test subject’s facial
expressions.
Siemens Corporate Research hopes this new facility at 755 College
Road will help its engineers develop better products — everything
from software to washing machines. For a long time product development
has been the purview of "techies" who, if they thought at
all about end users, figured that consumer needs were pretty much
a clone of their own requirements. "There was a general assumption
that the customer would adapt to the product," says Heidi
Kroemker,
head of corporate technology user interface design for Siemens
Corporate
Research.
"User friendliness" is no longer simply an option that adds
value to a product, but rather it is integral to product quality on
par with the product’s technical functions. "The quality of a
product is defined by the quality of the usability of that product
by the target market," says Kroemker. The press of new
technological
possibilities and the integration of computers into nearly all complex
products, she explains, is making software ergonomics, the
user-oriented
creation of software, ever more important.
Siemens Corporate Research opened its third User Interface Design
Lab this spring. Like its twins in Munich and Beijing, the Princeton
lab is responsible for designing user interfaces and evaluating how
real users respond to potential products. Although the lab will devote
about 80 percent of its time to evaluating Siemens products, its
services
eventually will also be available to outside companies.
The first Siemens usability lab, established in 1994, was the first
of its type in Germany. The lab was supposed to demonstrate how rapid
technological change and increasingly international markets are
mandating
sensitivity to product users.
But it takes education and exposure to usability concepts to convince
product developers. Even in the face of expert opinion from the labs’
interdisciplinary teams of computer scientists, industrial engineers,
physiologists, linguists, designers and psychologists, development
engineers are typically doubters. "Some of them couldn’t believe
their eyes when they saw how customers struggled with a product,"
says Kroemker. "We had to work hard to make development engineers
realize how important it was to consider the issue of user interface
design and usability engineering in the very early stages of product
development."
An example: Klaus Obersteiner was product manager of the group in
Munich that developed a new fire alarm system for Siemens Building
Technology. He says the doubters in his product development group
worried that user testing would mean lost time and money. But after
the product was tested against consumer needs, he says, the market
demand for the product doubled over initial projections.
So how did he get his group to incorporate a user perspective? They
wanted an innovative product, and money was available, so he just
had to convince them to "trust that it will work."
Kroemker highlights a number of factors that influence the procedures
employed in the user interface labs:
input that feeds product innovation, you need to develop special
methods
for acquiring and evaluating new ideas.
in the product’s next generation, you need procedures that analyze
functions for their value and meaning.
are joined together to serve the needs of particular users. To
effectively
combine these modules, designers must know how a particular group
of users thinks and what tasks it must complete.
demands systematic communication between representatives of marketing,
development, and sales to achieve a broad perspective that includes
all necessary knowledge and experience.
requires
more efficient methods of product development, including design.
internationally,
user interfaces must account for the specific cultural needs of
different
groups of users, and designers must be sensitive to different cultural
expectations.
sadness or mourning, white washing machines would be a no-no. User
preferences on how to learn to use a mobile phone — whether to
ask friends, use trial and error, ask sales assistant for help,
imitate,
or read documentation — also vary from culture to culture.
"The formation of a man-machine interface should be the start
of every design project," writes Kroemker. When the user interface
is developed early, "the development process can run smoothly
without expensive changes and time loss due to alterations."
So how does product design from a user perspective work? Kroemker
explains.
Determine necessary product functions. To do so, look systematically
at the tasks users must complete and the problems they must solve.
Information is gathered via focus groups, analysis of competitive
products, and expertise from marketing, development, and sales as
well as product development. Also, analyze how the user thinks and
operates. Given the increased number of possibilities for user
communication
with software — through movements and speech — designers must
find the correct combination of communication elements, depending
on both the user and the task at hand.
With this research complete, the next step is to create a tentative
product design. Then, perform usability tests to gather input from
experts and newcomers as well as frequent and infrequent users.
Gavriel Salvendy of Purdue’s School of Industrial Engineering, the
keynote speaker at a demonstration for clients in the first week of
May, suggests that two levels of usability must be evaluated: A
product’s
"apparent" usability is the user’s first impression of the
interface; if it does not seem accessible, the user may not even want
to try it. The second level is the inherent usability of the product,
what it feels like to use once the user has tried it out. After
initial
field tests, make adjustments to the product design based on user
input.
Those who came to the lab opening on College Road were treated to
a live usability test of the user interface of the website
Standing in the observation room, they watched as a visitor, Ana
Greenspan
of Somerset-based OSS Nokalva, performed the following task: You are
looking for a new job, but you don’t want to move, and you hate
commuting.
Find posted job offers within 25 miles of Princeton. As she performed
her task using the jobcircle.com site, she was asked to think aloud
about what hindered and helped her. As she was guided through the
task by Kem-Laurin Lubin, a human factors specialist at Siemens, she
made a number of discoveries about the site’s usability:
30.
well as the screen she was seeing on her computer, and these were
displayed on the wall of the observation room.
In a real test, Lubin explained, there would be follow-up to try to
clarify and objectify the user’s subjective reactions. The user might,
for example, be shown her confused facial expression at a particular
point in the test and ask what she was thinking then. Or, she might
be asked to rate the user interface on a scale of 1 to 7 and explain
her reasons for the number she gave. In a period of burgeoning
E-business,
problems in the interface with the user become apparent much sooner.
In the three months since it opened, the laboratory
has done work for various Siemens business units, including
redesigning
a business to business website to make it more efficient. "We
had a lot of Siemens’ clients volunteering and wanting to help to
improve the usability of the website," says Arnold Rudorfer, who
is in charge of the Princeton lab. "The main motivation is to
make the working tool more efficient to use. The purpose of the
website
is to order equipment and to get service, so a better website helps
them increase customer satisfaction by providing timely service."
The lab has two full-time and two temporary staff members and expects
to hire more next year. Rudorfer, 34, studied at the Technical
University
of Graz, and has a master’s degree in software engineering and a minor
in business administration. He has worked in Spain and Sweden and
came to this job in April, 2000.
In the real world, the results from user testing are sometimes
unexpected.
For Obersteiner’s fire alarm system, the final product included a
relatively high-cost touch screen to simplify use, a comfortable
keypad
for frequent use, shortcuts for easy operation, and icons designed
for easy recognition.
Improvements in product usability mean increased user productivity
and satisfaction, reduced sources of error and confusion, and
decreased
training time. "If the competition is doing a better job,"
says Lubin, "they’re way ahead."
— Michele Alperin
Forrestal Center, Princeton 08540. Thomas Grandke, president and CEO.
609-734-6500; fax, 609-734-6565.
Top Of Page
Product Testing: For Usability and Market Research
Testing products for either usability or market research
purposes is a big business, and Princeton has more than its share
of the latter. Princeton has more than 50 market research firms,
including
such well-known ones as Gallup, Opinion Research, Mathematica, and
Roper Starch.
But Princeton does not lead the country in actually hosting usability
focus groups. Opinion Research Corporation had maintained a focus
group room when it was located on North Harrison Street, but now only
one company, Research 100, can rent out an actual focus group room
with a one-way mirror for observation. The next nearest rooms are
located in Woodbridge, Edison, and Marlton.
Siemens has just set up a room like this, but as explained in the
article above, that is mostly for company use. If you want to rent
the Siemens usability lab, you have to knock on the door — the
company is not actively marketing it, at least not right now. And,
as the Siemens people point out, their usability lab is
technology-oriented
and meant for use by one person a time. It won’t hold a group of 10
people chatting with a moderator.
In contrast, Research 100 is eager to rent its room at Princeton
Service
Center for both usability and market research studies. Founded in
1975, it offers custom market research to manufacturers, advertising
and service organizations and has 10 employees in 5,000 square feet.
A 1981 graduate of Ithaca College, Mark H. Sandler went right into
the family business, and is now president, while his 65-year-old
father
remains CEO.
For usability test purposes, Research 100 works with only one person
at a time, but for market research discussions it can hold 12
respondents
plus the moderator. The 22 by 18-foot focus group room was recently
refurbished with a new modular conference table, new seating and
lighting,
and an upgraded camera. The viewing room, equipped with the requisite
one-way mirror, holds about 15 people and has an adjoining client
lounge for refreshments and discussion.
Any technology needed — computers, for instance — is added
to the room for a particular project. This contrasts with the
laboratory
at Siemens, which brings in whatever equipment is needed for each
test.
Just having a well-equipped room or lab facility is not enough, says
Sandler. The key to good research is who you find to test the
equipment
or give the opinions. And the jewel in Research 100’s crown is its
database of prospective participants. "We have a database of 7,000
people who have agreed to participate in focus groups. That allows
us to select people as appropriate for a particular project. We call
them on the phone to see if they qualify. If they use a certain brand
of shampoo. we invite them in," says Sandler.
For a standard two-hour group, consumers generally get paid $50.
Groups
of doctors might get $250 for 90 minutes. "You do those at 7 a.m.
and feed them breakfast and get them on their way," says Sandler.
Small business owners might get paid $100 to $125 depending on what
criteria they need to meet; if they are supposed to have a small
network
with a server, they would be paid more. "Virgin" participants,
those who have never participated in a market research project, are
particularly valuable.
Anyone who wants to participate in market research is invited to sign
up by calling Sandler, and by the end of the summer signups will be
offered at a website.
For clients to just rent the room costs $450 a session or $1,200 for
the day, including audiotaping. Add recruiting to that, for instance
— finding two groups of 12 women for a standard consumer product,
like toothpaste — would cost $2,500 for recruiting plus $900 for
the sessions. And to do an entire project — develop the
questionnaire,
write the topic guide, recruit and pay the respondents, provide the
facility and the moderator, tape the sessions, and deliver the written
analysis — costs from $10,000 to $12,000.
Don’t think that Research 100 depends on rents for its livelihood.
A maximum of 10 or 15 percent of the company’s income is related to
focus groups at this location. Most of its qualitative studies are
done in cities close to the client, such as New York, Chicago, Dallas,
Houston, Tampa, Fort Lauderdale, and Atlanta. The company also does
quantitative studies such as in-store or telephone research.
What makes a good facility?
respondents
who are what we asked for," says Sandler. "You know who the
ringers are when they open their mouth. They tell you what they think
you want to know about them."
helps, says Sandler. His favorite spot in Florida is located in a
high rise overlooking Tampa Bay, "but the reason we go to that
facility is that the recruiting is excellent."
And here’s an economy tip worthy of notice from Alan Greenspan.
"When
the economy gets soft," says Sandler, "our qualitative work
increases rather than our quantitative." Telephone surveys are
expensive, so when the accountants are seeing red, they are hard to
justify. "But rather than make a business decision blind, our
clients ask for `a small study,’ which might be a focus group."
"We have been tracking the relationship between our work and the
economy for more than 20 years," says Sandler, "and our work
tends to lead the economy by about six months. When our business goes
up in terms of qualitative, in six months the economy will go down.
Right now, based on the business mix we are seeing, we are suggesting
that the economy will be soft through the end of the year."
— Barbara Fox
08540. Mark H. Sandler, president. 609-924-6100; fax, 609-452-0138.
Corrections or additions?
This page is published by PrincetonInfo.com
— the web site for U.S. 1 Newspaper in Princeton, New Jersey.
Facebook Comments