Corrections or additions?
These articles were prepared for the January 10,
2001 edition of U.S. 1 Newspaper. All rights reserved.
Next Step for Testers: Susan Embretson
If you are sitting in front of a computer to take your
Graduate Record Exam, you might not want to know that a computer
is choosing the next question. But it is. Educational Testing
test delivery software chooses each question for individual test
based on their answers to the previous item.
ETS aims to go one step further, to enable the software to actually
devise the next question, so the question is being written "just
for you." This leading edge programming is called "on the
fly" adaptive testing, and Susan Embretson, a researcher
at the University of Kansas, is one of its pioneers.
Embretson comes to ETS to give the William H. Angoff lecture, entitled
"Second Century of Ability Testing: Some Predictions and
on Thursday, January 11, at 3 p.m. in the Chauncey Conference Center
on Rosedale Road. The lecture is free by reservation. Call Madeline
Moritz at 609-734-5035 (www.ets.org).
"Item response theory, a psychometric method by which tests are
scaled, permits GRE adaptive testing, where two people don’t get the
same test," she explains in a telephone interview. "The
item is selected for each person each time. One impact is that you
get shorter and more reliable tests. But item response principles
are just not well understood by psychologists."
Embretson’s latest book "Item Response Theory for
explains these new rules of item response theory to the test-making
psychologists. "Being able to understand the principles is the
most difficult," she says. "Other books introduce the topic
in a statistical way and give few examples."
Embretson went to the University of Minnesota, Class of 1967, and
earned her doctor’s degree there. She has taught at the University
of Kansas for 25 years, is past president of the Psychometric Society,
and a past vice president of the division of the American
Society. She has been associated with ETS on various boards since
the early 1980s. "ETS has always been the leader in psychometric
testing," she says. "It has just been fascinating for me to
see it evolve over time."
Why her initial interest in psychometric testing? "I was
in psychology, but my talents are in the mathematical area," she
says. "The first psychology book I bought that was not required
for a course was on psychometric methods, and I read it cover to cover
She focuses on the "processing burden," how many things the
test taker must keep track of. "We look at the item to understand
in what ways it is difficult," she says. She will cover 11
kinds of items in the public lecture. "I have models —
modeling of item processing — that will predict how difficult
the item will be for the item types."
The types can be divided into two major categories, those that measure
fluid intelligence versus those that measure crystallized
which allows for an individual’s encounter with culture.
Matrix completion, an item type that measures fluid intelligence,
can best be described by an example. Imagine a three by three square,
like a tic tac toe board. Your task is to discover the rules for the
nine elements and complete the matrix. The top row is simple
all black, decreasing in size. The next row is white squares, which
start with the largest. In the last row is a rectangle with a pattern,
but its third square is blank, and you have to find something to fill
it (probably a small rectangle with a pattern). Such very basic
get more complicated when test takers add different relationships.
In addition to the public lecture, Embretson will give a private
to ETS experts. "I have gone a step further," she says,
I find it interesting now to understand the items so well that they
can be constructed by a computer program on the spot to a target level
of difficulty. I have two generators for nonverbal test items, one
for the matrix completion problems and one for the spatial ability
problems." Her "on the fly" generators are embedded with
principles and stimuli. A principle might be how to combine figures
to obtain a certain difficulty level. A stimuli might be, for
circles, squares, or shading.
These generators make it possible — as a person is taking a test
— for an item to be written for that person on the spot. "This
requires a very deep understanding of the item," she says.
At ETS Embretson works with Isaac Bejar, senior director of
assessment design and scoring, a 25-person center in the research
division. Bejar went to InterAmerican University in San Juan, Class
of 1970, and has a doctoral degree from University of Minnesota.
"Our aim is to develop the `on the fly’ test delivery
says Bejar. "First we must assess the equivalence of `on the fly’
software with the regular computer-based GRE." The center recently
completed data collection in Philadelphia, at Fordham in New York
City, and at Michigan State University in East Lansing.
Initially the on the fly software will be used on mathematical
questions with analytic reasoning coming next. Another of Bejar’s
research area is adapting test content to other languages. "What
we do here are the studies," Bejar cautions. "Implementation
is in the hands of the business units that are responsible for the
testing programs. We hope our research will be so compelling that
the business units will want to deploy this software within the next
Should item writers fret that they are going to be out of their jobs?
"That’s right," says Embretson, "but they are going to
have different kinds of jobs. They will be looking for principles
If you as a test taker are worried about computers fashioning your
verbal questions, stop worrying, at least for now. "That is
people are only talking about," says Embretson.
"We are beginning a very strong initiative in verbal," says
Bejar, "but we anticipate the verbal will raise new challenges
we have not seen before, to put it mildly."
— Barbara Fox
Those new repetitive motion regulations — the ones
proposed by the federal Occupational Safety and Health Administration
— are running into trouble even before they are supposed to take
effect on Tuesday, January 16. William Guhl, administrator of
Lawrence Township, has issued a clarion call to other municipalities,
protesting that regulations will cost too much. And at least one
society, the American College of Occupational and Environmental
(ACOEM), has withdrawn its support for the Ergonomics Program
In response, the Corporate Health Center has put together a breakfast
workshop entitled "2001: Ready for the New Ergonomics Rule?
Motion Injury, Carpal Tunnel, Musculoskeletal Disorders."
for Thursday, January 11, at 7:30 a.m. at the center’s headquarters
on 832 Brunswick Avenue in Trenton, it will feature Michael J.
Makowski MD, the center’s director, and Robyn Agri
of the center. The workshop is free by reservation; call 609-695-7471.
The new standard covers 6.1 million general industry worksites with
more than 102 million workers (www.osha.gov). About 60 percent of
them are working for companies that have not yet addressed ergonomics,
so they are at risk for musculoskeletal disorders (MSDs) such as
tunnel syndrome and back injuries.
OSHA predicts that employers will need to fix or adjust 18 million
jobs over 10 years at a cost of $4.5 billion per year, but OSHA
this injury-prevention will reduce the number of MSDs by 50 percent.
Only half of work-related MSDs are reported, says an OSHA
but about 600,000 of the 1.8 million reported MSDs are so serious
that workers need time off. Injured workers are entitled to get
attention and receive pay and benefits for up to 90 days. In addition,
one-third of workers compensation costs are due to work-related MSDs.
"The workshop will cover medical treatment and workers
vis a vis the new OSHA standards," says the Corporate Health
Joe Whitty. Additional speakers will be Mary Rudakewych
of the State of New Jersey, a representative of federal OSHA, and
Mark E. Litowitz of Hill Wallack. Litowitz, who will cover how
employers can deal with repetitive motion injury in the context of
the New Jersey workers compensation system, was chief judge of the
workers compensation court division until 1994.
ACOEM has technical disputes with the medical provisions of the rules
and expects OSHA to have to defend them in court. "OSHA’s failure
to base the standard on a firm medical foundation lends credence to
the arguments that will be made in court," predicts Robert
L. Goldberg MD, president of ACOEM and assistant clinical professor
at the University of California, San Francisco.
ACOEM objects to the confusion that it says the new rules will bring
regarding whether an injury is work-related. The ACOEM doctors want
OSHA to apply the standard only to work-related disorders "for
which credible scientific evidence exists." For instance, they
feel a worker could attribute an injury to pounding a keyboard when
the injury might actually be related to the worker’s participation
on a weekly bowling team.
Goldberg faults OSHA for failing to include a medically accurate
of "musculoskeletal disorder." He also objects to OSHA’s
to require an appropriate medical diagnosis before the employee is
required to stop working from the current job. In contrast, most
compensation programs require a medical diagnosis before work is
So do the programs that deal with lead, arsenic, and asbestos
Charles N. Jeffress, assistant secretary of labor, says OSHA’s
ergonomics program standard will prevent 460,000 workers from having
painful injuries and save an average of $9.1 billion each year.
final standard identifies persistent signs and symptoms of
musculoskeletal disorders (MSDs) as problems that must be addressed,
and it increases worker participation in ergonomics programs."
Students in Mercer’s International Business Practice
Firm, a two-semester course, have the opportunity to found and run
a virtual company. This year’s company was "B-Gone Travel,"
a virtual travel agency, says Joan Jones, the instructor.
Credit courses begin Tuesday, January 16. Some are offered at off
campus sites including Ewing High School, Hamilton’s Health Career
Center, the Capital Health System in Trenton, and Trinity Cathedral.
Mercer has 62 associate degrees plus dual-admissions and transfer
opportunities to four-year schools. For information call 609-586-4800,
extension 3228 (www.mccc.edu).
B-Gone Travel is one of two "virtual businesses" that help
to train future entrepreneurs. The first company, Radiant Care, was
established in 1997 and sells personal care products via the Internet
to virtual companies at schools around the globe. For information
on Mercer’s IBPF, check www.mccc.edu. or contact Carol Weber,
program coordinator, at 609-586-4800, extension 3480.
In the first semester, students in B-Gone Travel wrote a business
plan for the company and came up with financing, each investing some
of their own "virtual" money. They issued stock,
a bank line of credit, and wrote an employee manual.
The business set up four departments: accounting, human resources,
purchasing/sales, and marketing. Depending upon their personal goals
and objectives, students in the firm took various jobs. If they wanted
to change jobs, they had to formally apply for the new position.
In this "on-the-job" learning environment, students took the
roles of actual travel agents by following up and insuring client
satisfaction. "Students developed a bank of information —
about travel to foreign countries, money exchange, passports, etc.
It included places to go and things to do," explained Jones. They
also developed a client database for future marketing.
The package they developed was a seven-night cruise, starting at $630
per person, departing from Tampa, with ports of call in the Grand
Caymans, Playa del Carmen, Cozumel, Mexico, and New Orleans. Some
members of the Mercer County Community College community, supported
the effort "virtual money to book virtual trips and have a virtual
"Our intent is give students an opportunity to establish and run
an agency," says Jones, who has been an instructor for Mercer’s
aviation and travel programs since 1985. "Students really buy
into this and it becomes their company. They take pride in it. If
there is a problem, we sit down together and they have to work it
through. It is great for learning how to work as a team, both within
departments and in the company as a whole."
"It is a lot of fun. Students all walk away feeling that they
have learned a lot more than in a traditional classroom. I shouldn’t
be called an instructor. In this class I am a facilitator."
Corrections or additions?
This page is published by PrincetonInfo.com
— the web site for U.S. 1 Newspaper in Princeton, New Jersey.