Corrections or additions?
This articles by Doug Dixon was prepared for the
2000 edition of
U.S. 1 Newspaper. All rights reserved.
Manifest Technology: Bill Joy
Our Technological Future: Star Trek or Terminator?
A Warning from Bill Joy of Sun Microsystems
by Douglas Dixon
In a controversial cover story in the April issue of
Wired magazine, Bill Joy, corporate executive officer and chief
at Sun Microsystems Inc., challenged technologists to consider the
moral issues of their work. Joy’s article, "Why the Future Doesn’t
Need Us," warns that the most powerful 21st century technologies
— robotics, genetic engineering, and nanotechnology — have
the potential to make humans an endangered species
Joy’s manifesto has made him the front man on this issue, as he calls
for a broader public discussion of the risks of technology, and even
the possibility of voluntary relinquishment of scientific
and technological progress in these areas. Since his article was
he has continued to address these issues in follow-up articles and
public talks, at a rate of at least one per week
At the invitation of the Institute for Advanced Study, Joy will speak
on "Science and Safety in the Information Age," on Wednesday,
October 11, at 4:30 p.m. in Wolfensohn Hall. This is the first in
a series of public lectures sponsored by the Institute’s School of
Social Science as part of a year-long exploration of "Information
Technology, New Media and the Social Sciences." It is free, and
a reception will follow. Call 609-734-8000.
Joy’s article has been compared to Einstein’s 1939 letter to President
Roosevelt alerting him of the possibility of a nuclear bomb. Joy
to the Pugwash Conferences, which have been held since 1957 to discuss
arms control, as a model of how to address these issues. "It’s
unfortunate that the Pugwash meetings started only well after the
nuclear genie was out of the bottle — roughly 15 years too late.
We are also getting a belated start on seriously addressing the issues
around 21st-century technologies — the prevention of
mass destruction — and further delay seems unacceptable,"
he says in the Wired article.
Bill Joy is no average Joe. He received his B.S. in electrical
from the University of Michigan in 1975 and an M.S. in electrical
engineering and computer science from the University of California,
Berkeley, where he led the design of the Berkeley version of the Unix
operating system. Berkeley Unix was an early example of the almost
viral spread of software in the computer industry, since it was made
widely available and in open source form for others to learn from
and improve. As a result, it became the standard in education and
research, and the common reference point for a generation of
Joy left Berkeley in 1982 as a cofounder of Sun Microsystems. He has
spearheaded Sun’s open systems philosophy and has lead its technical
strategy in both software and hardware. He became chief scientist
in 1998 and was called by Fortune magazine "The Edison of the
Joy was inspired in his thinking by Ray Kurzweil, a
prolific inventor and entrepreneur in artificial intelligence
In his recent book, "The Age of Spiritual Machines," Kurzweil
paints a glowing picture of the future evolution of mankind and its
merger with technology. At the current rate of technological progress,
he predicts that a personal computer will match the processing power
of the human brain around the year 2020.
This issue of the exponential rate of change is the underlying theme
of Kurzweil’s predictions and was the wake-up call to Joy. "We
will see 1,000 times more technological progress in the 21st century
than we saw in the 20th," says Kurzweil. "It’s remarkable
how people fail to internalize the implications of this."
But we live in linear time, living from day to day and year to year,
and have difficulty appreciating how exponential growth will cause
dramatic changes in only 10 or 20 years, changes that we can only
imagine today as far-out science fiction. This is difficult even for
technologists and scientists; the recent success in sequencing DNA
is only one example of an accomplishment achieved much sooner than
predicted by the experts, or even thought possible only a few years
The rate of change in computing is described by Moore’s law, named
for Gordon Moore of Intel, which has correctly predicted the
rate of improvement of semiconductor technology for decades. And we
have all experienced this improvement, as processor speed has
to double and redouble every year or two, jumping from 5 to 20 to
1,000 MHz (million operations per second) over the past two decades.
Now, imagine this rate of change continuing for the next 20 years,
with speeds for common personal computers growing from billions to
trillions of operations a second. And it’s not just processor speed
changing at this rate, but all areas of technology, from storage
to communications bandwidth, to shrinking component sizes. This kind
of growth is not just seen in computer technology, but in other areas
like biotechnology as well.
Some have argued that the end is in sight for this accelerating growth
in performance and shrinking of components, that chips are reaching
physical and atomic limits. But Kurzweil points out that computer
design has reached physical limits before, yet has actually continued
on this curve because of a series of paradigm shifts to new
from electro-mechanical, to relays, vacuum tubes, transistors, and
silicon. And the technological basis for the next paradigm shift is
in sight: molecular and optical computing.
Once you really accept the implications of this exponential rate of
growth, as Joy has done, then it becomes clear that what were once
thought of as wild science-fiction scenarios will happen, not only
in our lifetime, but in the next decades.
"Because of the recent rapid and radical progress in molecular
electronics — where individual atoms and molecules replace
drawn transistors — and related nanoscale technologies,"
Joy in Wired, "we should be able to meet or exceed the Moore’s
law rate of progress for another 30 years. By 2030, we are likely
to be able to build machines, in quantity, a million times as powerful
as the personal computers of today."
Kurzweil has a utopian view of these possibilities in his book,
subtitled "When computers exceed human intelligence." Joy
is less sanguine, arguing that concerns about these kinds of 21st
century technologies are more serious than even the nuclear technology
of the 20th century:
"The 21st-century technologies — genetics, nanotechnology,
and robotics — are so powerful that they can spawn whole new
of accidents and abuses. Most dangerously, for the first time, these
accidents and abuses are widely within the reach of individuals or
small groups. They will not require large facilities or rare raw
Knowledge alone will enable the use of them."
"Thus we have the possibility not just of weapons of mass
but of knowledge-enabled mass destruction, this destructiveness hugely
amplified by the power of self-replication."
"Genetic engineering technology is already very far along,"
he writes. "The USDA has already approved about 50 genetically
engineered crops for unlimited release; more than half of the world’s
soybeans and a third of its corn now contain genes spliced in from
other forms of life."
The promise of genetic engineering seems wonderful, from increasing
crop yields to creating cures for diseases, and even increasing our
life span and our quality of life. But Joy’s concern is that it will
become too easy, even for individuals, to make new and dangerous life:
"It gives the power — whether militarily, accidentally, or
in a deliberate terrorist act — to create a White Plague,"
some kind of new and highly contagious plague that kills widely but
selectively, like the disease in the book by Frank Herbert.
Concern with progress in robotics seems more far-out, given today’s
feeble progress in machine intelligence. Joy describes the dream of
robotics as developing "intelligent machines that can do our work
for us, allowing us lives of leisure, restoring us to Eden," but
once an intelligent robot exists, it is only a small step to a robot
species, an intelligent robot that can make evolved copies of itself.
"Given the incredible power of these new technologies," he
writes, "shouldn’t we be asking how we can best coexist with them?
And if our own extinction is a likely, or even possible, outcome of
our technological development, shouldn’t we proceed with great
Joy’s third concern, nanotechnology, is based on manipulation of
at the atomic level. Molecular-level "assemblers" could make
possible low-cost solar power, augmentation of the human immune
and almost complete cleanup of the environment.
Joy predicts that "the enabling breakthrough to assemblers seems
quite likely within the next 20 years. Molecular electronics should
mature quickly and become enormously lucrative within this decade,
causing a large incremental investment in all nanotechnologies."
But, he warns, "unfortunately, as with nuclear technology, it
is far easier to create destructive uses for nanotechnology than
ones. An immediate consequence of the Faustian bargain in obtaining
the great power of nanotechnology is that we run a grave risk —
the risk that we might destroy the biosphere on which all life
This threat has become known as the "gray goo problem,"
the uncontrolled spread of masses of replicators able to obliterate
life. "The gray goo threat makes one thing perfectly clear: We
cannot afford certain kinds of accidents with replicating
So, will the future be the bright hope of Star Trek, where earthling
morality guides the stars, or the dismal gloom of the Matrix or
where machines control the planet and humanity is dead or oblivious
to the truth?
Joy’s primary purpose in writing the article was to start a
"The new Pandora’s boxes of genetics, nanotechnology, and robotics
are almost open, yet we seem hardly to have noticed. Ideas can’t be
put back in a box; unlike uranium or plutonium, they don’t need to
be mined and refined, and they can be freely copied."
He does not offer much in the way of practical approaches to
these problems, beyond a proposal for relinquishment of research and
development. "If we could agree, as a species, what we wanted,
where we were headed, and why, then we would make our future much
less dangerous — then we might understand what we can and should
relinquish. Otherwise, we can easily imagine an arms race developing
over [these] technologies, as it did with the [nuclear] technologies
in the 20th century. This is perhaps the greatest risk, for once such
a race begins, it’s very hard to end it."
This is Joy’s challenge to technologists, and society at large:
"Perhaps it is always hard to see the bigger impact while you
are in the vortex of a change. Failing to understand the consequences
of our inventions while we are in the rapture of discovery and
seems to be a common fault of scientists and technologists; we have
long been driven by the overarching desire to know that is the nature
of science’s quest, not stopping to notice that the progress to newer
and more powerful technologies can take on a life of its own."
Institute for Advanced Study www.ias.edu
Sun Microsystems www.sun.com
"Why the Future Doesn’t Need Us" – Bill Joy Wired 8.04 – Apr
Rants & Raves, Wired 8.07 – July 2000 – Responses
Killjoy by Damien Cave Salon / Technology – April 10, 2000
Technology is changing our world — and we should be afraid! Sun
Microsystems chief scientist Bill Joy envisions a frightening future
of self-replicating machines.
Valley to Bill Joy: ‘Zzzzzzz’ by Lakshmi Chaudhry Wired News 3:00
a.m. Apr. 5, 2000
Kurzweil Technologies, Inc. www.kurzweiltech.com
Ray Kurzweil www.kurzweiltech.com/aboutray.html
The Age of Intelligent Machines, MIT Press, 1990 – The Age of
Machines, When Computers Exceed Human Intelligence (Viking hard cover,
Penguin paperback, 1999)
Corrections or additions?
This page is published by PrincetonInfo.com
— the web site for U.S. 1 Newspaper in Princeton, New Jersey.