Corrections or additions?

This articles by Doug Dixon was prepared for the

October 4,

2000 edition of

U.S. 1 Newspaper. All rights reserved.

Manifest Technology: Bill Joy

Our Technological Future: Star Trek or Terminator?

A Warning from Bill Joy of Sun Microsystems

by Douglas Dixon

In a controversial cover story in the April issue of

Wired magazine, Bill Joy, corporate executive officer and chief


at Sun Microsystems Inc., challenged technologists to consider the

moral issues of their work. Joy’s article, "Why the Future Doesn’t

Need Us," warns that the most powerful 21st century technologies

— robotics, genetic engineering, and nanotechnology — have

the potential to make humans an endangered species


Joy’s manifesto has made him the front man on this issue, as he calls

for a broader public discussion of the risks of technology, and even

the possibility of voluntary relinquishment of scientific


and technological progress in these areas. Since his article was


he has continued to address these issues in follow-up articles and

public talks, at a rate of at least one per week


At the invitation of the Institute for Advanced Study, Joy will speak

on "Science and Safety in the Information Age," on Wednesday,

October 11, at 4:30 p.m. in Wolfensohn Hall. This is the first in

a series of public lectures sponsored by the Institute’s School of

Social Science as part of a year-long exploration of "Information

Technology, New Media and the Social Sciences." It is free, and

a reception will follow. Call 609-734-8000.

Joy’s article has been compared to Einstein’s 1939 letter to President

Roosevelt alerting him of the possibility of a nuclear bomb. Joy


to the Pugwash Conferences, which have been held since 1957 to discuss

arms control, as a model of how to address these issues. "It’s

unfortunate that the Pugwash meetings started only well after the

nuclear genie was out of the bottle — roughly 15 years too late.

We are also getting a belated start on seriously addressing the issues

around 21st-century technologies — the prevention of


mass destruction — and further delay seems unacceptable,"

he says in the Wired article.

Bill Joy is no average Joe. He received his B.S. in electrical


from the University of Michigan in 1975 and an M.S. in electrical

engineering and computer science from the University of California,

Berkeley, where he led the design of the Berkeley version of the Unix

operating system. Berkeley Unix was an early example of the almost

viral spread of software in the computer industry, since it was made

widely available and in open source form for others to learn from

and improve. As a result, it became the standard in education and

research, and the common reference point for a generation of


and programmers.

Joy left Berkeley in 1982 as a cofounder of Sun Microsystems. He has

spearheaded Sun’s open systems philosophy and has lead its technical

strategy in both software and hardware. He became chief scientist

in 1998 and was called by Fortune magazine "The Edison of the


Joy was inspired in his thinking by Ray Kurzweil, a

prolific inventor and entrepreneur in artificial intelligence

technologies (

In his recent book, "The Age of Spiritual Machines," Kurzweil

paints a glowing picture of the future evolution of mankind and its

merger with technology. At the current rate of technological progress,

he predicts that a personal computer will match the processing power

of the human brain around the year 2020.

This issue of the exponential rate of change is the underlying theme

of Kurzweil’s predictions and was the wake-up call to Joy. "We

will see 1,000 times more technological progress in the 21st century

than we saw in the 20th," says Kurzweil. "It’s remarkable

how people fail to internalize the implications of this."

But we live in linear time, living from day to day and year to year,

and have difficulty appreciating how exponential growth will cause

dramatic changes in only 10 or 20 years, changes that we can only

imagine today as far-out science fiction. This is difficult even for

technologists and scientists; the recent success in sequencing DNA

is only one example of an accomplishment achieved much sooner than

predicted by the experts, or even thought possible only a few years


The rate of change in computing is described by Moore’s law, named

for Gordon Moore of Intel, which has correctly predicted the


rate of improvement of semiconductor technology for decades. And we

have all experienced this improvement, as processor speed has


to double and redouble every year or two, jumping from 5 to 20 to

1,000 MHz (million operations per second) over the past two decades.

Now, imagine this rate of change continuing for the next 20 years,

with speeds for common personal computers growing from billions to

trillions of operations a second. And it’s not just processor speed

changing at this rate, but all areas of technology, from storage


to communications bandwidth, to shrinking component sizes. This kind

of growth is not just seen in computer technology, but in other areas

like biotechnology as well.

Some have argued that the end is in sight for this accelerating growth

in performance and shrinking of components, that chips are reaching

physical and atomic limits. But Kurzweil points out that computer

design has reached physical limits before, yet has actually continued

on this curve because of a series of paradigm shifts to new


from electro-mechanical, to relays, vacuum tubes, transistors, and

silicon. And the technological basis for the next paradigm shift is

in sight: molecular and optical computing.

Once you really accept the implications of this exponential rate of

growth, as Joy has done, then it becomes clear that what were once

thought of as wild science-fiction scenarios will happen, not only

in our lifetime, but in the next decades.

"Because of the recent rapid and radical progress in molecular

electronics — where individual atoms and molecules replace


drawn transistors — and related nanoscale technologies,"


Joy in Wired, "we should be able to meet or exceed the Moore’s

law rate of progress for another 30 years. By 2030, we are likely

to be able to build machines, in quantity, a million times as powerful

as the personal computers of today."

Kurzweil has a utopian view of these possibilities in his book,


subtitled "When computers exceed human intelligence." Joy

is less sanguine, arguing that concerns about these kinds of 21st

century technologies are more serious than even the nuclear technology

of the 20th century:

"The 21st-century technologies — genetics, nanotechnology,

and robotics — are so powerful that they can spawn whole new


of accidents and abuses. Most dangerously, for the first time, these

accidents and abuses are widely within the reach of individuals or

small groups. They will not require large facilities or rare raw


Knowledge alone will enable the use of them."

"Thus we have the possibility not just of weapons of mass


but of knowledge-enabled mass destruction, this destructiveness hugely

amplified by the power of self-replication."

"Genetic engineering technology is already very far along,"

he writes. "The USDA has already approved about 50 genetically

engineered crops for unlimited release; more than half of the world’s

soybeans and a third of its corn now contain genes spliced in from

other forms of life."

The promise of genetic engineering seems wonderful, from increasing

crop yields to creating cures for diseases, and even increasing our

life span and our quality of life. But Joy’s concern is that it will

become too easy, even for individuals, to make new and dangerous life:

"It gives the power — whether militarily, accidentally, or

in a deliberate terrorist act — to create a White Plague,"

some kind of new and highly contagious plague that kills widely but

selectively, like the disease in the book by Frank Herbert.

Concern with progress in robotics seems more far-out, given today’s

feeble progress in machine intelligence. Joy describes the dream of

robotics as developing "intelligent machines that can do our work

for us, allowing us lives of leisure, restoring us to Eden," but

once an intelligent robot exists, it is only a small step to a robot

species, an intelligent robot that can make evolved copies of itself.

"Given the incredible power of these new technologies," he

writes, "shouldn’t we be asking how we can best coexist with them?

And if our own extinction is a likely, or even possible, outcome of

our technological development, shouldn’t we proceed with great


Joy’s third concern, nanotechnology, is based on manipulation of


at the atomic level. Molecular-level "assemblers" could make

possible low-cost solar power, augmentation of the human immune


and almost complete cleanup of the environment.

Joy predicts that "the enabling breakthrough to assemblers seems

quite likely within the next 20 years. Molecular electronics should

mature quickly and become enormously lucrative within this decade,

causing a large incremental investment in all nanotechnologies."

But, he warns, "unfortunately, as with nuclear technology, it

is far easier to create destructive uses for nanotechnology than


ones. An immediate consequence of the Faustian bargain in obtaining

the great power of nanotechnology is that we run a grave risk —

the risk that we might destroy the biosphere on which all life


This threat has become known as the "gray goo problem,"


the uncontrolled spread of masses of replicators able to obliterate

life. "The gray goo threat makes one thing perfectly clear: We

cannot afford certain kinds of accidents with replicating


So, will the future be the bright hope of Star Trek, where earthling

morality guides the stars, or the dismal gloom of the Matrix or


where machines control the planet and humanity is dead or oblivious

to the truth?

Joy’s primary purpose in writing the article was to start a


"The new Pandora’s boxes of genetics, nanotechnology, and robotics

are almost open, yet we seem hardly to have noticed. Ideas can’t be

put back in a box; unlike uranium or plutonium, they don’t need to

be mined and refined, and they can be freely copied."

He does not offer much in the way of practical approaches to


these problems, beyond a proposal for relinquishment of research and

development. "If we could agree, as a species, what we wanted,

where we were headed, and why, then we would make our future much

less dangerous — then we might understand what we can and should

relinquish. Otherwise, we can easily imagine an arms race developing

over [these] technologies, as it did with the [nuclear] technologies

in the 20th century. This is perhaps the greatest risk, for once such

a race begins, it’s very hard to end it."

This is Joy’s challenge to technologists, and society at large:

"Perhaps it is always hard to see the bigger impact while you

are in the vortex of a change. Failing to understand the consequences

of our inventions while we are in the rapture of discovery and


seems to be a common fault of scientists and technologists; we have

long been driven by the overarching desire to know that is the nature

of science’s quest, not stopping to notice that the progress to newer

and more powerful technologies can take on a life of its own."


Institute for Advanced Study

Sun Microsystems

Bill Joy

"Why the Future Doesn’t Need Us" – Bill Joy Wired 8.04 – Apr


Rants & Raves, Wired 8.07 – July 2000 – Responses

Killjoy by Damien Cave Salon / Technology – April 10, 2000

Technology is changing our world — and we should be afraid! Sun

Microsystems chief scientist Bill Joy envisions a frightening future

of self-replicating machines.

Valley to Bill Joy: ‘Zzzzzzz’ by Lakshmi Chaudhry Wired News 3:00

a.m. Apr. 5, 2000,1282,35424,00.html

Kurzweil Technologies, Inc.

Ray Kurzweil

The Age of Intelligent Machines, MIT Press, 1990 – The Age of


Machines, When Computers Exceed Human Intelligence (Viking hard cover,

Penguin paperback, 1999)

Previous Story

Corrections or additions?

This page is published by

— the web site for U.S. 1 Newspaper in Princeton, New Jersey.

Facebook Comments