INTERVIEW:
Question 1
What are neural networks?
Answer
A neural network is a model of the way real nerves, real sensors like eyes and ears and
brains, work. It tries to imitate so that it will work in the same way and do the same
things.
Question 2
E' possibile costruire macchine, computer e altre apparecchiature con le reti neurali?
Answer
It is possible. We believe that our thinking works in a way like that and we want to find
out how real brains work, and also to build machines to do some of the same things that
our brains, our minds do.
Question 3
But these machines are not programmable. Will they learn by themselves?
Answer
One hopes so. They do learn by themselves, by their own experiences but not as much as
people do. They are still very simple. The kinds of tasks that these machines can now do
are low-level tasks. As science improves, as the engineers and scientists, the people at
SMAU, work them and practice with them they get better, but they are still very far from
real people.
Question 4
Can you compare the ability of neural networks with the ability of animals or children?
Answer
It is not an age so much. The neural network in the machine keeps trying, but an
intelligent child stops trying after a while and gets bored. Our machines do not get bored
yet, which is a sign that they are very elementary indeed. There are tasks which they can
do for us. They will keep track of the right way to do a very easy task. But as yet they
do not have much sense of purpose of their own beyond what they are given by the people
who build them.
Question 5
That is interesting because they have to understand from the environment. How can they
understand from the environment?
Answer
That is a very interesting point. It is not that they understand so much, it is that they
work with the environment to get something done, to perceive something, to have the right
effect. But they do not really understand what the environment is or how it works. So
neural networks today do not make a model of the environment in the way that you and I
make a model of the environment, instead they merely play with what they can do until it
works.
Question 6
And can you compare the goals of cybernetics and the goals of neural networks?
Answer
The goals of neural networks are much more cybernetic than present day computers. Our
computers are nearly all programmed, that is, they are told exactly what to do. Neural
networks are not told exactly what to do. The study of cybernetics started out with
Professor Norbert Wiener at MIT, who was my adviser, studying how gets to a particular
place. The word cybernetics comes from the Greek word for the steersman on a boat, who
moved the tiller or the rudder to get the boat where he wanted to go. The steersman is
performing the goal, the seeking of the goal, the going where he wants to. At a very low
level neural networks move their connections and rewire themselves so that the machine
will do what it is programmed to want to do. In computers the programs are written so the
machine will do what the designer wants them to do. So the machines in computers do not
want. Neural networks are beginning to want, to care, to have purpose.
Question7
And when exactly did research start on neural networks ?
Answer
The beginning was a very startling paper written 52 years ago, in 1943, by two friends of
mine, a philosopher neurophysiologist called Dr. Warren McCulloch and Walter Pitts. Walter
Pitts was my roommate at one point and when he wrote this paper with Warren McCulloch he
was only 20 years old. It was a paper showing that, theoretically, a very simple neural
net could do anything that any computer could do. This was published in Chicago in 1943.
Another paper came out 3 years later by the same two people on the same subject going a
little further into this theory.
Question 8
And in 1958 you wrote "Pandemonium". What does pandemonium mean? What was the
concept of this?
Answer
The concept, "pandemonium" was a word first used by John Milton in a very long
English poem called "Paradise Lost". Pandemonium comes from the Greek
"pan", meaning all and "demonium", meaning the demons. The idea of
pandemonium is that in recognizing something - for example, recognizing a face or a
character on a page - we have a little demon for each feature, for each part of the
picture. And when the demons see themselves in the picture they shout, That's me! That's
me! and then a higher level demon listens to these other demons and decides who shouts the
loudest. If you are reading a character, a letter in a word, if the higher level demon
hears the "A" demon shout the loudest, then he knows it is an "A". The
idea is that we have separate neural nets, say, representing the demons, and what they
shout, their output, is the amount of themselves that they see, that they perceive in what
they are looking at.
Question 9
So it's a network of neural networks at the end.
Answer
Yes, in the long run neural networks will have to be built up of pieces that are neural
networks. But they still have to work together. Then the whole system does not have simple
purposes or goals but very complex ones, just like people. In that sense the neural
network is very different from the network of computers which we are talking about now
because here it is a social thing. In our society not every piece, not every computer
wants the same thing. They want to communicate but not because there is a single purpose;
they want to communicate because everybody wants to do something different. In the neural
network, in the good neural networks, they are all contributing to the same end.
Question 10
And in which direction is your research going now?
Answer
I believe there is not enough attention paid now in computing to having the machine do its
own learning and changing. They are still having to be programmed by the brilliant people
we see around us here, the very clever people who are very clever in getting the computer
to do step-by-step what they specify. What I want is for the machine to learn a little to
try its own improvements. They don't have to be clever, they only have to try a little bit
so that they get better. And if we find out how the machines can make themselves better
without people telling them exactly what to do, then we have started finding out how to
have machines which do not have to be programmed in the way we program them today. In the
long run we have to do that. We cannot keep going with the kind of programming needed to
build the very complex software which is going to be needed.
Question 11
What is the difference between intelligence and ability? All these studies are called
artificial intelligence. Is this definition right?
Answer
The difference is a baby has not much capability. A baby cannot write a poem or read a
book or make love or change his clothes, but it is the beginning of intelligence that a
baby learns and learns and adapts. It has lots of learning and adapting to do, and it has
to learn many things to get started to build in itself the tools it will need to do more
learning. One of the most difficult things a baby starts to do is to learn to talk. It
learns to talk because it wants to do the things that its mother does, its father does. So
probably there is a built-in need to talk. But as it learns to talk, it develops the
skills needed to do the next things. And it not only develops the need, it develops the
goals and how it wants to do more. You do not program a baby, you give it the right kind
of experiences and it programs him or herself. And we don't necessarily want machines to
do exactly that, but some of that would be very helpful. An example of this: suppose I
have an AI program which is reading my e-mail and picking out the important pieces which I
should see at once, by reading the words and by knowing some words are important for me,
like, money or some particular person or my child, it will read that and filter that
message out for me to look at quickly. As time changes, I get interested in different
things, I don't want to have to re-program the computing myself to change that. I want the
machine to understand. I might say: "That's not important" about a message. And
it should change the filter it uses to pick out the messages, to keep me happy.
Question 12
That's wonderful because it changes the relation between man and machine. But generally
don't you think that people are afraid of programmable machines?
Answer
Most people don't really understand about machines or programs anyhow and people will be
afraid of machines. Fear is probably more a result of unfamiliarity than it is about real
dangers. When we first had the automatic tellers, the banks when you put your bank card in
and you get money out, in America it was a long time before anybody used them because they
were unfamiliar. They were afraid of them. Now, they are more convenient and faster and
they are open weekends so we all use the ATMs because they're familiar to us. As these
machines work for us and we get familiar with them, we will end up trusting them.
Question 13
What is the most extraordinary realization made with neural networks?
Answer
What is extraordinary is the progress of the science of the research, which is not going
fast but it is going, in understanding the process and the kinds of controls needed.
Neural networks these days are 20 thousand, 20 million times faster than 20 years ago.
That is the extraordinary thing.
Question 14
But just to compare with biology: how many neurons can be built into a neural network, how
many connections?
Answer
Well, typically these days we are talking about thousands or millions. In the head it is
billions or trillions. Now the real neurons that I have in my head don't work very fast,
they work at hundred times a second, a thousand times a second. The neural networks in the
machines work a million times a second, but that's not the same difference. So it is the
size of the head and remember also the size of the training. I mean, this head has been
around a long time and it has been working all that time. We don't know really how to make
that kind of complexity and that kind of training. We don't know how to train neural
networks. If we knew how to build them, we would still not know how to train them well.
And we are still doing more programming and telling the network exactly what to do much
more than in real animals or people. It is not just the size of the head, there are some
large brains which are not very clever. The brain of a cow is a fair-sized piece of meat,
yet a cow is not very intelligent.
Question 15
Now the neural networks are very specialized. What are the typical applications?
Answer
It's hard to say. They have to do with the particular kind of control, either controlling
an output or controlling the analysis of an input like an artificial retina, an eye. You
want the eye to tell some piece of machinery is properly connected and you then use the
neural network to examine the picture and to learn what are the right features to look for
the correct connection.
Question 16
And what do you think about the hardware, the neural chip, for example, or the research
into neural computing ?
Answer
I do not think neural computing will replace computing but I do think that the possibility
of getting in hardware neural nets is very promising indeed, depending on the engineering
and the research that is still needed in understanding how to organize and how to organize
to try the right goals and to aim for the future, the kinds of flexibility in purposes and
goals, which we do not know very much how to do.
Question 17
Now it is more the universities than the companies, the industry that invest in neural
networks. What do you think about that?
Answer
I think that is the right thing. I think it is a shame that all the companies in America
want applications that will work today and make money today. It will be a long time before
anyone makes a lot of money from neural networks. What is needed is for companies to
explore and do research before they make money. And there is too much emphasis today in
getting into neural networks in October and making money by December. That will not
happen. It will be years before we find out how to use these things correctly. I do not
know how to use them correctly. I am quite sure that there are profitable uses but not
very soon.
Question 18
Looking at the future, the next decades, do you think we will arrive at a hybrid between
man and machine?
Answer
Well, we already have. Machines do not work without people, so that in a sense we already
have hybrid systems. A modern airplane is a hybrid control system with the man controlling
and the auto-pilot controlling. The fastest way to get machines that can adapt and learn
is to make a hybrid of an ordinary computer with its program and some pieces of the system
which are neural nets which are little by little adapting and changing. For example,
suppose I have a machine which is going to read and it wants to read handwriting. I will
use neural nets to learn the right way to read your handwriting, my handwriting by
adapting what it does, how it analyses the strokes of the pen. It will then tune those to
make sense in response to the interpretation from the program. So I think that the future
will have to start by our putting together hybrids of neural nets and ordinary computers,
and we as engineers will have to find out how they communicate properly, how they control
each other back and forth to do what we need. That is the way to make a system that is not
only flexible and responsive to us but can learn what we want and also learn the best way
to do that.
Question 19
What is your vision of the future? Are there other important goals to reach?
Answer
There are scientific goals. One of the real problems is not just to put them together but
to form layers of control. If you have an army fighting wars, at the bottom the soldier
pulls the gun, fires the guns, at the top the general worries about which countries to
invade, whether to deal with the politicians, whether to surrender and so on. And in
between every layer does something different, some control battalions or platoons. What we
need to do is to find out how to make the layers of control; at each layer there is
learning and changing, interpretations and modifications so that very rarely in an army
does the general tell the soldier when to fire the gun. The general says to the colonels:
Do such and such. And the colonel says to the majors: Try to do this and try to do that.
And it works. We do not know how to put together such structures in machines. This is the
research task which lies ahead and as we begin to play with the hybrid systems you
mentioned we have to learn how to put them together in structures of organization. These
are not small structures. If we want to have intelligence like people, these structures
have to be quite large I believe.
Question 20
At MIT you work with Professor Marvin Minsky. In his book "The Society of Mind",
Minsky described something like this.
Answer
The concept of separate agents with their own purposes but still, as you said, cooperating
collectively, that is right. Minsky is an absolute scientific giant in the field. He and I
were very interested and helped to start the field over 40 years ago.
|
