But this long history of learning how not to fool ourselves--of
having utter scientific integrity--is, I'm sorry to say, something
that we haven't specifically included in any particular course that
I know of. We just hope you've caught on by osmosis.
The first principle is that you must not fool yourself--and you are
the easiest person to fool. So you have to be very careful about
that. After you've not fooled yourself, it's easy not to fool other
scientists. You just have to be honest in a conventional way after
that.
I would like to add something that's not essential to the science,
but something I kind of believe, which is that you should not fool
the layman when you're talking as a scientist. I am not trying to
tell you what to do about cheating on your wife, or fooling your
girlfriend, or something like that, when you're not trying to be
a scientist, but just trying to be an ordinary human being. We'll
leave those problems up to you and your rabbi. I'm talking about
a specific, extra type of integrity that is not lying, but bending
over backwards to show how you are maybe wrong, that you ought to
have when acting as a scientist. And this is our responsibility as
scientists, certainly to other scientists, and I think to laymen.
For example, I was a little surprised when I was talking to a
friend who was going to go on the radio. He does work on cosmology
and astronomy, and he wondered how he would explain what the
applications of this work were. "Well," I said, "there aren't any."
He said, "Yes, but then we won't get support for more research of
this kind." I think that's kind of dishonest. If you're
representing yourself as a scientist, then you should explain to
the layman what you're doing--and if they don't want to support you
under those circumstances, then that's their decision.
One example of the principle is this: If you've made up your mind
to test a theory, or you want to explain some idea, you should
always decide to publish it whichever way it comes out. If we only
publish results of a certain kind, we can make the argument look
good. We must publish both kinds of results.
I say that's also important in giving certain types of government
advice. Supposing a senator asked you for advice about whether
drilling a hole should be done in his state; and you decide it
would be better in some other state. If you don't publish such a
result, it seems to me you're not giving scientific advice. You're
being used. If your answer happens to come out in the direction the
government or the politicians like, they can use it as an argument
in their favor; if it comes out the other way, they don't publish
it at all. That's not giving scientific advice.
Other kinds of errors are more characteristic of poor science. When
I was at Cornell, I often talked to the people in the psychology
department. One of the students told me she wanted to do an
experiment that went something like this--it had been found by
others that under certain circumstances, X, rats did something, A.
She was curious as to whether, if she changed the circumstances to
Y, they would still do A. So her proposal was to do the experiment
under circumstances Y and see if they still did A.
I explained to her that it was necessary first to repeat in her
laboratory the experiment of the other person--to do it under
condition X to see if she could also get result A, and then change
to Y and see if A changed. Then she would know that the real
difference was the thing she thought she had under control.
She was very delighted with this new idea, and went to her
professor. And his reply was, no, you cannot do that, because the
experiment has already been done and you would be wasting time.
This was in about 1947 or so, and it seems to have been the general
policy then to not try to repeat psychological experiments, but
only to change the conditions and see what happens.
Nowadays there's a certain danger of the same thing happening, even
in the famous (?) field of physics. I was shocked to hear of an
experiment done at the big accelerator at the National Accelerator
Laboratory, where a person used deuterium. In order to compare his
heavy hydrogen results to what might happen with light hydrogen"
he had to use data from someone else's experiment on light
hydrogen, which was done on different apparatus. When asked why,
he said it was because he couldn't get time on the program (because
there's so little time and it's such expensive apparatus) to do the
experiment with light hydrogen on this apparatus because there
wouldn't be any new result. And so the men in charge of programs
at NAL are so anxious for new results, in order to get more money
to keep the thing going for public relations purposes, they are
destroying--possibly--the value of the experiments themselves,
which is the whole purpose of the thing. It is often hard for the
experimenters there to complete their work as their scientific
integrity demands.
All experiments in psychology are not of this type, however. For
example, there have been many experiments running rats through all
kinds of mazes, and so on--with little clear result. But in 1937
a man named Young did a very interesting one. He had a long
corridor with doors all along one side where the rats came in, and
doors along the other side where the food was. He wanted to see if
he could train the rats to go in at the third door down from
wherever he started them off. No. The rats went immediately to the
door where the food had been the time before.
The question was, how did the rats know, because the corridor was
so beautifully built and so uniform, that this was the same door
as before? Obviously there was something about the door that was
different from the other doors. So he painted the doors very
carefully, arranging the textures on the faces of the doors exactly
the same. Still the rats could tell. Then he thought maybe the rats
were smelling the food, so he used chemicals to change the smell
after each run. Still the rats could tell. Then he realized the
rats might be able to tell by seeing the lights and the arrangement
in the laboratory like any commonsense person. So he covered the
corridor, and still the rats could tell.
He finally found that they could tell by the way the floor sounded
when they ran over it. And he could only fix that by putting his
corridor in sand. So he covered one after another of all possible
clues and finally was able to fool the rats so that they had to
learn to go in the third door. If he relaxed any of his conditions,
the rats could tell.
Now, from a scientific standpoint, that is an A-number-one
experiment. That is the experiment that makes rat-running
experiments sensible, because it uncovers the clues that the rat
is really using--not what you think it's using. And that is the
experiment that tells exactly what conditions you have to use in
order to be careful and control everything in an experiment with
rat-running.
I looked into the subsequent history of this research. The next
experiment, and the one after that, never referred to Mr. Young.
They never used any of his criteria of putting the corridor on
sand, or being very careful. They just went right on running rats
in the same old way, and paid no attention to the great discoveries
of Mr. Young, and his papers are not referred to, because he didn't
discover anything about the rats. In fact, he discovered all the
things you have to do to discover something about rats. But not
paying attention to experiments like that is a characteristic of
cargo cult science.
Another example is the ESP experiments of Mr. Rhine, and other
people. As various people have made criticisms--and they themselves
have made criticisms of their own experiments--they improve the
techniques so that the effects are smaller, and smaller, and
smaller until they gradually disappear. All the parapsychologists
are looking for some experiment that can be repeated--that you can
do again and get the same effect--statistically, even. They run a
million rats no, it's people this time they do a lot of things and
get a certain statistical effect. Next time they try it they don't
get it any more. And now you find a man saying that it is an
irrelevant demand to expect a repeatable experiment. This is
science?
This man also speaks about a new institution, in a talk in which
he was resigning as Director of the Institute of Parapsychology.
And, in telling people what to do next, he says that one of the
things they have to do is be sure they only train students who have
shown their ability to get PSI results to an acceptable extent--
not to waste their time on those ambitious and interested students
who get only chance results. It is very dangerous to have such a
policy in teaching--to teach students only how to get certain
results, rather than how to do an experiment with scientific
integrity.
So I have just one wish for you--the good luck to be somewhere
where you are free to maintain the kind of integrity I have
described, and where you do not feel forced by a need to maintain
your position in the organization, or financial support, or so on,
to lose your integrity. May you have that freedom.
http://www.lhup.edu/~DSIMANEK/cargocul.htm