I just finished a fascinating book by Richard Feynman, the Nobel Prize-winning physicist and overall interesting character. Entitled “Surely You’re Joking, Mr. Feynman!”, the book was published in 1985 but most of the chapters are reminiscences of Mr. Feynman’s from well before this. I like the book because Feynman likes to rant about some of the same things that I do, but in a much funnier (and smarter) way.
One of the things that drives Mr. Feynman nuts is that people don’t challenge their assumptions. Let’s take, for example, education.
In the scientific method, an experimenter designates an hypothesis and then tests to see whether the hypothesis holds up. Statistically speaking, the researcher designates a null hypothesis – i.e, he posits that nothing will happen, and then sets out to prove himself wrong. If an experiment yields a positive result the good scientist says “Ok, it worked that time in this set of circumstances. Let’s see what happens if I repeat it.” The question is asked and repeated over an over in every conceivable iteration until the scientist is grudgingly willing to admit that “maybe it works.” Then a bunch of other scientists repeat what he did to see if it works the same way for them, in their labs, with their equipment, in their environments. Then, if it still holds up, the scientist will construct a theory and see if the theory fits not only his own discovery but also explains something else and works in multiple applications.
After many years of sitting kids in a classroom, talking to them about some topic, and then testing them on the topic, somewhere, someone came up with the theory that sitting kids in a classroom, talking to them about some topic, and then testing them on the topic is the way to teach kids stuff. This person decided that if the kids did well on the test, they must have learned the material, so the method of teaching worked. Someone else decided that if the test scores don’t prove that this way of teaching works, it is not the theory that is wrong but some element of the application. So, maybe the people talking to the kids about topics are no good. In fact, let’s extrapolate that good test scores equate with good teachers, and since good teachers are those who can stand in front of kids and talk to them about some topic and get them to score well on a test, let’s require that all students get tested. Then we can decide if the teachers are good.
The problem here is that the theory only fits what has already been decided. It doesn’t make something else work out also, or explain a phenomenon that had been previously unexplained.
The final chapter of Mr. Feynman’s book is taken from a commencement address Feynman gave at Caltech in 1974, and its relevance has not diminished. Here he is talking about scientific inquiry. He has just given us an example of something everyone believed but hadn’t actually been satisfactorily proven, and he goes on to say:
So I found things that people believe, such as that we have some knowledge of how to educate. There are big schools of reading methods and mathematics methods, and so forth, but if you notice, you’ll see reading scores keep going down – or hardly going up – in spite of the fact that we continually use these same people to improve the methods. There’s a witch doctor remedy that doesn’t work. It ought to be looked into; how do they know that their method should work? Another example is how to treat criminals. We obviously have made no progress in decreasing the amount of crime by the method that we use.
There is something missing in [this pseudoscience]. It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty – a kind of leaning over backwards. For example, if you’re doing an experiment you should report everything you think might make it invalid – not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment and how they worked – to make sure the other fellow can tell they have been eliminated. Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you an to explain it. When you put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition. In summary, the idea is to try to give all the information to help others judge the value of your contribution; not just the information that leads to judgement in one particular direction or another.
The problem with education is that the assumption made at the very beginning, that good test scores equal learning, was perhaps false. The experimenter, to extend the analogy, didn’t consider ways in which it was possible to disprove this assumption. Everything follows from that lack of intellectual honesty.