19 July 2014

Peer Review



I remember a number of times hearing about “Room temperature superconductivity”, and then, some years later, in small print, a retraction. The retraction never got the headlines the original claim got, but, still, better science than non-science, right? This is something that was, to use the phrase of art, “FALSIFIABLE” , That is to say, it can be demonstrated to be reproducible (which is NOT the same thing as proven), or it can be demonstrated to be FALSE. You know the phrase “the exception proves the rule”?
In this case “proves” means to demonstrate the falsifiability of a hypothesis. In simple English, an exception demonstrates that the “rule” is fucked, and that further thought is needed. The word “Proof” is derived from the Latin for “Test”. (If you're interested, the word in Spanish is “prueba” and means “test”.)

Now let's move down a few doors from the hard disciplines like chemistry, physics, biology, math, geology, and engineering: where the unfriendly instructors expect you to get the math straight, to the lighter, more user friendly area of the model sciences, where the numbers dance to the tune of the designer.

In these rooms we find the economists (the “dismal science”, but it ain't, any more than boxing is the “sweet science”, and that's because neither one is a science), the political scientists, and the climate scientists. But they aren't, really, because SCIENCE has a method. Generate a hypothesis, conduct experiment to verify, make honest assessment, publish. These soft sciences do not have a lab in which to carry out any experiments, they cannot reproduce experiments, all they can do is formulate models to predict future trends. All the attempts at mathematical precision are so much hokum. The only thing that matters in this case is Predictive Value, and in this they uniformly fail.

Back in High School, I was taught that the average of averages was statistically useless. I didn't know then about Standard Deviations, Long Tails, and Strange Attractors. Hell, I didn't even know about the Oxford comma, except that I used it to make things clear.

But back to the topic. It comes as no big surprise to me to find that something on the order of 70% of “peer reviewed” papers in some journals are bieng retracted. Professorial courtesy kinda thing. The bulk of them, so far as I can figure, are in microbiology, which is not a topic that I follow closely, nd is one that requires a certain amout of discrection on the part of the person personing the microscope. (To say “maning” would be sexist). A person tasked with monitoring {x} will look for that. I call it the “green Impala” syndrome, from when in High School the most common mommy-vehicle was a , you guessed it green Impala. Must have been thousands of them. But that's what I was looking for. I'm sure there were more Chevy Novas and Ford Galexy500s. The thing is that you find what you're looking for.



No comments: