Chronology | Current Month | Current Thread | Current Date |
[Year List] [Month List (current year)] | [Date Index] [Thread Index] | [Thread Prev] [Thread Next] | [Date Prev] [Date Next] |
It seems to me that the explanation needs to be more subtle than just a systematic error, or some degree of inadvertent or unconscious observer bias (I am assuming here that the effects the article talks about are not the ones that show up every so often that are outcome-driven--those are usually pretty quickly discovered and the penalties for that tend to be severe enough that they don't happen very often). But the article talks about effects that are good at the beginning and then get worse with time, I don't recall that any examples were given that went the other way. Nor were any cited in which the changes in the results were variable--some better, some worse. What we are looking at it a systematic decline in the measured effectiveness of new drugs as their time on the market passes.
An interesting question might be whether the phenomena have actually
shifted?
I don't think that most scientists would buy into this, but sometimes it may
actually happen. So in biology a drug which once worked can become
ineffective due to resistance. But another distinct possibility is the
problem of systematic error.
The very first measurement of any phenomenon might have some systematic
error which was not discovered. To publish another value one may have to do
a better job, so subsequent measurements may have lower systematic errors.
Sometimes researchers might be accused of falsifying data when in reality
there were factors that were different from subsequent research. In biology
it is often very difficult to control all of the factors, but the same thing
can happen in physics. We did some very accurate Neutron cross sections
which disagreed with Karlsruhe. It seemed that they used a multi shot TAC
which required some very complicated corrections. But the corrections were
wrong which filled in the valleys of the cross sections. Their data was
considered to be definitive and published in "The Barn Book", but they tried
to get high statistics at the expense of accuracy. They even admitted they
had problems when we talked to them personally, but it wasn't in writing.
But science works by checking other groups measurements. So eventually you
hopefully zero in on more accurate results.
There certainly are some measurements which may have been tweaked too much.
An example is the Hawthorne effect. The original data was lost, and I
understand that subsequently others have not really been able to replicate
the data. But the original paper is so compelling that it is quoted as
being true. The effect may not actually exist, or be very weak. For those
who are not familiar with it, it is a psychological placebo effect. When
you pay attention to workers they produce more output.