Re: [Phys-l] case study: how to detect bogus data

[Brian Whatcott <betwys1@sbcglobal.net>]

John Denker wrote:
> I quote from the summary at
>   http://www.dailykos.com/storyonly/2010/6/29/15117/8738
>
>   
> > I have just published a report by three statistics wizards showing,
> > quite convincingly, that the weekly Research 2000 State of the Nation
> > poll we ran the past year and a half was likely bunk.
> >     
>
>  ... [a couple of pages] ...
>
> The longer report is at
>   http://www.dailykos.com/story/2010/6/29/880179/-Research-2000:-Problems-in-plain-sight
>
> =============
>
> I mention it in this forum because we all deal with data for
> a living.
>
> Scientific fraud is very rare, but it is also hard to catch.
> It is instructive to see what spurious data looks like, and
> to see the techniques used to detect it.
>
>   
Classifying a weekly political poll as an embodiment of scientific fraud
is stretching it in my view.  Reviewing its basis after the poll is 
rejected for
poor match to reality was disappointing. Give me more prospective warnings
of poll reliability by all means.    Still, seeing far too many odd/even 
matches
in matched pair tabulations represents a worthy observation which needs
no great statistical depth to confirm. And seeing too few zero deltas
brings to mind another point that John was recently mentioning -
reminding me of Sherlock Holmes and (was it?)
the  Hound of the Baskervilles which did not bark in the night.

The comparison to the noted science fraud concerning IQ
determinations in an English psychologist's opus
comes immediately to mind. THIS was unexpected, and affected many
childrens' lives.

Brian W