Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] science +- politics +- religious denominations (plural)

In June of 2012, as a fellow in DC, I had the great pleasure of meeting with the directors of research at the Pew Center. The group was small and statistics savvy, about 12 of us, and they carefully explained what they did to try to undo the biases that RT is talking about. 

In short, it is hard. The dirty secret of the industry is that most people won't talk to pollsters and much of the rest of the population is hard to find.

Pew works on finding those people and they believe they are better at that than the other firms.

First, they want to protect their image as being non-partisan by not asking leading questions and by hiding in some ways even the subject of their poll by adding additional questions. Second, they do polls just to discover who answers polls. As a side note, they are allowed to call people on Do Not Call Lists as they aren't selling anything.  As RT figures, people with and without cellphones are different sorts of people. More on that later. Pew tracks them down in a variety of ways, but the key one is that Pew will call back at the pollee's convenience and Pew pays them with gift cards or credits to their account. It makes polling more expensive but seems to make it more accurate as well. 

Pew handles the problem from another angle.  They use some of the above additional questions to cross reference to make sure that they are getting the full scope of the population. For example, even on a poll on say teaching they might ask if people watch motor sports. They have reason to know from other polling how much of the population watches motor sports and if they don't find that in their sample now, there might be a problem with their sample.

They also over sample parts of the population that are small so that the randomness of answers doesn't blow up the data for a subset group. Every book on stats says you have to do this, but most polling groups don't do it since it is very expensive.

Finally, Pew is endowed and independent. They don't take money from any group and do the polling that interests them. They don't have to generate press to drum up business. 

The consequence of this is that Pew surveys seem to be better than most, demonstrated out by national and local elections.

This was made clear last June. At the time Pew was explaining to us how different their polling was compared to other groups, especially Gallop which they heavily criticized for its failure to handle cellphones and younger parts of the population. They emphasized at the time that Gallop and others were going to get the election wrong if they didn't fix their problems. Well, they didn't fix their problems and you could see the results the day after the election.

Marc "Zeke" Kossover
Albert Einstein Distinguished Educator Fellow 2011-2012
National Science Foundation



________________________________
From: Richard Tarara <rtarara@saintmarys.edu>
To: Phys-L@Phys-L.org
Sent: Saturday, June 8, 2013 10:42 AM
Subject: Re: [Phys-L] science +- politics +- religious denominations (plural)


Back to the Gallop poll that has been a major focus over the past week.

I've discussed this with a number of people, including one who is very
savvy in the media (owned our city newspaper, is CEO of a consortium of
radio stations, and teaches at the Cronkite School of Journalism as ASU)
and no one gives any credence to the figures. Maybe Texas is like this,
but here in Indiana I don't know a soul (liberal, conservative,
Republican, Democrat, or Independent who could be included in the 58,
41, 39% stated in the survey--(sure there are some, put my personal
'statistics' don't support the numbers).

All of which brings to my mind the question of how one does a really
random survey today?  Most people with half a brain are on NO-CALL lists
and refuse any phone surveys that 'illegally' make it through.  We see
every political season (is there an 0ff-season?) that polls are not
necessarily that accurate.  EXACTLY how questions are framed can clearly
sway responses.  I've started and then discontinued a number of surveys
(often ones requested by my school) when the sequence and/or tone of the
questions were clearly trying to get a certain response (or to prove
some psycho-babble point or other).

To really believe the poll in question, we have to conclude we are a
nation of illiterate idiots--and I guess I'm not quite ready to buy into
that.  (We are not the sharpest tacks in the world, but.....)

A last thought here---religious belief, while not necessarily aligned
with scientific illiteracy, sets us up to disbelieve what we want.  If
belief in 'fairy tales' is OK, to be respected (at least tolerated),
then disbelief in almost anything else is easy to fall into.  As
scientists we all decry the fact that so many dismiss the science, but
we 'go along' with or even share in the religious beliefs.  Not sure we
can really have it both ways!

rwt


--
Richard Tarara
Professor of Physics
Saint Mary's College

free Physics educational software
www.saintmarys.edu/~rtarara/software.html

_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l