Re: Finding information on the Internet

["John S. Denker" <jsd@MONMOUTH.COM>]

On 04/07/2003 06:18 PM, Tim Folkerts wrote:
> ... there are basically two methods to judge the accuracy of information.
> * Method 1)  check it yourself, from scratch
> * Method 2)  develop some rules of thumb to correlate the accuracy to other
> factors.

Let's not commit what I call the black/white
fallacy:  arguing that because something is not
completely white it must be completely black.

Typically one doesn't re-do everything "from
scratch", but one needn't jump to the opposite
extreme and judge books solely by their covers.

> Since we have progressed way too far for any one person to check any
> significant amount of info at any fundamental level, we rely almost
> entirely on Method 2.

I disagree.  Strongly.  The "other factors" in
question are somewhere between non-scientific
and anti-scientific.

http://campusgw.library.cornell.edu/t/help/res_strategy/evaluating/evaluate.html

> When reading an article that has been *published* in Phys Rev, I place a
> great deal of trust in the accuracy for two reasons.  I) I have personally
> seen a great deal of accuracy in the articles that I have look at closely.
> I may just have gotten lucky and most of the articles are way off base, but
> the odds are way against it.  II) I know that two someones like John D have
> already reviewed it carefully before it was accepted.

That's not very relevant.
Look again at the Subject: of this thread.
The reviewing process and imprimatur provided
by Phys Rev is mostly lacking from the internet.
(It is also lacking from most books in the
library.  It is also lacking from most news
media.  So usually just "trusting the editorial
authority" is a bad strategy.  Other skills are
necessary.)

> For an article that has only been *submitted* to Phys Rev. then I can't use
> either of these criteria, so I find other criteria.

OK, this returns to relevance.  Evaluating raw submitted
manuscripts is analogous to evaluating web sites.

> If I can verify their
> starting point in published articles and I can reproduce their calculations
> and/or experiments, then I trust the accuracy.

Or you might find published experiment B which
tends to confirm experiment A.  You don't need
to do everything "from scratch".

> So, then the question becomes, what secondary characteristics have a
> significant correlation with accuracy?  Here I think the guidelines are
> right on!

If you are going to rely on "secondary characteristics"
you might as well hang a sign around your neck saying
"I am not a scientist".


> > Under Authority it says:
> > *  Is the author identifiable? Look for links that say "Who We
> > Are," "About This Site, " or something similar.
> > * Is there contact information for the author? (e.g. e-mail
> > address, mailing address or phone number)
> > * What is the author's background? (e.g. experience, credentials,
> > occupation, have they written other publications on the topic?)
> > * Does the author cite his or her sources?
> > * Is this site linked to often by other sites?
> > * Do links on this site lead to other reputable sites?
> > * Are there spelling errors or incorrect use of grammar?
> > * What domain does the site belong to? (e.g. edu, gov, com, etc.)?
>
>
> Sure, no one of these will guarantee accuracy.  Heck, all of them together
> won't guarantee accuracy.

I agree with that!

> Nothing will guarantee accuracy.

To be clear:  none of the aforementioned non-scientific
methods will guarantee accuracy.

However, scientists do have a process that leads to
a reasonable guarantee of accuracy, namely having
people independently reproduce the result.

> But taken as a whole, I bet [secondary characteristics]
> are very strongly correlated to accuracy.

What's the evidence of that?

And (!) what is the price you're paying in the form
of false negatives, i.e. wise voices who are
unheard because they don't have a fancy return
address or fancy credentials?

http://www.monmouth.com/~jsd/physics/authority.htm

> That aside, judging "sense" requires a preexisting understanding.  It is a
> very high level thinking task.  The fact that you are searching for such
> info suggests you aren't already an expert.  Then you have to judge on
> other factors.

This is another instance of the black/white fallacy.

Suppose you are not the world's #1 expert in the field.
Fine.  You are still allowed to think!  You are still
allowed to use judgement!  You are still allowed to
judge a book by its contents, not its cover!

>     "The only thing necessary for the triumph of evil is for good men to do
>     nothing." - Edmund Burke

I agree.

As a corollary:  when good men start judging books
by their covers, it is almost as bad as doing nothing.