Re: [Phys-L] uncertainty on the uncertainty

[Peter Schoch <pschoch@fandm.edu>]

Hello,

Our Chemistry instructor and I have had this discussion for longer than I
care to think about.

I teach at a community college, and I'd love to say there's a REALLY good
reason I make the students do a standard error calculation in lab.  But...
 the best I've got is this:

1.  I want/need them (see #4) to understand that no set of measurements is
'perfect' and that errors PROPAGATE when used to determine other values.
 (Yes, I make them do partial derivatives, etc. and calculate the
propagated error as in Taylor's text.)  No, I don't think this is overly
onerous for a freshman class.  Yes, I know that the analysis being applied
is not the best to use.  But, this is the one they'll see in our stats
class, and...

2.  They need to know how to use a spreadsheet for 'quickie' analysis of
things - avg., std. dev., std. error, least squares regression. (again, see
#4)

3.  I do expect them to then compare their average +/- the standard error
to an accepted value (if possible) or to analyze the results; i.e., what if
the error is artificially low?  What could have caused that?  Is the sample
size unreasonably small?  This is the point of writing a 'lab report' and
their grade suffers accordingly if they just crunch numbers but don't
analyze them.

***4.  Our transfer institutions EXPECT us to introduce this concept of
error, propagation of error, and computer use of error calculation.  I have
numerous emails of thanks from our senior institutions stating how pleased
they are that our students are among the few who can
mechanically/mathematically do these calculations and rudimentarily
understand them.  They, then, take the next steps to introduce the more
robust (better) statistics for small sample sizes, etc.

Is this the best system?  Absolutely not.  Can I afford to take time out of
class/lab to teach them other stats methods -- unfortunately no.  The
laundry list of things that I have to cover in 2 semesters to have the
courses considered "transferable" by the senior institutions is extensive
and non-negotiable.  Can I leave the stats out -- not and have my students
fully prepared for transfer.

In this case reform needs to come from the top and perk down.  Or, at the
least, a robust discussion between the sciences and the stats Faculty to
try and alter/add to the stats class. (Unfortunately, though, they are
under the same constraints over transfer that we are.)

I think this is another topic that just has huge inertia, and will take a
great deal of effort and time to change.

Peter Schoch



On Wed, Jul 5, 2017 at 11:20 PM, John Denker via Phys-l <
phys-l@mail.phys-l.org> wrote:

> Hi Folks --
>
> Over the years, I have spent an unreasonable amount of time looking
> into the issue of uncertainty on the uncertainty, especially for
> small sample sizes.
>
> Suppose there is a sample consisting of N readings, drawn from
> some parent population, and the task is to compute the mean and
> standard deviation.
>
> In particular, consider the factor of (N-1) that appears in the
> denominator in the formula for the standard deviation.  That shows
> up in lots of places, including e.g. Baird equation 2.9, where it
> is touted as the "best estimate".  It is baked into the stdev()
> function in every spreadsheet I've ever seen.
>
> The question arises, where did that come from????!?!?!!!!!?
>
> Contrary to what is nearly-universally claimed, (N-1) does not
> produce an unbiased or optimal estimator for the standard
> deviation of the parent population.  It turns out that (N-1.5)
> is vastly better across the whole range, and even that's not
> perfect for small N.
>
> On top of all that, there's the fact that even if it were unbiased,
> it would still be an insanely /noisy/ estimator (for small N).
> There's a huge uncertainty on the uncertainty.
>
> There shouldn't be any big mystery about this.  The properties
> of the chi distribution have been understood for quite a while
> now.
>
> I take this as yet more evidence that most of the people who
> insist that it's super-important to calculate a number for
> the uncertainty never actually use it for any authentic
> purpose, and never check it against experimental reality
> (or even mathematical reality).
>
> I am reminded of the proverb of which the contrapositive
> says:  If it's not worth doing right, it's not worth doing.
>
> Why do people pretend this is important, in introductory lab
> situations where it clearly isn't?  Why not just skip it most
> of the time, and save it for the rare occasions where you
> have a big-enough N to make it worthwhile?
> _______________________________________________
> Forum for Physics Educators
> Phys-l@mail.phys-l.org
> http://www.phys-l.org/mailman/listinfo/phys-l