Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] uncertainty on the uncertainty



On 07/05/2017 08:20 PM, I wrote:

consider the factor of (N-1) that appears in the
denominator in the formula for the standard deviation. That shows
up in lots of places, including e.g. Baird equation 2.9, where it
is touted as the "best estimate". It is baked into the stdev()
function in every spreadsheet I've ever seen.

The question arises, where did that come from????!?!?!!!!!?

So far, nobody has accepted that challenge.

Just now I put up a new section that spills the beans:
https://www.av8n.com/physics/stdev-estimate.htm#sec-chi-squared

The basic answer is this:

Even though the factor of √(N−1) is wrong for estimating the standard
deviation, the factor of (N−1) is correct for estimating the variance.
Let's be clear: The best estimate for b^2 is not the square of the
best estimate for b. Taking the average does not commute with taking
the square root. Life can be quite nonlinear sometimes.

We can quantify this by using the chi-squared distribution. It is
famous, and textbooks give it far more attention than the chi distribution
that we discussed previously. Sometimes it’s what you want – but sometimes
it isn’t. It has some peculiar properties.

Tangentially related: Keep in mind that the central limit theorem is
not magic, and it does not solve all the world’s problems. Averaging
a few points is not guaranteed to produce something close to Gaussian.

For the rest of the discussion, including a bunch of diaspograms
showing how very hard it is to estimate the variance from a few
observations, see:
https://www.av8n.com/physics/stdev-estimate.htm#sec-chi-squared