Re: [Phys-L] LSF Slope Versus Average of Single Point Slopes

[John Denker <jsd@av8n.com>]

On 05/04/2012 01:49 PM, Donald Polvani wrote:

> I thought that the LSF technique would produce the optimum result.  

Which it does.

> But the
> average point slope result seems to be just as good and much simpler to
> calculate.  

There must be more to the story, as can be seen from the following
ultra-simplified example.  Suppose there are only two points,
          X     Y
          0     0.0001
          1     1.0001

with an uncertainty of ± 0.1 on the Y-values.  If you fit a straight
line to that, with zero Y-intercept, you get a slope of very nearly
unity.  If you calculate the pointwise slopes and take the average,
weighting each point equally, you get a very much larger answer.
Infinitely larger.

> This result must be particular to the case where the y-intercept
> is zero 

That's an important part of the story.  The other part of the
story has to do with weighted averaging.

> nonetheless, I'm surprised and puzzled.

This is an interesting puzzle.  It takes some sophistication
to even notice that there's a puzzle here.

> This result must be particular to the case where the y-intercept
> is zero 

That's half of the answer, right there.

The other half is this:  Every sum is a weighted sum.  Every average
is a weighted average.  If you assume a particular weighting function
(be it equal weighting or otherwise) you do so at your peril.

Let's follow that road and see where it leads ... but before we 
do that, let's first take a step back and think about the overall 
structure of the problem.  Least squares fitting is a minimization.
As always, that means varying some parameter Q (possibly multi-
dimensional) and minimizing some objective function E(Q).  This
typically involves some sort of average or sum -- a weighted
average or weighted sum -- over the observed data.

In the case of a one-parameter slope-only model, Q *is* the slope,
θ.  You are calculating E as a function of θ and that's all there
is to it.

It is well known that one-parameter least-squares fitting is
tantamount to taking the average.  You can prove this with a
few lines of algebra.  So all there is left to discuss is what 
kind of /weighted/ average is appropriate.

Presumably we know the weights (i.e. the inverse error bars) on 
all the data points.  I am *not* assuming they are all the same.
However, for simplicity I will assume that the uncertainty in 
the X-direction is zero (which in real life is *not* always a 
safe assumption).

> From the point of view of least-squares fitting, points relatively
far from the origin contribute relatively more to the objective 
function (E), because a change in θ moves the fitted line a greater
amount along the error bar for that point.  In other words, each
point is effectively weighted by a factor that depends on how far 
from the origin it is.  The least-squares fitting routine knows
this implicitly, based on the given error bars on the observed
data ... and based on the form of the fitted function.

Now, let's switch gears.  Instead of least-squares fitting, we just
calculate the pointwise slopes.  Now the uncertainty in each slope 
is ΔY / X.  By rights, you should do a *weighted* least-squares 
fit on the slopes, with a weighting factor proportional to 
      W = 1 / (1 / X)
        = X

If (and only if) you apply this weighting factor, the weighted 
average will give you the right answer.  You can check that the
two-point example given above passes this test.

==========================

The following point is only tangential to the interesting part of
the story:

> This result must be particular to the case where the y-intercept
> is zero 
> (as otherwise the point slope approach, which assumes the line goes
> through the origin, isn't even a correct model of the situation);

Maybe, maybe not ... depending on what model is being used
*and* on what physics is being modeled.  A model that assumes
the Y-intercept is zero is appropriate _if and only if_ the
actual physics requires the Y-intercept to be zero.