Re: [Phys-L] determine k

[John Denker <jsd@av8n.com>]

On 02/10/2015 05:00 AM, Paul Lulai wrote:
> How does the student insert the uncertainty value of the ith point if
> each point is raw data and has no uncertainty?

OK, that's a tricky concept.  The short answer 
is:  "The tolerance comes from the model".

Now let me explain that answer.  First, let me
describe the desired end state:

You will notice that in my spreadsheet I did *not*
put little error bars on the raw data points.
Instead I used the model to plot a best-fit line
(the black line) and a 1σ tolerance band (between
the red lines).  The model (including the tolerance
band) is plotted in the background, and then the
zero-sized pointlike data points are plotted in
the foreground.

I am not the only person to do things this way.
Here is a slide from the team that was hunting
the Higgs.  These guys know what they're doing:
  https://vixra.files.wordpress.com/2011/08/wwslideeps.jpg
They show the 1σ (green) and 2σ (yellow) bands.
The data points are pointlike.  We get to ask
whether the points are outside the band.

Here's another way of visualizing the concept.
  https://www.av8n.com/physics/probability-intro.htm#sec-converge-continuous
Putting error bars on the data points is just
wrong.  It will give you the wrong fitted
parameters.  I am quite aware that every data-
analysis text on earth tells you to do it the
wrong way.



Now the question arises, how do we achieve the
desired state.  The fitting procedure requires 
weights in order to do the fit, yet the fitted 
parameters are needed in order to calculate the
weights.  It's a chicken-and-egg problem.

The solution is to pick some initial estimate
for the weights, do an initial fit.  Start with
uniform weights if you must.  Or in the oscillator
case, start with a massless spring.  Or, better,
start with an effective mass that includes 1/3rd
of the mass of the spring.  Then use the resulting 
parameters to generate better weights.  Then iterate.

Note that nonlinear least-squares fitting is
always iterative.  Hunting for suitable weights
makes even linear least-squares fitting iterative.