Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] weighted curve fitting



On 09/18/2017 09:03 AM, David Bowman wrote:

But if one properly propagates the error bands through the
transformation and then actually uses the proper weights that the
transformed error bars demand then the problem Brian mentions would
not occur. Properly used, the time-honored method stands the test of
time.

Amen, brother!

Let me expand on that a bit: In the physics business, virtually
every fit should be a weighted fit. If you find data points
that happen to be equally weighted, usually that means somebody
devoted a lot of engineering effort to make it so ... or it's
your lucky day, and you ought to go buy a whole bunch of
lottery tickets.

If you find a curve-fitting routine that does not provide for
weights on the input points, throw it away and get one that
does. Do not get into the habit of unweighted fitting.

Once upon a time there was a grad student to kept on doing
unweighted fits, even after being warned repeatedly. He
came to a bunch of wrong conclusions, leading to more than
a year of wasted effort. This is NOT a way to win friends
and influence people.

There is an inherent dilemma when /presenting/ data, because
to a first approximation people like to see data that falls
along a straight line /and/ (other things being equal) they
like to see equally-weighted data.

There are a couple of ways to alleviate (if not cure) this
dilemma. For one thing, it helps a lot to show the error
bands. People can see at a glance whether the points fall
within the error bands or not.

Also, you can fit things in one space and display them
in another. This is particularly useful if the fitting
representation has symmetrical error bands and the display
representation has lopsided error bands.

Another thing that often helps is to show the general
trend of the data one one scale and then show the
residuals on another scale, suitably zoomed in. This
is a big help when the residuals are small.

============

Note that in the vast majority of cases, you want error
bands that attach to the model ... *not* error bars that
attach to the points. Here's an example:
http://cms.web.cern.ch/sites/cms.web.cern.ch/files/styles/large/public/field/image/ltc_cls_comb_logx.png

This is reason #137 why the sig figs ideas are so poisonous:
they require you to attach a notion of error bars to every
data point (and everything else), which is almost never
correct.