Chronology Current Month Current Thread Current Date [Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

# [Phys-L] Fitting an Exponential to Noisy Data [topical, yet also quite general]

• From: John Denker <jsd@av8n.com>
• Date: Fri, 17 Apr 2020 21:08:12 -0700

Hi Folks --

New document:
https://www.av8n.com/physics/noisy-exp-fit.htm

When an exponential process is producing only a few events, it
requires a bit of skill to model the process. The appropriate
methods are worth learning, because they are quite general.
They could apply to radioactive decay, or to the growth of
yeast in bread or beer, or (!) to the spread of disease in
some local area.

Note that we are long past the stage where you can model the
coronavirus pandemic on a country-by-country basis, or even a
state-by-state basis. There are a squillion different outbreaks,
each busily exponentiating at its own rate.

This creates extra work for folks who are trying to make sense
of the data. In particular, when you look at a local area,
there are very few events to work with. Extracting reliable
estimates of the growth rate is not trivial. There will be
a lot of uncertainty in the extracted parameters.

One thing you cannot get away with in situations like this is
taking the logarithm of the ordinate and fitting a straight
line to it using linear regression. You can appreciated this
by looking at figure 1: Some of the data points are zero. If
you take the logarithm of that, you get minus infinity. You
cannot fit to such points using a straight line. And you
cannot afford to ignore these points. So skip the logarithm
and skip the linear regression, and use the industrial-strength
nonlinear fit.

If you have a greater number of events at each
x-value, you might try the trick of fitting a
straight line on semi-log paper, but even then
you risk introducing bias into the results.

Another thing you cannot get away with is an unweighted fit.

For the next level of detail, including a bunch of diagrams,
see:
https://www.av8n.com/physics/noisy-exp-fit.htm