Re: [Phys-L] Analysis of Half-Life measurement using PyStan

[John Denker <jsd@av8n.com>]

On 10/12/21 11:34 AM, Paul Nord wrote:

> I was hoping for some feedback on this analysis.  Specifically, what did
> you think of my conclusion:
> "All of these models generate curves which are very close to the data.
> While the errors seem very large, they are actually a better representation
> of the true uncertainty in applying this model to this data.  Many
> least-squares fitting functions will give uncertainties which give too much
> confidence in the model predictions."

I have been following this with interest.

Here's why this is important: AFAICT there are very few examples of
assignments where students are expected to measure the uncertainty
of the actual data set

In contrast, there are eleventy squillion assigments where they
are required to calculate a predicted uncertainty, but then don't
check it against experiment, which is IMHO insane.

So my zeroth-order feedback is: You're on the right track.
AFAICT you are making solid progress in an important direction.
I'll help if I can.

=============

At the next level of detail, I don't know enough to say anything
definite about the conclusion quoted above. However I will say:

 -- As a rule of thumb, it's true that:
   a) most least-squares routines are trash, even when applied to
    Gaussians.
   b) applying least squares to Poisson data is begging for trouble.
   c) when there are 5 fitting parameters, it's likely that there are
    correlations, whereupon the whole notion of "error bars" becomes
    problematic (to put it politely).

 -- If you post the raw data somewhere (google docs or whatever) I might
  find time to take a whack at it with my tools. No promises.

  I assume each row in the data set is of the form
	bin start time, bin end time, counts in the bin

  or something like that. Time-stamped events would be better, but I
  can cope with binned data if necessary.