Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

[Phys-L] re Analysis of Half-Life measurement using PyStan




-
- Paul NordI was hoping for some feedback on this analysis. Specifically, what did you think of my conclusion: "All of these models generate curves which are very close to the data. While the errors seem very large, they are actually a better representation of the true uncertainty in applying this model to this data. Many least-squares fitting functions will give uncertainties which give too much confidence in the model predictions." Paul On Fri, Oct 8, 2021 at 3:07 PMTue, Oct 12 at 1:35 PM
-

-  Paul Nord <Paul.Nord@valpo.edu
- > wrote: 
- > I did a thing:
-  >  <https://sites.google.com/valpo.edu/double-exponential-decay/
- > Surprisingly, the resulting uncertainties are bad while the fit looks
-  > quite good. I've read that this sort of analysis gives a better estimation 
- > of the true uncertainty than one often gets with least squares fitting.
-  > Plotting a selection of 1000 models generated from this analysis shows that 
- > they all lie very close to the mean. The assumptions of this model do not 
- > constrain the model parameters given the particular data collected. 
- > > Said again: any of the terms in this model can be adjusted about 10%. You 
- > can still get a similar and reasonable fit. You just need the right
-  > tweaking of the other parameters. 
- > > Bad model assumptions here almost certainly include:
-  > A) The background rate is constant 
- > B) The stability of the detector > > Paul 
-

- > __________ Forum for Physics Educators Phys-l@mail.phys-l.org https://www.phys-l.org/mailman/listinfo/phys-l
-

-

- John Denker via Phys-l <phys-l@mail.phys-l.org>UnsubscribeTo:phys-l@phys-l.orgCc:John DenkerTue, Oct 12 at 2:41 PMOn 10/12/21 11:34 AM, Paul Nord wrote:

I was hoping for some feedback on this analysis.  Specifically, what did
you think of my conclusion:
"All of these models generate curves which are very close to the data.
While the errors seem very large, they are actually a better representation
of the true uncertainty in applying this model to this data.  Many
least-squares fitting functions will give uncertainties which give too much
confidence in the model predictions."

I have been following this with interest.

Here's why this is important: AFAICT there are very few examples of
assignments where students are expected to measure the uncertainty
of the actual data set

In contrast, there are eleventy squillion assigments where they
are required to calculate a predicted uncertainty, but then don't
check it against experiment, which is IMHO insane.

So my zeroth-order feedback is: You're on the right track.
AFAICT you are making solid progress in an important direction.
I'll help if I can.

=============

At the next level of detail, I don't know enough to say anything
definite about the conclusion quoted above. However I will say:

-- As a rule of thumb, it's true that:
  a) most least-squares routines are trash, even when applied to
    Gaussians.
  b) applying least squares to Poisson data is begging for trouble.
  c) when there are 5 fitting parameters, it's likely that there are
    correlations, whereupon the whole notion of "error bars" becomes
    problematic (to put it politely).

-- If you post the raw data somewhere (google docs or whatever) I might
  find time to take a whack at it with my tools. No promises.

  I assume each row in the data set is of the form
    bin start time, bin end time, counts in the bin

  or something like that. Time-stamped events would be better, but I
  can cope with binned data if necessary.
********************************************************************************************************
I expect John was looking for the data set which you provided at 
https://github.com/paulnord/bayesian_analysis/blob/main/my_data.csv

and which to the jaundiced eye seems to be gardened at two time stamps: 2.5 and 3 minutes.


|                        t |                       count |
| | 0.5 | 87 |
| | 1 | 164 |
| | 1.5 | 259 |
| | 2 | 331 |
| | 3.5 | 494 |
| | 4 | 551 |
| | 4.5 | 604 |
| | 5 | 659 |
| | 5.5 | 714 |
| | 6 | 766 |
| | 6.5 | 825 |
| | 7 | 875 |
| | 7.5 | 924 |
| | 8 | 971 |
| | 8.5 | 1007 |
| | 9 | 1054 |
| | 9.5 | 1100 |
| | 10 | 1143 |
| | 10.5 | 1179 |
| | 11 | 1221 |
| | 11.5 | 1260 |
| | 12 | 1305 |
| | 12.5 | 1346 |
| | 13 | 1388 |
| | 13.5 | 1418 |
| | 14 | 1456 |
| | 14.5 | 1490 |
| | 192.3833333 | 11184 |
| | 196.8333333 | 11407 |
| | 307.15 | 16679 |
| | 312.1833333 | 16919 |
| | 916.6666667 | 40583 |
| | 918.6666667 | 40660 |
| | 1086.633333 | 46063 |
| | 1089.583333 | 46138 |
| | 1271.666667 | 51484 |
| | 1273.666667 | 51538 |
| | 1446.666667 | 56114 |
| | 1448.666667 | 56166 |
| | 1629.1 | 60640 |
| | 1631.85 | 60697 |
| | 1747.716667 | 63394 |
| | 1748.716667 | 63414 |


I want to begin by congratulating you on finding a tool which offers the desired Bayesian probabilities for the parameters in question.
I spent some time in modeling your data with count = D0 (1 - exp (- t/tau) ) + BG*t and then with  count = D1(1 - exp( -t/tau1) + D2(1 - exp(-t/tau2) + BG*tusing a non linear regression app (NLREG)    with lightly constrained bounds for tau1 and tau2.    ( then converting to 1/2 life = ln(2)*tau  as a convenience)
 I found judicious bounds could elliminate failures to converge on an explanatory parameter set due to correlation of terms, and it was not difficult to arrive atparameters which could explain 99.99% of data variance but which were far from your values and far from the published values.This is of course the usual fate of curve fitting to exponentials. It was amusing to see that if I set the time for a zero count at -11 min for zero counts.my estimation of the longer mean life time looked more respectable    This is just gardening the data, of course!
I expect to expend more effort in processing your dataset. It was not clear if this set is synthetic, or in fact, the data from experimental observations.