Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] Bayesian Inference in Half-Life measurement

On 9/22/21 8:47 PM, Paul Nord wrote:

You'd really enjoy David MacKay's lecture 10 on Information theory. He
really rips into physicists regarding "old school" methods. He's a
physicist, actually. So he should know.

Yeah. I've actually discussed these topics with David MacKay.
He definitely knows what he's talking about.

As for the video, I just now got around to watching it. IMHO
the first 25½ minutes is the least interesting part. I'm not
interested in hearing about all the "old school" methods that
don't work.

The rest of the video is great.
https://youtu.be/mDVE0M-xQlc?t=1537

===============

Tangential point: Beware that the word "Bayesian" means different
things to different people. There are some absolute kooks who
call themselves "Bayesians" as if that automatically proved the
correctness of whatever they feel like saying.

The correct approach is just math. As MacKay points out, the math
was used to good effect by Laplace, years before Rev. Bayes came
along. You can call it Bayesian or not.

I don't call myself a Bayesian, for the same reason that I don't
call myself a Laplacian or a Newtonian.

This terminological snafu is one of the reasons why it is hard to
google for information on the topic. Sometimes the right answer
will be called Bayesian, and sometimes not.

================

As MacKay points out, we start with a probability in the form of
a product of factors. We can take the logarithm, which gives us a
sum of terms, with exactly the same meaning. It's a sum of log
probabilities, or log improbabilities.

Note that the weighted sum of log improbabilities looks an
awful lot like a cross entropy. It also looks a lot like a
Kullback-Leibler information divergence. So you sometimes see
those terms appearing in articles on this subject.

The weighting factors in the aforementioned weighted sum play
the same role as Bayesian priors. So once again, it's just math.
You can call it Bayesian or not. The math doesn't care.