Re: [Phys-L] Bayesian statisitics

[brian whatcott <betwys1@sbcglobal.net>]

On 9/2/2013 8:54 PM, John Denker wrote:
> On 09/02/2013 12:07 PM, Brian Blais wrote:
>
> > ... wondering what you think of E. T. Jaynes' approach to Bayesian
> > inference.  He does not make use of set-theoretic definitions, but
> > in my reading of him, he does seem to admit that these have
> > identical consequences in applications.
> >
> > 1) do you agree?
> In general I don't like terms like "Bayesian" or "Darwinian".
> /snip/
> > 2) do you find some *quantitative* improvement using the
> > set-theoretic definitions.  I mean, is there an actual problem where
> > one method works and the other not.
> > 3) is there some *practical* improvement using the set-theoretic
> > definitions.  I mean, are there problems that are much easier to
> > solve, even if both methods yield the same result in the end?
> /snip/
> The development guys had a huuuge software system that was doing OCR with
> a 2% error rate, which was the same as people could do on the same data
> set, so this was considered quite an achievement.
>
> /snip/ ... one symptom was expressed by John
> von Neumann, who was not the village idiot:  "With four adjustable
> parameters I can fit an elephant, and with five I can wiggle his tail."
>
> Then word got out that my buddies and I were fitting 100,000 adjustable
> parameters, with good results.
[In case any one of the seriously humor-challenged missed it - this was 
a wonderful joke!]

/snip/
> Then word got out that I had a scheme to learn maximum_a_posteriori (MAP)
> not maximum likelihood.  This is P(a|b) instead of P(b|a).  The statistics
> research guys did not believe this was possible.  The development guys were
> skeptical, but after much inveigling and cajoling they tried my idea, and
> the error rate went down from 2% to 0.2%.
/snip/

An amusing and dramatic story of the perils  of sharp edged research!

I think the thrust was this:
Sometimes, it's not enough to judge the next item on the basis of the 
accumulated prior knowledge.
In some cases it can be helpful to suspend judgment until the spectrum 
of a posteriori outcomes is
made manifest in a particular data packet, when the numerous prior 
predictions of the next datum
 can be largely eliminated -  at least reduced by a factor of ten in 
some cases.
This has a curious parallel with the common statistical processing 
method for noisy data sets: the
Kalman(-Bucy)  filter (1958)  which is labeled Bayesian.

That would lead to the following answers to Brian's questions:

E. T. Jaynes' approach to Bayesian
inference... does not make use of set-theoretic definitions, but
... he does seem to admit that these have
identical consequences in applications,  [to using Set-Theory].

1) do you agree?

No

2) ...using the
set-theoretic definitions...is there an actual problem where
...[this]  method works and the other not.

Yes.

3) is there some *practical* improvement using the set-theoretic
definitions... are there problems that are much easier to
solve, even if both methods yield the same result in the end?

Undecided here.


Thanks for the interesting thread

Brian Whatcott    Altus OK