From: "JACK L. URETSKY (C)1998; HEP DIVISION, ARGONNE NATIONAL LAB ARGONNE, IL 60439" <JLU@HEP.ANL.GOV>
Date: Sun, 18 Jul 1999 07:44:17 -0500
Hi all-
John Denker raises some interesting points that remind me
of a problem that I first encountered many years ago in a physics
context (See Taylor, Moravcik, and Uretsky, Phys Rev 113 (1959) 689).
The problem then was to fit a coupling constant by extrapolating
physical data to imaginary values of a certain angle. We used the
minimum of chi-squared to evaluate the goodness of fit. The chi-squared minimum
measures the fewest number of parameters that give a "good" fit to the experimental
data. More parameters will not increase, and may decrease, the "goodness" of
the fit. The question raised by John's posting is, why is this a good way
to proceed?
John writes, at one point:
************************************************************************
In the pattern recognition field, there is a formalism that seems to work
pretty well: You imagine a whole ensemble of sequences, and you throw them
at a some "learning machines" and see which machine most often predicts the
end of the sequence (the "test data") given the beginning (the "training
data"). I'm leaving out a lot of details here for obvious reasons.
**********************************
OK, pattern recognition, as I understand it, works in situations
where there is an existing "culture", as I think John M. pointed out in his
response. That is, there is a fixed , predetermined set of choices from
which to choose; one seeks the most economical way of making the choices
subject to some minimum acceptable error rate. Too many parameters and
one is no longer "predicting" but merely reproducing data. I further
suspect that the data usually includes noise as a confounding factor.
In the physics case, the fact that one could fit experimental
data with many fewer parameters than data points meant that we were in
fact predicting data points from other data points. Theory gave us a
handle on the behavior of our fitted function as we extrapolated outside
the region covered by experimental data. Chi-squared provided a (somewhat)
objective measure of our confidence in the fit.
The bottom line, I think we all agree, is that identifying short
sequences is a highly culture-specific activity, a point that is lost
on many MF's (modern faculty members).
Regards,
Jack
"I scored the next great triumph for science myself,
to wit, how the milk gets into the cow. Both of us
had marveled over that mystery a long time. We had
followed the cows around for years - that is, in the
daytime - but had never caught them drinking fluid of
that color."
Mark Twain, Extract from Eve's
Autobiography