Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] maximum likelihood versus minimum cost



John Denker on 5/10/12 at 1:13 PM made some good points:
The point is that there is a fourth requirement, namely that your fitting function must be an LCBF to begin with. Otherwise you don't have a linear estimator at all, let alone a "best" linear >estimator.
As you can see from the three examples given above, in physics we see lots of things that are LCBFs ... and also lots of things that aren't.

In the first year college physics lab course I'm involved with, one of the course objectives is to get students to recognize that sometimes simple mathematical transformations can make a non-linear equation take on a linear form. For example, we do an experiment involving measuring the end deflection (d) of an end-loaded cantilevered beam. Ideally this is given by:

d = k*L^n

Where k is a constant involving the beam's geometry and Young's modulus, L is the length of the free end of the beam, and n is an integer. In the lab, we want to determine the exponent n and show the students that taking logs of both sides yields:

log(d) = n*log(L) + log(k)

So, using simple linear least squares, we can get a value for both n and k, if we consider y to be log(d) and x to be log(L).

The object isn't so much to get "the best" values of n and k, but to demonstrate the value of strategic math transformations.

Similarly, in a thin optics lab, the students are asked to determine the focal length (f) from measurements of object (p) and image (q) distances using the thin lens equation:

1/p + 1/q = 1/f

One such suitable transformation is:

pq = f(p + q)

So letting y = pq and x = p + q again yields a linear equation for f. And, again, the object is more to demonstrate the value of thinking about a solution technique than getting the best possible answer.

I suppose some of the emphasis on "thinking" about a clever solution technique is not as helpful in this age of superfast computers. I went to college when slide rules were the norm for your homework problems and punched cards were the norm for your graduate school thesis problems. Under such conditions, taking a lot of time to think of a clever solution technique usually saved a lot more time doing non-clever calculations. Today this is not as true.

This is a true story. I once was working on an underwater magnetic detection system and wanted to plot the contours of constant magnetic potential from a magnetic dipole. I wanted a closed form solution with y on one side and x on the other side of the equation for a fixed potential value, so plotting the 2d contours would be "easy". Somehow, from looking at a typical plot in a textbook, I got the mistaken idea that they might be ellipses, so I tried for several hours to get the equation of an ellipse and only succeeded in proving that the contours were not ellipses. Meanwhile, the young college intern helping me had disappeared, but reappeared just as I was giving up on the ellipse idea. He produced a nice computer generated plot of several complete contours which looked just like the ones in the textbook (which was the main objective of the exercise). When I asked how he had done that, and explained that I had toiled for several hours and several pages of algebra without being able to come up with a closed form solution with x and y separated, he said: "Oh, nobody does it that way anymore!" He just took the potential equation with x and y mixed together, picked a value of potential and a value of x, and then tried all possible values of y until he found that y which yielded the correct potential value. He then incremented x and searched for the new y value satisfying the same potential, etc. I was taken aback. Such an approach would have been foolhardy when I was his age doing hand calculations with a slide rule. But nowadays the ability to do large numbers of calculations quickly and accurately can obviate the need for the clever solution techniques we used to so prize in my day.

Nonetheless, I think that some pre-thinking about solution techniques will always be useful even with the most powerful computers, and the simple math transformation in our beginning labs help reinforce this point.

As for John Denker's example A(t) = A(0)exp^(kt). In the spirit expressed above, I did linearize this as ln(A(t)) = kt + ln(A(0)) and solved for k and A(0) using linear least squares. But I also used the "exponential fit" routine in Excel (presumably a nonlinear routine). I tried this with and without gaussian zero mean errors in both the measurements (A(t)) and the independent variable (t) and got exactly the same results for k and A(0) from the least squares and exponential fit results. I'm not familiar enough with Excel to know what's going on with their "exponential fit" routine. I would have expected similar results for no or small errors in t, but was surprised that there still was no difference with significant t errors present (which violates the Gaussian-Markov theorem for least squares being the best linear estimator). I will say it was just as easy to push the "exponential fit" button as it was to push the linear fit button, while with the linear fit I had to do the extra step of computing exp^ln(A(0)) in order to get A(0) whereas the exponential fit routine gave k and A(0) directly.

Don

Dr. Donald Polvani
Adjunct Faculty, Physics
Anne Arundel Community College
Arnold, MD 21012