Chronology | Current Month | Current Thread | Current Date |
[Year List] [Month List (current year)] | [Date Index] [Thread Index] | [Thread Prev] [Thread Next] | [Date Prev] [Date Next] |
On Mon, 31 Dec 2001, John Clement wrote:
then an effectMost of the educational
literature effect sizes are less than 1.0 and a curriculumwhich achieves
anything over 0.5 is usually considered to be very effective.We're not concerned with "usually". We"re doing straight
mathematics. If the curve is taken to represent an estimate of a
probability distribution, and the distribution is normal,
variance of ansize of 1.0 may be interpreted as a 30% chance that the mean did not
change.
Absolutely not. The standard deviation being used is not the
individual students scores it is the variation over thepopulation. Each
student is in a different state. The 30% chance would be correct if onechange in score
compared the variation of a single students' scores with the
of that student. If one could give the same student the sametest multiple
times, one would come up with a much smaller variance than onesees for the
whole ensemble of students. There is no way of measuring thevariance for
the score of 1 individual student, and it will be differentfrom one student
to the next.
That is exactly the point. You not only do not know the
individual variances (meaning the variance of the distribution of this
individual's scores) you don't know the distribution. In your
spectroscopy line example, you know quite a bit about the
width distribution of the line's illumination just from observing it;
that example is therefore not analogous.
using just 1
Student scores are not clean data which are not easily compared
figure of merit. Student scores are determined by a large number ofcomparing physics
variables, so some of the simple strategies employed in
data are not appropriate. The situation in physics educationresearch is
more analogous to the early days of the physical sciences.
Not at all. The mathematics of statistical inference has also
come a long way since those days. There are standard tests for estimating
whether samples from unknown distributions come from the same
distribution ("the null hypothesis" in your case). The trouble with the
effect size measure is that there is a huge overlap of the two
distributions when the effect size is 1, but you know nothing about the
populations in the overlap region. As you have noted, when you look at
individual student score, pre- and post- test, some decrease and some
gain, but the effect size test gives you no clue as to where any increase
is coming from.
My "30%" comment was just intended to call attention to this
overlap.
the number of
looks like a
The initial curve (Lawson test) in JCST (figure 2) essentially
normal distribution. The final one is also similar, but movedover by about
1 SD. I am judging this by the curve. The result is that
dramatically reduced. Istudents who would be classified as concrete is
individual studenthave found for the Lawson test that when one looks at
the RH tail isscores they do not all move up, but rather each student moves adifferent
amount, with some making dramatic gains, and others none atall. The curve
in JCST unfortunately moves so far to the right that the righthand tail is
cut off by saturation on the test.
A test may be likened to a measurement in the lab. When the
needle pegs, the measurement is invalid.
But this is not a needle. One has the curve and only part of
missing. One can still deduce what the mean will be byanalyzing the curve.
Only if you have great faith that the curve was not influenced by
the fact that the test was one that gave a cut-off tail.
student test
As I recall the original discussion centered on the fact that
scores tended to fall dramatically after 2 weeks away from class. BTW Istudent scores on
looked up your response to the TPT article that shows that
the FCI do not seem to decrease with time after a reformedphysics course.
You correctly said that the study only showed that for thestudents tested,
the scores did not fall. While this is true as far as it goes, it is
unlikely that scores only stayed the same for the tested students.
I don't think that's quite what I said. As I recall, the thrust
of my remarks was that the paper seemed to be dealing with a very biased
sample.