| Chronology | Current Month | Current Thread | Current Date |
| [Year List] [Month List (current year)] | [Date Index] [Thread Index] | [Thread Prev] [Thread Next] | [Date Prev] [Date Next] |
The conclusion is that the difference between 10 and 7 is
highly significant. ... I am concerned with odds favoring
the null hypothesis. If somebody asked me about the error
bar to be placed around the observed difference of 3 then I
would say it is plus or minus SE=0.848, or 28% [not
16.2 % as I wrote yesterday]. This would be close to
Jack’s initial conclusion.
Hmmm....this thread is taking an unexpected turn.
Three people have come to greatly different conclusions about
two given samples.
Here are some notes which may help you sift the evidence.
Ludwik quotes Standard Statistics, Moore on
the usual expression for the standard error of the difference of the
means of two samples.
It seems to me that the expression he quoted,
SE=sqrt( s1^2/25 + S2^2/25)
compares well with the expression I used, namely
"standard error of the difference of the sample means is
sqrt[(3/5)^2 + (3/5)^2] "
I hope that Moore intended s1, s2 to represent the standard
deviations of the two samples.
I fancy Ludwik used the standard error of each sample mean instead.
Now to the objection voiced by Jack, whose basis is that the population
(of student results) from which the two samples are drawn is not normal,
and that the sigma (the standard deviation of the whole population)
is unknown.
I find that Jack's objection is probably justified, though it works in the
opposite direction to the one he suggests.
I come to this conclusion by supplying as much evidentiary support
for the purely statistical inference as I can, as follows.
It is not PC to mention it, but an important predictor of
academic test results is that old faithful, the IQ measure.
This measure is designed to be normally distributed.
A high school may expect to see student results in general that are
reasonably normal, even if test results are not 'marked on a curve'
[i.e. transformed to a normal distribution] for this reason alone.
I expect that the tails will be somewhat curtailed and the majority
of results compressed rather closer than the normal curve would predict,
from reading about the test marking guidance offered by school
administrations (as described on this list).
However, physics students are selected, one way or another to
represent something less than the entire IQ distribution - it's a
difficult topic in general.
Accordingly, you'd expect to see a similar distribution in physics
test results as in tests of intelligence in lawyers, electricians,
physicians, HS teachers: it's skewed with a cutoff at the low end.
(I fancy that's one reason that physics teachers get depressed by
test results - they tend to bunch at the lower end unless transformed).
I should also mention some straight-forward statistical issues brought
up by Jack and Ludwik:
If a population is not normally distributed, then it is [of course] probable
that smallish samples will not attain normality. That is one justification
for Gosset's "Student's t" distribution, which models the abnormality
of small samples. This seems to disqualify the Hogg and Craig cite,
if the assumption there is that the sample means are normally
distributed for a population of unknown distribution.
It is very often the case that the sigma and mu of the whole population
from which samples are taken is unknown. You could say that is the
whole point of statistical inference!
I used the t statistic and assumed normality of the underlying population
casually, in order not to exaggerate the value of the statistical measure
cited by John Clement.
[I have to hope that nobody finds a more pernicious problem with the
procedure I used - it can be the basis of judging the value of drug
treatments, public hygiene measures & etc.]
Finally, I should mention that Ludwik's objection that the two samples are
significantly different at a rather low confidence level for chance effects,
is not in conflict with my assertion that the samples are significantly
different at the 1% level.
So for my part, I expect that Jack's statistical method though correct
for its own assumptions, gives misleading results here.
By contrast, I suggest that Ludwik's calculation,
flawed as it may be in my view, is in the appropriate direction.
Which just goes to show, as Twain wrote, "there are lies, damned lies..."
Sincerely
Brian W
At 08:30 PM 1/1/02, Ludwik wrote:
... The
expected standard error of the differences between m1 and m2
(assuming many samples of the same size were collected) ,
according to a textbook (David Moore, Standard Statistics), is:
SE=sqrt( s1^2/25 + S2^2/25) = sqrt (4/25 + 1.96/25)=0.488
This leads to the two-sample statistics t = (m1-m2)/SE=6.14.
....
Ludwik Kowalski
**********************************************
Jack Uretsky wrote:
...
the sigma of the distribution is an unknown quantity, and one
should redo the t-test taking that fact into account. Using his
numbers I find that the probability that the two means are the
same is about 10% - somewhat less than the 30% of my previous
rough estimate, but far larger than Brian's1%.
The t-test for the difference of two means when the s.d. of the
underlying distribution is the unknown is described in Hogg & Craig,
Section 6.4. Of course, all this assumes that the samples consist
of normally distributed random variable, an assumption that is
almost certainly untrue.
......
[BW]
I take twenty five samples from a pile and find the mean is 7
and the standard deviation is 3
I take twenty five samples from a processed pile and find the mean is 10
and the standard deviation is 3.
Am I justified in concluding the piles are significantly different using
Clement's statistic, effect size = 1??
I casually assume the standard error of the difference of the sample
means is sqrt[(3/5)^2 + (3/5)^2] = 0.849 or ~ 0.85
t statistic = (10-7)/0.85 = 3.53
Degrees of Freedom = 25 -1 + 25 -1 = 48
A table gives significance at 1% level for 48 D of F as 2.68
I conclude there is a significant difference at the 1% level.
Brian Whatcott
Altus OK Eureka!