Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: science for all?



Randomly generated samples are just that. What do you conclude
about the populations in question? Or are you saying that you cannot
reach conclusions?
After we're done discussing the samples, I'll tell you how they
were generated?
Regards,
Jack


On Tue, 8 Jan 2002, Brian Whatcott wrote:

Jack,
I've probably said all that I can that is useful about your numbers.
None of them would lead a person to suppose that the 'after' distribution
is different from the 'before'. I mentioned your samples one and six
in particular because they don't seem to be consonant with the way
a physics test is marked - on a scale of 1-10, 1-30, 1-100, these
samples look unusual.

I'm still waiting for you to explain the basis of my monstrous error - or
have you concluded the error was yours yet?

Brian W

At 10:35 AM 1/8/02, you wrote:
Very good, so far. But what more information do you need about
the samples in cases 1 and 6? These are real, randomly generated
samples. My question is, what can you tell me about the differences
of the populations from which the samples are drawn? Are you saying that
mean and SD are not enough? What more do you need?
Also, what would you say about the difference between the second
population in case 3 and the first population in case 6?
Regards,
Jack

On Mon, 7 Jan 2002, Brian Whatcott wrote:

At 04:03 PM 1/6/02, Jack Uretsky wrote:

... here's a challenge to John and Brian:

I have set up a program to generate random pre- and post- test
scores for classes of approximately 100 students. In the following six
cases (done in sequence with no selection), some of the pre- and post-
tests were taken from identical pre- and post- distributions (there was
no change in "student understanding" between the tests), and some from
cases where the statistical properties of the population changed
between the tests. Tell me which cases are which; after you have done
so, I'll tell you what the statistical properties of the populations from
which the samples were drawn actually were:
pre-test post-test
case mean SD mean SD n 'SE Delta m'/t/dof
1 50.9 333 5.6 4.9 100 33.3 t=1.36 dof 198

2 7.0 15 5.1 1.6 101 1.5 t=1.27
dof 200
3 5.3 3.9 5.0 1.2 100 0.41
t=0.74 dof 198
4 6.4 9.2 6.1 7.3 101 1.17
t=0.26 dof 200
5 9.7 26 8.2 24 99 3.56 t=0.48
dof 196


6 8.7 18 51 432 100 43.2 t=0.98 dof 198

I looked over the numbers looking for data gathering errors first:

Cases one and six seem to have unusual ratios of SD/mean for
school test results so I set them aside for follow up and verification.

Cases 2 and 5 are mildly in the same direction, but I processed them
anyway. Though the sample size is large enough not to need the small
sample size correction offered by the t statistic, I used it anyway,
as harmless.

In cases 2-5, I find the t value is too small to allow one to reject
the null
hypothesis that the pre and post are significantly different.
(For large samples from a reasonably normal population, the figure is 1.96
for the probability of wrongly rejecting the null at 5%)

As before, I used this formulation:
Standard error of the difference of the sample means =
sqrt [ SD1 squared /sample size + SD2 squared /sample size ]

t statistic = (diff of sample means) / (Std err of diff of sample means)

degrees of freedom = (sample size 1 - 1) + (sample size 2 -1)

I referred to a table from Billy Turney's and George Robb's
'Statistical Methods for Behavioral Science', IEP lifted from
Fisher & Yates "Statistical Tables..." Oliver & Boyd, Edinburgh

So Jack:
are the figures I used for case 1 and 6 the numbers you
intended to represent scool test grades in these cases?

The overall null conclusion - 'no difference', implies that your
processing of the trial population was either not dramatic enough
or not uniform enough to show up on this statistic.
You seem to have provided sample pairs with a SE of the
difference of 1/4 , 1/2, 3/4, 1 , 1 1/4 etc. Not enough!

Regards

Brian W

Brian Whatcott
Altus OK Eureka!


--
"But as much as I love and respect you, I will beat you and I will kill
you, because that is what I must do. Tonight it is only you and me, fish.
It is your strength against my intelligence. It is a veritable potpourri
of metaphor, every nuance of which is fraught with meaning."
Greg Nagan from "The Old Man and the Sea" in
<The 5-MINUTE ILIAD and Other Classics>