Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

[Phys-L] Re: Math SAT



At 12:09 -0700 8/31/05, Shapiro, Mark wrote:

The problem is that questions change from year to year, but I think the
test makers are supposed to "normalize" the tests so that they are
comparable from year to year.

I can testify from personal experience that the CB does very
carefully monitor the questions they use so that the tests are very
closely standardized from year to year. While there are occasional
questions that fall through one crack or another and make their way
into various publications as the howlers we see every so often, they
are few and far between, and the statistics they keep on each and
every question allows them to make sure that no test will differ
significantly from the others from year to year. Of course, the
student population does vary with time, depending on who takes the
test (in the beginning it was only the top students, but now it is
more like nearly every student, whether they are college-bound or
not), and the preparation of those students also varies (some have
had prep courses, for what they are worth, and some have not; some
have taken the test several times; some have had better school
preparation than others, etc.), so scores will vary from year to
year, but the changes are more likely to be in the students than in
the tests.

The tests are normalized, but not in the sense that they set the
median reported score (500) at the median achieved score, but that
they are calculated to achieve the same median score each year,
assuming that the students are equivalent from year to year.

A change of 2 points may be statistically significant and it may mean
that students are doing slightly better on essentially equivalent
tests, but I wonder if it is educationally significant.

Hugh
--

Hugh Haskell
<mailto:haskell@ncssm.edu>
<mailto:hhaskell@mindspring.com>

(919) 467-7610

Never ask someone what computer they use. If they use a Mac, they
will tell you. If not, why embarrass them?
--Douglas Adams
******************************************************