Re: Assessment; evaluation of GRE scores

[Jim Peters <Jim.Peters@HILLSDALE.EDU>]

I must praise what John Denker wrote below.  Is there something we can do
about this situation?

My call to the AIP statistics division was not able to get the US average
physics GRE score.  It is apparently not known.   Apparently they are not
studying this problem.  The only statistics available show that in the years
1993 to 1999 international students studying graduate physics in the US
declined 10.5% while the corresponding US students decreased 40.1%.  US
physics graduate students are less than 50% of the total now.  If the US GRE
average is something like in the 35th percentile then these low scores could
be a factor in discouraging US students from attending graduate school in
physics.  The same could happen unfairly to any student or group of
international students who took the test by the rules.  Low GRE scores lead
to decreased graduate school opportunities.

This also seems unfair to physics teachers who spend years teaching their
students how to think and the general subject of physics, only to have their
students outclassed by students who are given test questions to "learn".

There is one problem on gravity that I saw on the GRE test but no where else
in many years of teaching.  Should I teach my students this problem and
others like it because of the high probability of them seeing these problems
on the GRE or should I teach them general principles and how to think?  I
prefer the latter and I resent the test rewarding the former since I
consider it a lesser accomplishment even aside from its moral implications.

Thus it seems that if physics teachers obtain test materials from previous
test takers and distributed them, the students who have them will do much
better on the GRE physics test.  If you do not do this and others do, then
your students will do worse.  If you use GRE scores for assessment, you will
rate poorer than you deserve. Is there something we can do about this
situation?

Jim Peters
Hillsdale College










-----Original Message-----
From: John S. Denker [mailto:jsd@MONMOUTH.COM]
Sent: Thursday, August 23, 2001 12:32 PM
To: PHYS-L@lists.nau.edu
Subject: Re: Assessment; evaluation of GRE scores


At 09:03 AM 8/23/01 -0400, Jim Peters wrote:

> http://www.pha.jhu.edu/groups/sps/physicsgre.html.

which raises quite a number of interesting issues.  We could discuss it for
a month.

At 11:43 AM 8/23/01 -0400, Michael Edmiston wrote:
> If we abide by the "rules" stipulated by ETS, we faculty members
> are not allowed to look at the exam.

It is bizarre that Michael Edmiston and his students can't benefit from
looking at past exams.  In contrast, according to the URL cited above
         ... in most Chinese physics departments there are books ... about
         an 80-85% chance of having all the questions on the next exam to
         be given.

It seems foolish of the ETS to set "rules" that can't be uniformly
enforced, especially when this gives such a huge advantage to the
rule-breakers.

This is particularly foolish on a test where there is tremendous time
pressure (1.8 minutes per question).  Memorizing the answers to a modest
subset of the questions can greatly increase the time for thinking about
the remaining ones.

Note that it doesn't have to be this way.  For example, the tests designed
by the Federal Aviation Administration are largely immune from this
particular problem, because
   a) they prepare a huge number of questions, typically about 1000, of
which only 50 or 100 are selected to appear on any given test, and
   b) they publish all the questions.

As Mr. Spock would say, this is crude but effective.  It eliminates one
entire category of rule-breaking.  Anybody who wants to memorize the
answers to all 1000 questions is free to do so.

As a corollary, this takes the previously-underground practice of "teaching
to the test" and moves it above ground.

(Of course umpteen other categories of rule-breaking remain.  You can send
in a ringer, you can take extra time, .......)

Anybody who wants to provide a meaningful test, even a pretense of a
meaningful test, needs to do SOMETHING to control the most obvious forms of
rule-breaking.  I'm not fussy about exactly what controls are used, but you
need to do SOMETHING that works.

Testing is important.  Testing is really really important.  It's a shame we
can't have better tests.