Here are a few random thoughts about using standardized exams for evaluation
of students, faculty, and programs.
(1) Students do benefit from practice exams. This is especially true if
they are not familiar with multiple choice exams. In a small college like
mine we give mostly essay questions or computational problems where the work
is graded in addition to the final answer. Students are not used to having
the total problem credit determined by one mark on the answer sheet.
Students don't automatically possess skills of looking for flaws in the
"distracter answers," and will choose the first answer that seems reasonable
without seeing that a different answer is actually better or more correct.
They also need to be coached about ruling out a couple wrong answers then
choosing between the remaining.
For this type of practice they don't need the same test they are going to
take, or even retired versions of the same test. Practicing with old GRE
exams when studying for the Major Field Test would be fine.
(2) Having said the above, it would be helpful for professors and department
chairs to have access to some of the same exams (even if retired) that the
students will take because it allows us to see the scope and complexity of
the questions. Unfortunately, these are not always available. Sometimes
this is because the exam is too new and there aren't any "retired" editions.
Sometimes I get cynical and think exam copies are not available because the
writers don't want us to find the flaws. I once evaluated an exam intended
to test the science preparation of elementary school teachers. In my
opinion about half the questions were outright wrong in wording, or
misleading, or had more than one correct answer, or no correct answer.
Indeed, the more science you knew, the more likely you would do poorly
because it was often hard to guess what the writers had in mind. Sometimes
viewing the question very simplistically I could see what the writer was
after, even though I knew it was wrong or incomplete. Other times I had no
clue what the writer was trying to assess... it just made no sense.
Example: Consider a cold front marked on a map. One end of the line is in
Wisconsin. From there the line curves south and west and ends in Colorado.
The question asks, "Which direction is this cold front moving?" The
possible answers are north, south, east, and west. Gee, let me think... in
Wisconsin it is mostly moving east. In Colorado it is mostly moving south.
In Nebraska it is moving southeast, and that might be the "average
direction" although it wasn't one of the possible answers.
East was the "correct" answer, which either implies we were supposed to
imagine we are in Wisconsin, or it implies the writer was trying to see if
students would view weather patterns as moving west-to-east across the US
(which isn't really true even though a lot of people view it that way).
If exams are full of this type of question, what are we testing?
(3) Our future teachers have to take a "Praxis II" exam in their teaching
area. None of us who teach teachers have any idea about the true nature of
this exam because back issues are not available. Our college has said it
would pay the registration fee for any of us (professors) who want to take
it. We can't keep copies, and we can't even make written notes, but we can
make mental notes of what the questions were like and what the overall
coverage of the exam was like. Few professors have done this yet, but some
have.
I have mixed feeling about this. On one hand, knowledge about the test
could lead us to teach to the test, and I do not want to do that. If many
professors took the test it could skew the norms. One the other hand, there
are a couple good things that could happen. (a) If I actually take the test
and get scored on it I will have a ruler to judge the students. If I only
get 70% I might not feel bad about students who get 50%. (b) I might decide
the test is a poor test for what we want to evaluate. (c) I might decide
our program needs improvement in a particular area.
Of course (b) and (c) could be accomplished with retired exams if available
and assuming the current exam is similar to the retired exam.
(4) In Ohio, public school students and school districts are judged by
proficiency exams taken at particular intervals (such as every other year).
Results are published in the newspapers for school districts and grade
levels. The school can receive sanctions for poor scores.
For the "off years" our local school also has the students take the tests
(for their class level) even though these are not required by the state.
The school district uses the off year exams to assess "at risk" students and
also teacher performance. The latter use has led teachers to "teach to the
test."
The on-year test has high security. If a teacher sees the test in advance
and teaches specifically to the test, he can be fired and also prosecuted...
it is simply illegal. But the off-year tests are not secure. Some teachers
were actually using copies of the current test in the classroom and teaching
specific answers to specific questions, and doing this in lieu of regular
lessons. This shows how much pressure is on these teachers to have their
students perform well on these proficiency tests. Last year several
teachers were caught doing this and were disciplined. My wife did not teach
to the test, is opposed to it, but as a union officer had to support
teachers who did teach to the test because there were no rules that said
they couldn't do it (in off years). It was a real mess.
My overall opinion...
It's a good thing to assess our programs and our students. But this is way
out of hand. We are placing way too much emphasis on standardized tests for
evaluation of students, teachers, and programs.
Michael D. Edmiston, Ph.D. Phone/voice-mail: 419-358-3270
Professor of Chemistry & Physics FAX: 419-358-3323
Chairman, Science Department E-Mail edmiston@bluffton.edu
Bluffton College
280 West College Avenue
Bluffton, OH 45817