Re: Physics Tests

["John S. Denker" <jsd@MONMOUTH.COM>]

At 04:04 PM 11/6/01 -0700, Jim Goff wrote:
> I allow the students a double period (2hrs) to do this.
> most of my students actually finish in about 1.5 hrs.

That sounds right to me.  I've never seen much value in applying time
pressure just for the sake of time pressure.

> I try to adjust the exam
> problems not to be "tricky" but rather straight-forward, but definately
> requiring more than just "plug-and-chug".

That sounds right as far as it goes, but it is not just a straight line
from "tricky" to "plug-and-chug".   It's multi-dimensional.  One can devise
 a) problems that require insight
 b) problems that require understanding the concepts
 c) problems that are long and laborious
 d) problems that are tricky and misleading

Items (a) and (b) make things hard in a good way, item (c) makes them hard
in a bad way, and item (d) is very bad.

> As a test of fairness and
> difficulty, I expect to be able to do the same test in 1/3 to 1/2 the time it
> takes the students (or the bulk of them).

That's the right idea, but you have to be careful:

1) Taking your own test is essential, but it doesn't do 100% of the job.
If the question is ambiguous, open to misinterpretation, you won't detect
it, because _you_ know what you meant.

It really helps to administer the test to your Teaching Assistant(s) in
advance.  Check for misinterpretations, and check the timing.

If you don't have suitable TAs, administer the test to one or two
carefully-chosen students in advance.  Of course you have to really, really
trust them not to give away the questions to their buddies.


2) Depending on the type of questions, using yourself as a timing standard
might not be very informative.  When I was an undergrad, the typical exam
contained a lot of problems that revolved around finding a clever way to
simplify the analysis (maybe symmetry?  maybe 4-vectors?  maybe principle
of virtual work?  maybe virial theorem?  maybe scaling laws?  maybe Green
functions?) ... so once you had the problem figured out, you could write
down the solution as fast as you could write.  On the other side of the
same coin, if you didn't see the clever way to simplify the problem, it
would take you eleventeen hours to solve it using the plug-and-chug method.

This type of question has many advantages, but carries some risks:  If
there is a chance that some of the students have seen the trick before, the
exam degenerates into a test of experience rather than a test of thinking
ability.

In any case, it would be easy to complete such a test in 10 minutes (for a
nominal 2-hour test) with the benefit of 20-20 hindsight;  the only
question is how long it takes them to noodle out the clever line of attack
if they haven't seen it before, and you can't do the time-check by testing
yourself.

=================

Even though it is hard work, the point remains:  It is a really good idea
to vet your tests for ambiguities and timing problems.

====================================================================
Tangentially related:

Some recent British Columbia grade-12 exams are online:

http://www.bced.gov.bc.ca/htbin/p_srch98e?sender=1&subject=Physics&month=All
&year=All&doc=Provincial+Exam

I'm not commenting on these, just pointing out that they exist.