Re: [Phys-L] true or false questions

[John Denker <jsd@av8n.com>]

When discussing T/F, the devil is in the details.  Consider the
following dilemma:

 ++ T/F questions wouldn't be so bad if the test-takers showed
  their work, and were graded thereon.

 -- I assume the selling point for T/F exams is that they are
  easy to grade.

This dilemma is crucial to any discussion of T/F questions, as
you can see from the following:  Several of the Millenium Prize
problems are true-false propositions:
  https://en.wikipedia.org/wiki/Millennium_Prize_Problems
You can win a million dollars apiece for answering them.
However, that does not mean you have a 50/50 shot, because you
have to show the work.  Not so easy.


On 05/09/2017 11:12 AM, Alex. F. Burr via Phys-l wrote:

> There has been here some discussion of T or F questions.

Yes.

> A good example of such questions which require real physics thinking (not  
> equation memorization) appeared in the recent Physics Teacher in the column 

Those can be considered relevant examples, but I wouldn't
call them "good" examples.

For starters, the final two out of six require no "real physics
thinking" AFAICT, and indeed serve no legitimate purpose AFAICT.

  They serve as a way to telegraph the desired answers,
  so that a student can get a good score without doing
  any physics at all.

  This is particularly blatant when we get to the sixth
  question, because if you think the first five are false,
  then the sixth becomes the Epimenides paradox.  It is
  true if and only if it is false.

I reckon question "D" is true, but that does not require much
"real physics thinking".  It requires recalling the definition,
and knowing what the words mean, but still the answer is
obtained in a single step, or not at all.  In contrast, in
my world, real physics questions require multi-step reasoning,
combining multiple ideas in creative ways.

This leaves us with questions A, B, and C, which are even
more problematic.

Reason #132 why I don't like true false questions is that they
are exceedingly vulnerable to questions of interpretation:
 -- Does T mean /reliably/ true?
 -- Does F mean /reliably/ false?
 -- What if it's only "usually" true?
 -- What if it's a 50/50 proposition?

This is relevant here, because skydiving is very much a
three-dimensional sport.  Several details in the diagram at
  http://aapt.scitation.org/doi/pdf/10.1119/1.4981029
suggest that Nellie is tracking sideways, possibly at quite a
high rate.
  https://en.wikipedia.org/wiki/Tracking_(skydiving)

I categorize problems like this as being ESP-complete.  That
is, the key to answering them is to read the mind of whoever
composed the question.

In this case, probably the best way to proceed is to ignore
the plain language of the question, where it says Nellie knows
something about the physics of the sport, and to assume that
the questioner does not.  This is what happens when we have
cartoonists teaching physics, without bothering to become the
least bit familiar with the subject matter.

So maybe we should disregard the diagram and disregard the
plain language of the question, and assume the motion is
one-dimensional.  When it talks about decreasing force,
velocity, or acceleration, perhaps it means the vertical
component thereof.  In this case, A, B, and C are true or
false, depending on an arbitrary choice of coordinate system.

Or maybe not.  Maybe we should assume that Nellie really does
know what she's doing, and debug other parts of the questions.
In particular, when it talks about decreasing force, velocity,
or acceleration, maybe we should interpret that in terms of
the /magnitude/ of a vector quantity, in which case A, B, and
C are true about half the time and false about half the time
in any given jump, independent of the coordinate system.

  Hint:  Show that the derivative of the speed is V•F/m, with
  no obvious connection to the rate-of-change of F.

Bottom line:  There is no way to answer these questions using
physics.  The more you think about the physics, the worse off
you are. I recommend using E and F to game the system, so you
can write down the answers without thinking about the physics
at all.

Suppose a teacher is trying to interpret the results of such
a test.  What would it mean if some student got one or more
questions "wrong"?  I have no idea, since there might be
perfectly good physics reasons for the unconventional answers.
Since the TPT article does not discuss this part of the job,
it must be considered very incomplete.

===================

There are "some" real-world jobs that never require multi-step
reasoning and instead revolve around true/false questions such
as "Would you like fries with that?"

However, the students who sign up for physics probably are
hoping for something better than that.

I'm not saying it is impossible to design decent true/false
questions ... but it's not easy, and the Nellie questions
should be considered object lessons, not good examples.

I understand the attraction of something that is super-easy
to grade.  Oftentimes that's a plain old necessity.  However,
let's not make a virtue of necessity.  We should be on the
lookout for assessments that are reasonably easy to grade
but not quite so problematic in terms of undermining the
purpose of the course.  This is mostly an unsolved problem,
but I'm not ready to surrender just yet.

Trade-and-grade is one option to consider.  It allows long
form answers, and allows /showing the work/ to be part of
the grade, as it should be.  This comes at a cost, since it
cuts into class time, but the alternatives impose terrible
costs in other ways.