Chronology | Current Month | Current Thread | Current Date |
[Year List] [Month List (current year)] | [Date Index] [Thread Index] | [Thread Prev] [Thread Next] | [Date Prev] [Date Next] |
Clearly not everything that can be captured by open response
can be captured by multiple choice. Just consider writing an essay.
That is not to say, however, that in most situations -- with
notable exception like writing a composition or identifying
an unknown substance in a lab -- MC items are not as good or
better than an open response. Certainly cheaper.
Over the last 3-4 decades a lot of research was done by ETS,
College Board and the likes. They generally found that
for*similar* kind of knowledge MC is more reliable. They
also couldn't find types of knowledge that don't fall in this
category on most normal tests like AP.
I am traveling and w/o access to references, but they are
relatively easy to find.
Ze'ev
Sent via BlackBerry by AT&T
Issue 8, pp.Can free-response questions be approximated by multiple-choice
equivalents? Shih-Yin Lin and Chandralekha Singh
American Journal of Physics -- August 2013 -- Volume 81,
multiple-guess format.624In answer to your question, here's my five-word review:
Not OK.
Travesty of science.
===============================
Longer version:
Consider the two assertions:
+A) There exist one or more questions (X) such that X can be
+represented
in free-response format AND in multiple-guess format.
+B) For all questions (X), if X can be represented in free-response
format then it can also be represented in
+B is false.
I assume everybody on this list knows that +A is true and
I might go so far as to say that +A is obviously true andbe represented
+B is obviously false.
As a point of formal logic, the negation of +B is:
-B) There exist one or more questions (X) such that X can
in free-response format but not in multiple-guess format.everybody on
which is obviously true. Proof by construction. I hope
this list can come up with relevant examples.general case,
The paper in question can be found via:
http://ajp.aapt.org/resource/1/ajpias/v81/i8/p624_s1
It starts by proving the obvious. It uses two examples to prove
assertion (+A).
It then *appears* to claim that two examples prove the
i.e. to prove assertion (+B). Wow, that's quite a leap, from two*appears* to prove
examples to the general case. I emphasize that it
this, because the English is so non-specific that I cannot be surereasonably
what it is claiming. The key conclusion is:
The findings suggest that research-based MC questions can
explicitly sayThis claim *appears* to apply to all possible questions, but areflect the relative performance of students on the free-response
questions ....
Philadelphia lawyer could argue that the paper doesn't
/what/ the conclusions apply to. Therefore:something is
-- If we are generous, the conclusions apply only to two hand-
selected examples, and the paper is obviously trivial.
-- If the conclusions are meant to apply more generally, the
paper is obviously wrong.
-- In any case, the paper is so badly written that we cannot
tell whether it is trivial or wrong!
==================================
Sometimes people who ought to know better assume that if
published in the peer-reviewed literature, it must be OK. This isthe entire
certainly not true ... especially in the PER literature.
The publication of papers like this reflects badly not just on the
authors, but also on the reviewers, on the journal, and on
field.