Re: Conceptual test access no longer available

[John Clement <clement@HAL-PC.ORG>]

Unfortunately the research based tests are very vulnerable because you can't
just change numbers and the number of questions are limited so it is
possible to memorize all of the answers without having any understanding.
However a random retest two weeks later will reveal that this has happened
as they will revert to the distracters.  The effort needed to make a new
variant of the test is also very large because it has to be tested and
verified against the original test.

Hestenes claims that just working through the FCI will not improve scores
because the distraters are so strong, but I do not entirely believe that.
Keeping the reliability of a research based test is very important.  If all
it took was cleverness, then this type of test would have been invented and
used long ago.

Unfortunately security will not solve the problem that some instructors will
use the test carelessly or even unscrupulously.

John M. Clement
Houston, TX 

> Well. security is one option, but a really clever guy could make it
> unnecessary.  For example, where numbers are used, they can be replaced at
> random so that people who have downloaded the old test can benefit from
> the experience of answering the problems without memorizing answers.
>
> When I was a freshman at MIT, living in the dorms, the Juniors made their
> class notes, old problems, and tests from their Freshman year available to
> us.  I alsways worked through old tests the day before one of our tests -
> as did a lot of my classmates - and that was just part of the educational
> process.
>
> I suspect that the FCI kind of test can be made pretty secure just by
> randomizing the order of the questions.  Then the student who has
> memorized the sequence: bbbdcefab, will immediately give himself away (I
> saw this done once in a navy preflight class - with some uproarious
> results).
>                                 Regards,
>                                         Jack