Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: Conceptual test access no longer available



Well. security is one option, but a really clever guy could make it
unnecessary. For example, where numbers are used, they can be replaced at
random so that people who have downloaded the old test can benefit from
the experience of answering the problems without memorizing answers.

When I was a freshman at MIT, living in the dorms, the Juniors made their
class notes, old problems, and tests from their Freshman year available to
us. I alsways worked through old tests the day before one of our tests -
as did a lot of my classmates - and that was just part of the educational
process.

I suspect that the FCI kind of test can be made pretty secure just by
randomizing the order of the questions. Then the student who has
memorized the sequence: bbbdcefab, will immediately give himself away (I
saw this done once in a navy preflight class - with some uproarious
results).
Regards,
Jack


On Thu, 8 Apr 2004, Robert J. Beichner wrote:

Conceptual test users,

In case you are looking for NC State's repository of research-based
conceptual tests, <http://www.ncsu.edu/PER/TestAccess.html> and the
tests it had linked to have been removed from the web. The TestAccess
service to faculty and researchers had remained undetectable and
readily available for 6 or 7 years, but must now be locked up. A
Google search found the following links to the TestAccess page:

1. < http://www1.phys.uu.nl/esera2003/programme/pdf%5C131S.pdf >
2. <http://www.sbfisica.org.br/rbef/Vol21/Num1/v21_103.pdf>
3. <http://ie.sdsmt.edu/IEAssess/Reports/ConceptsRpt.pdf>
4. <http://ie.sdsmt.edu/IEAssess/Literature.html>

There was also a direct link to a copy of the TUG-K. (I'm not putting
it in this e-mail and have already asked the offender to remove the
copyrighted test from his site.) I had considered not listing the
results of my Google search to avoid embarrassing the linkers, but I
then realized that anyone could do their own search anyway.

Once I figure out a more secure way to make things easily available
again, interested people will be able to contact me for access. My
guess is that I'll put the tests on the password-protected SCALE-UP
site. This is a hassle for those not interested in SCALE-UP, giving
them one more password to remember and clogging up our database of
SCALE-UP instructors with irrelevant information. But the tests are
too valuable to leave them in the open for students to find, which
undermines the instruments' validity. (I hired a consultant to develop
the SCALE-UP site and don't have the funds to have another secure site
built nor the inclination to learn how to do it myself.) The test
information page will remain available at
<http://www.ncsu.edu/PER/TestInfo.html>. I will begin working on a
replacement for the TUG-K as soon as I can. It will be several years
before it is ready. (I have no idea whether the new version will again
be available in six languages, since the translators of the original
did their work for free.) In the meantime, researchers using version
2.6 of the TUG-K must recognize the potential threat to external
validity.

For your edification, part of the TestAccess webpage is listed below.
You may want to read the first sentence of link #1 above, where the
authors give a very nice quote from Paul Hewitt, and consider how it
applies to this situation. Sorry for the diatribe, but too many people
have done too much work building assessment instruments to let this
slip by without complaint. To quote from the e-mail signature of one of
the offenders when they originally requested access to the tests, "No
good deed goes unpunished."-Ferengi Rules of Acquisition #285.

From the TestAccess.html page:
-------------
RESTRICTIONS ON USE:

Please do not release this URL to students. Also, DO NOT put a link to
this page on ANY other web page. If you do, search engines will be able
to find it. So far we've been able to make the page accessible to
teachers for several years without Google or Yahoo locating it. A
single link from a publically accessible web page will ruin this
security. I've had e-mails from students in Singapore who were clearly
aware of the title of the TUG-K, but could not locate it on the web.
Their teacher would be giving it to them as a test soon and the
students told me they wanted to "prepare."

Obviously, do not allow students to keep copies of any of these tests.
It takes years of development effort to create and validate a reliable
assessment instrument. If it is released to the public domain, students
will locate it and all that work will be for naught. In fact, you
might not want to use the formal name of the test on the versions you
have students take. None of these tests are to be incorporated into a
web-based question delivery system without adequate security to prevent
printing or other unauthorized access by students. Contact the authors
for more details.
------------------

Robert J. Beichner, Ph.D.
Alumni Distinguished Professor
North Carolina State University
Department of Physics
506D Cox Hall
Raleigh, NC 27695-8202
919-515-7226
http://www.ncsu.edu/PER


--
"Trust me. I have a lot of experience at this."
General Custer's unremembered message to his men,
just before leading them into the Little Big Horn Valley