2. The length (19 K) of this post. Please DON'T HIT THE REPLY BUTTON
and inflict it yet again on suffering subscribers.
In his AP-Physics post of 31 Jan 2003 05:59:28-0800 titled "FCI
Review Program," Mitch Jordan wrote (slightly edited - my CAPS):
"Rick [Tarara] - While I applaud your development of the lab modules
and releasing them to the physics community, I MUST PROTEST
VEHEMENTLY TO YOUR FCI STUDY GUIDE. In my opinion the problems
should not be verbatim from the test. Can you change the questions
to cover the same concept of a problem without using it word for
word? Your graphs and formulas would remain the same but would
require the students to understand the concept being tested by the
FCI, not memorizing the specific questions. FOR ALL TEACHERS PLANNING
ON USING THIS FCI REVIEW, I ASK YOU PLEASE DON'T. Use it as a guide
to develop your own conceptual problems. IT TOOK MANY YEARS AND MAN
HOURS TO CREATE THIS TEST AND IT MAKES NO SENSE TO THROW IT OUT. Its
validity and reliability are well proven and it is the standard in
the physics community."
In his response of 31 Jan 2003 10:56:23-0500, Tarara wrote (CAPS in
the original):
"The purpose of this 'review' is for instructors who go over the test
with students AFTER they have taken it. Such is a legitimate use of
the FCI _as_it_was_made_available_ by the authors . . . . the
publishing of the test and its release was NOT restricted to research
applications -- something that we've hashed out on the PhysLrnr list
over the years . . .[PhysLrnR Archives (2003)]. . . . As a research
tool, the test IS openly published -- at least the 1992 version --
and is now over 10 years old. The test IS a good assessment tool for
some basic Newtonian concepts, and as such can be of use to
individual instructors. It is a CLASSIC, but just that."
Statements regarding use of the FCI that appear in the Hestenes et
al. (1992) paper are:
p. 142: "A copy of the instrument the 'Force Concept Inventory,' is
included for teachers to use in any way they see fit."
p. 150: "Like its predecessor, the 'Mechanics Diagnostic'. . .
[Halloun & Hestenes (1985 a,b)]. . . the 'Force Concept Inventory'
can be used for both instructional and research purposes. . .
(diagnostic tool, evaluating instruction, and placement exam." [But
Henderson (2002) concludes that "the FCI is NOT appropriate for use
as a placement test."]
p. 151: "The analysis of the 'Force Concept Inventory' and the exam
itself are given in the following pages. They may be reproduced for
classroom use."
As far as I know, Hestenes et al. never envisaged that physics
teachers such as Tarara would effectively broadcast the FCI to ALL
students by placing it on the web. Such web publication undermines
the effectiveness of the FCI, not only for the
physics-education-research community that Tarara advises [PhysLrnR
Archives (2003)], BUT ALSO FOR INDIVIDUAL INSTRUCTORS SUCH AS MITCH
JORDAN WHO WISH TO USE THE FCI IN FORMATIVE ASSESSMENT OF THEIR OWN
TEACHING.
I have repeatedly emphasized the need for the confidentiality of
standardized diagnostic tests [Hake (1998a, 1998b, 2001, 2002),
PhysLrnR archives (2003)]. In Hake (1998a) I wrote:
"It is unfortunate that the national-assessment value . . .(I should have added
"and the formative assessment value for individual teachers") . . .
of arduously constructed and validated standardized tests such as the
FCI and the MB . . [Hestenes & Wells (1992)]. . . is gradually being
eroded by distribution of answers to students at some institutions. .
. . . At Indiana, the FCI test is always given and referred to as a
"diagnostic mechanics exam" in an attempt to shield . . .[Hestenes et
al. (1992)]. . . We collect ALL pre-and posttests from students and
none is returned. The pre- and post-test scores are posted by ID, but
questions and answers are neither posted, disseminated, NOR SHOWN AS
COMPUTER ANIMATIONS. After the posttest, instructors are quite
willing to discuss FCI/MB questions privately with any student, but
answer keys are not posted. BECAUSE THERE ARE MANY SOURCES. . .[Hake
(1998b - footnote 66)] . . . OF GOOD CONCEPTUAL QUESTIONS, THERE IS
LITTLE NEED TO DRAW ON THE STANDARDIZED TESTS FOR QUESTIONS TO BE
USED FOR ORDINARY CLASS DISCUSSION AND TESTING. Indiana students
understand that the FCI must be treated just as the MCAT, and there
is little dissatisfaction. BECAUSE OF THE ABOVE MENTIONED DISPERSAL
OF ANSWERS AT SOME INSTITUTIONS, AND THE FACT THAT THE FCI AND MB
TESTS WERE PUBLISHED IN THE OPEN LITERATURE, THEIR USEFUL LIVES MAY
NOT EXTEND FOR MORE THAN ANOTHER YEAR. New and better tests (treated
with the confidentially of the MCAT) are sorely needed in time for a
calibration against the original or revised FCI."
Footnote 66 of the sadly unpublished Hake (1998b) (PER HAS NO
ARCHIVAL JOURNAL!) is (see that online article for the references -
they now need updating, I have updated the URL's below):
Other useful problem sources are: (j) A.B. Arons, ref. 43d; (k) R.E.
Gibbs, "Qualitative Problems for Introductory Physics" (Kendall Hunt,
1990); (l) C.W. Camp and J.J. Clement, "Preconceptions in Mechanics"
(Kendall Hunt, 1994); (m) L.C. Epstein, "Thinking Physics" (Insight
Press, 1990); (n) L. Nedelsky, "Science Teaching and Testing"
(Harcourt, Brace, and World, 1965); (o) H. R. Crane, "Problems for
Introductory Physics," Phys. Teach. 7, 371-378 (1969); ibid. 8,
182-187 (1970); "Better Teaching with Better Problems and Exams"
Phys. Today 22(3), 134-135 (1969); (p) F. J. Blatt, "Principles of
Physics" (Allyn and Bacon, 3rd ed., 1989), excellent
conceptually-oriented multiple-choice questions at the end of each
chapter; (q) C. E. Swartz, "Used Math" (AAPT, 1993); C.E. Swartz and
T. Miner, "Teaching Introductory Physics: A Sourcebook" (AIP Press,
1997), chap. 1; (r) Univ. of Maryland, "Physics Problems from the UMD
PERG" at <http://www.physics.umd.edu/perg/problems.htm>; (s) E.
Kashy, S.J. Gaff, N.H. Pawley, W.L. Stretch, S.L. Wolfe, D.J.
Morrissey, Y. Tsai, "Conceptual questions in computer-assisted
assignments," Am. J. Phys. 63, 1000-1005(1995).
REFERENCES
Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory
physics courses," Am. J. Phys. 66: 64-74; online as ref. 24 at
<http://www.physics.indiana.edu/~hake>.
Hake, R.R. 1998b. "Interactive-engagement methods in introductory
mechanics courses," online as ref. 25 at
<http://www.physics.indiana.edu/~hake>, a crucial companion paper to
Hake (1998a): average pre/post test scores, standard deviations,
instructional methods, materials used, institutions, and instructors
for each of the survey courses of Hake (1998a) are tabulated and
referenced. In addition the paper includes: (a) case histories for
the seven IE courses of Hake (1998a) whose effectiveness as gauged by
pre-to-post test gains was close to those of T courses, (b) advice
for implementing IE methods, and (c) suggestions for further research.
Hake, R.R. 2000."What Can We Learn from the Biologists About
Research, Development, and Change in Undergraduate Education?" AAPT
Announcer 29(4), 99 (1999); available on the web as ref. 7 at
<http://www.physics.indiana.edu/~hake>. THE POTENTIAL OF THE WWW AS A
MECHANISM FOR PROMOTING INTERDISCIPLINARY SYNERGY IN EDUCATION REFORM
IS EMPHASIZED.
Hake, R.R. 2001. "Suggestions for Administering and Reporting
Pre/Post Diagnostic Tests"; online as ref. 14 at
<http://www.physics.indiana.edu/~hake>.
Hake, R.R. 2002. "Assessment of Physics Teaching Methods,"
Proceedings of the UNESCO-ASPEN Workshop on Active Learning in
Physics, Univ. of Peradeniya, Sri Lanka, 2-4 Dec. 2002; also online
as ref. 29 at <http://www.physics.indiana.edu/~hake/>: "Because of
the almost unavoidable slow diffusion of test questions and answers
to student files, replace each Diagnostic Test at approximately 5- or
10-year intervals, such that it can be meaningfully calibrated
against the previous test(s). [SO FAR THIS HAS NOT BEEN DONE FOR THE
NOW OVERUSED 1992/95 VERSIONS OF THE FCI; IN MY OPINION, AS TIME GOES
ON, RESEARCH RESULTS BASED ON THE 1992/95 FCI WILL BECOME MORE AND
MORE DOUBTFUL.]"
Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of
college physics students." Am. J. Phys. 53: 1043-1055; online at
<http://modeling.asu.edu/R&E/Research.html>.
Halloun, I., R.R. Hake, E.P Mosca, D. Hestenes. 1995. Force Concept
Inventory (Revised, 1995); online (password protected) at
<http://modeling.asu.edu/R&E/Research.html>. (Available in English,
Spanish, German, Malaysian, Chinese, Finnish, French, Turkish, and
Swedish.)
Henderson, C. 2002. "Common Concerns About the Force Concept
Inventory," 40(9):542-547; online at
<http://ojps.aip.org/dbt/dbt.jsp?KEY=PHTEAH&Volume=40&Issue=9>. [I
think Henderson's conclusion that "There is little difference between
FCI scores when the test is given graded vs ungraded" must be treated
with caution. If the posttest is not graded then instructors may
attempt to exclude "no-effort" posttest scores from their analysis.
In Hake (2002) I wrote:
"Give the posttest UNANNOUNCED near the final day of classes, and
preferably as part of the final EXAM WITH SIGNIFICANT COURSE CREDIT
GIVEN FOR POSTTEST PERFORMANCE. Giving course credit probably
motivates students to take the posttest more seriously and thereby
demonstrate more adequately their understanding, especially if time
devoted to the posttest subtracts from time spent on the rest of the
final exam. IF NO GRADE CREDIT IS GIVEN FOR PERFORMANCE ON THE
POSTTEST then selective removal of "no-effort" tests [see e.g.,
Henderson (2002), Mallinckrodt (2001)] by different investigators
with different no-effort criteria will lead to uncertainty in
comparing normalized gains of different courses."
Hestenes, D., M. Wells, & G. Swackhamer. 1992. "Force Concept
Inventory," Phys. Teach. 30: 141-158). For the 1995 update see
Halloun et al. (1995).
PhysLrnR Archives. 2003. For web ethnographers who wish to uncover
some of the PhysLrnR posts relevant to FCI confidentiality (or lack
thereof) go to the archive search engine
<http://listserv.boisestate.edu/cgi-bin/wa?S1=physlrnr&X=-> and type
into the "Subject slot" (without the quotes): "On-line FCI" (16
hits); "FCI as a measure" (11 hits); "Use of the FCI" (4 hits); "What
Does the FCI Tell Us?" (15 hits); "need for password-protected FCI"
(43 hits); "What's the point of teaching intro physics" (5 hits). For
the (usually conflicting) Tarara and Hake posts on these subjects,
repeat the above but enter (without the quotes) either "Tarara" or
"Hake" into the "author slot." For a broader survey of FCI related
PhysLrnR posts type "FCI" into the "Search for" slot for 562 hits.
(All hit numbers are as of 2 Feb 2003 3:10:00-0800.)
PhysLrnR is almost unique among discussion list in prohibiting access
to its archives by non-subscribers (purportedly to protect
subscribers from being spammed by non-subscribers). But it takes only
a few minutes to subscribe and then unsubscribe by following the
simple directions at
<http://listserv.boisestate.edu/archives/physlrnr.html> / "Join or
leave the list (or change settings)" where "/" means "click on."
Rather than unsubscribe after using the archives, it's easier to
subscribe in the first place using the "NOMAIL" option under
"Miscellaneous." Then, as a subscriber, you may access the archives
and/or post messages at any time, while receiving NO MAIL from the
list! [This strategy allows one to monitor many LISTSERV lists (e.g.,
Phys-L, Physhare, Physoc, Biopi-L, Chemed-L, AERA-D, POD) while
retaining a modicum of sanity and hard-drive space.]