In his PhysLrnR post of 18 May 2005, Jerry Epstein wrote [bracketed
by lines "EEEEEEEE. . . ."]:
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
We need a pre-test and a post-test. The pre-test is given I suppose
in the first or second class of the semester (in class time?). Are
students told that it will not count towards their final grade?
Do people give the post-test as part of the final exam? If so, does
it matter that the students were told at the pre-test that it does
not count toward the grade?
If the post-test is given in class-time, how do people get students
to come? Is there a significant fraction who took the pre-test but do
not show up for the post-test (what we are finding from two of the
pilot test sites), and if so, is it a self-selecting sample?
Do they know that they are going to be taking a post-test during that time?
Is there any pattern, in terms of scores on the pre-test, to those
who do not take the post-test.
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
Great minds run in the same direction. Similar questions were asked
by Jack Bishop (2005) in an AERA-D post of 17 Feb 2005, and answered
by me in a post of 20 Feb 2005 [Hake (2005)].
In "Assessment of Physics Teaching Methods" [Hake (2002)], I
summarized pre/post test administration and reporting procedures that
have been proven effective during two decades of pre/post testing in
introductory mechanics courses.
Among the eleven suggestions for administering pre/post tests, those
that appear especially relevant to Jerry's concerns are: [bracketed
by lines
"HHHHHHH . . . . "; see that article for the references]:
HHHHHHHHHHHHHHHHHHHHHHHHHHHHH
3. Do NOT allow students to take either the pretest or the posttest
anonymously, because nonanonymity allows:
(a) Proper incentive for students to exert effort on the test.
(b) Analysis of "matched" pre/post test data, i.e., obtaining the
average class pretest score by counting only the scores of those
students who took the posttest, and thus allowing a more rigorous
calculation of the class average normalized gain <g>. [For a
discussion of "matched data" see Hake (2002a); and also Hake (1998b),
Table I, footnote "c" on page 7.]
(c) Knowledge of the normalized gain g for each single student in the
class, thus allowing a calculation of the average of the
single-student gains: "g-ave." [See Hake (1998a - Sec. V and also
footnote #46), for a discussion of systematic and random errors in
pre/post testing and the connection between low correlation of single
students g's with their pretest scores, and the small difference
between values of g-ave and <g>.]
(d) Analyses of single student normalized or actual gains in terms of
single-student characteristics or performance on other tests.
(e) Calculation of the correlation of individual student g's with
their pretest scores.
4. If possible, give the pretest on the FIRST day of class. Take
great care that all question sheets and answer sheets are returned
and verify such return by counting those given out and those
returned. **In order to promote serious effort on the pretest by
students, explain that although their scores on the pretest will NOT
count towards the course grade, their scores will be confidentially
returned to them and will assist both themselves and their
instructors to know the degree and type of effort required for them
to understand mechanics.**
5. Give the posttest UNANNOUNCED near the final day of classes, and
preferably as part of the final exam with significant course credit
given for posttest performance. Giving course credit probably
motivates students to take the posttest more seriously and thereby
demonstrate more adequately their understanding, especially if time
devoted to the posttest subtracts from time spent on the rest of the
final exam. *If no grade credit is given for performance on the
posttest then* selective removal of "no-effort" tests [see e.g.,
Henderson (2002), Mallinckrodt (2001)] by different investigators
with different no-effort criteria will lead to uncertainty in
comparing
normalized gains of different courses.